Jump to content
xisto Community
Sign in to follow this  
jlhaslip

Robots And Sub-domains Do the Bots see the Sub-domains

Recommended Posts

I guess I have had a site up for a few months now and the Googlebots and others are beginning to find the site, according to the Web Stats at least.My question is: when the Bots travel through and spider a site, do they actually see the sub-domains associated with the main domain? And do the contents of the sub-domains affect the schedule for the return of the Bots in the same way as good Content would affect the return? For instance, if you regularly update a sub-domain and keep adding or changing the content, will the Bots be back quicker or more regularly? Are the domain and the sub-domain inter-related with respect to the Bots?Another question, if I may: similarly, should the Meta-tags for pages other than the Index page by different than the Meta-tags for the Index page? I would think that the Meta-tags for the inside pages could be more specific to the particular page, and if the Meta-tags for the secondary pages are different, will the Bots pick these up and handle the secondary any differently? (assume the robots.txt file does not exclude bots from sub-domains, of course.) :)

Share this post


Link to post
Share on other sites

robots.txt will not do anything to the subdomains. From what I've heard, some bots sees the subdomains as external sites. But after what I've experienced, that's partially wrong. My site is not indexed, and none of the subdomains are. Seems like the domain is banned for some reasons, and that the subdomains are banned too.But bots follows external links too, so if you have links to the subdomains from your main page, I'm sure the subdomains will be crawled too.

Share this post


Link to post
Share on other sites

well yep... i think the robots are supposed to search into your subdomains as soon as you register your domain with the search engines.But another way to do it is to have a refferece in your main page or either in some of your pages registered into your search engines

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×
×
  • Create New...

Important Information

Terms of Use | Privacy Policy | Guidelines | We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.