tuxlinux 0 Report post Posted October 27, 2004 Hi.Just a quick question.I was wondering what would the google bot do, when it faces lots of new pages. I opened my forum for google, and i was wondering what will google do when it sees all those new pages.Will it keep crawling till it gets them all?Or will it crawl 2 or 3 levels, and stop? (may be it will continue later)Is there a limit to the depth of Googlebot in new pages, or will it just go endlessly? Have any body tried to monitor this?Thank you very much in advance. Share this post Link to post Share on other sites
ashiezai 0 Report post Posted October 28, 2004 i think that every crawl of google bot is a level deeper ... i cant verify this .. but google bot has been very active lately .. crawling my site at least twice a day .. but the listing of my website on google is still only is the main site .. hmm ... i think it will take some time for all pages to be indexed Share this post Link to post Share on other sites
