midnitesun 0 Report post Posted June 12, 2006 i was wondering how long it took for your sites to be indexed by google and other main search engines, i submitted my sites over 5 days ago and googlebot have been visiting my site everyday since then , i know that it may take upto six months for a new site to be indexed but i have read somewhere about a site which was visted by googlebot only once a month , so am i doing good or is it just normal for googlebot to crawl my site once a day ? so tell me how long it took for your site to be indexed and how often googlebot visited your site Share this post Link to post Share on other sites
Albus Dumbledore 0 Report post Posted June 12, 2006 my site was listed within the first month...i know that, because i was getting hits by people searching for certian things.... so just my answer...it took my site less than a month to be indexed by Google and then i got tons of hits and i think it took my other site a little over or under a month...somewhere around exactly a month Share this post Link to post Share on other sites
Paul 0 Report post Posted June 12, 2006 Submitting your URL on the submit URL page of Google will take longer to index than having a high PageRank link pointing to the page.So the best way to get your 'new' sites indexed is to place a link to the new site from a well established 'old' site that is visited by the Googlebot often. Share this post Link to post Share on other sites
brandice 0 Report post Posted June 12, 2006 Exactly, Paul. I've never submitted any of my sites to google, but I do little things such as making sure I fill in the "website" field in my profile when I sign up at a message board and I usually put a link in my signature if that's allowed. Using Blogger, and having links to my site in my blog is also huge. Share this post Link to post Share on other sites
AlanDS 0 Report post Posted June 13, 2006 You like to be on Top, Don't you? My SEO Story  I started my web with good and cute META tags enhanced for good indexing.  I got listed in as many as 40 Search Engines and got a top ranking too. It was good to feel at top. But Google was not one of the 40.  It could not read META tags. (I would rather say that it doesnt)  I now had two options:  1) I had to wait GoogleBot for indexing of my site 2) I had to submit my sites to Google through thier own site or some other one.  I could not wait and I obviously chose the second option : "submitting my sites to Google through thier own site or some other one"  I made a counter attack (lol)  I flooded submissions at once through many sites. Within a week, it was indexed at the first position. A week later, it was not on top. (Just very annoying)  I just cant make out the process of awarding these positions. Share this post Link to post Share on other sites
midnitesun 0 Report post Posted June 13, 2006 ok i didnt have robots.txt in my root directory before , i know it is unprofessional not to have a robots.txt and some people can send malicious programs identifying as robots and steal sensitive information from your cig/bin or private directories , but what awed me was i used to get hit by googlebot once a day previously but yesterday after i put robots.txt googlebot hit my site 26 times ! , didnt know it was important before Share this post Link to post Share on other sites
Sinari 0 Report post Posted June 13, 2006 What exactly does robot.txt consist of? I need to find a way to get my site out there so I can have more members. I'm really new at this as well so I know little about all the "secrets" of getting my page viewed. Share this post Link to post Share on other sites
icemarle 0 Report post Posted June 13, 2006 I think mine took 2 months or so... I also mass submitted to a lot of other search engines. My site was more of a personal page before. It has since evolved and since my site's topic is rather popular... It got indexed in around a month when it had a make-over. Share this post Link to post Share on other sites
midnitesun 0 Report post Posted June 13, 2006 it is a text file which tells search engines (robots) which pages not to index in their databases , like your cgi/bin and private directories or whatever other files you dont want to be indexed the syntax is like thisUser-agent: * Disallow: /support/orders.html Disallow: /support/technical.html Disallow: /support/helpdesk.html Disallow: /support/index.html where * stands for robots Share this post Link to post Share on other sites
Bkag 0 Report post Posted June 13, 2006 Ok 2 questions 1. how do you know when or if googlebot has been spidering your site?2. I have a robots tag in my code it reads <meta name="robots" content="all">Is that the same as a robots.txt file? Share this post Link to post Share on other sites
brandice 0 Report post Posted June 13, 2006 putting that bit in your meta tags gives robots instructions for just that page. While robots.txt will give a robot instructions for all of the files in your webspace. You can ad lines to tell them which folders not to look in.http://forums.xisto.com/no_longer_exists/ Share this post Link to post Share on other sites
midnitesun 0 Report post Posted June 13, 2006 well i check my web statistics from my control panel awstats and it will show all the statistics for your site with time and is updated every 24 hours and yes what you are reffering to has the same function as robots txt Share this post Link to post Share on other sites
Dragonfly 0 Report post Posted June 13, 2006 For me it usually takes 4 weeks for Google to index my site. I also recommend linking from an old, well established site than submitting in the google search engine first. No matter how many times the googlebot visits your site if you are not linked by any good websites your site will probably end up indexing only the frontpage and sometimes when the site isn't very useful for site visitors or a sort of a mirror site google doesn't want to even index the frontpage. Share this post Link to post Share on other sites