Crawl.googlebot.com how and why ?

[yordan 10]

I had a look at my private forum, and I saw a guest connected. I had a look at his IP address, did a whois, and found that this guest wascrawl-66-249-73-137.googlebot.com How did this googlebot come to my private website ?Is there a way to prevent these bots from crawling my website ?

[hfbvm 0]

just make a robots.txt file.robots.txt will work as you want. You can even disable crawling for specific pages, so you don't have to move your pages into a folder. Something like:

User-agent: *Disallow: /index.phpDisallow: /main.phpDisallow: /inc/Disallow: /img/Disallow: /images/Disallow: /js/Disallow: /css/

And so on and so forth its easy more queries ask me maybe i could help you

[yordan 10]

Seems to be exactly what I want. Thanks, I'll try that way.

[hfbvm 0]

where's my rep for this?

[yordan 10]

where's my rep for this?

Your rep is here : http://forums.xisto.com/topic/96784-topic/?findpost= Can't you really see it ?

[zorba4 0]

where's my rep for this?

I also see your rep, you said :

User-agent: *Disallow: /index.php
Disallow: /main.php
Disallow: /inc/
Disallow: /img/
Disallow: /images/
Disallow: /js/
Disallow: /css/

does your browser have a problem? I use IE7

[yordan 10]

does your browser have a problem? I use IE7

I use Firefox 2, and I see hfbvm's rep. Maybe he started sleeping ?

[zorba4 0]

Maybe he started sleeping ?

Or he forgot refreshing his browser window?

[hfbvm 0]

yes i can now see it sorry for the inconvenience or whatever.I also use firefox.

[aloKNsh 0]

yeah i can also see....and i use safari

[yordan 10]

yeah i can also see....and i use safari

Why safari ? Are you a Mac User ?

[FirefoxRocks 0]

Why safari ? Are you a Mac User ?

You do realize that there is also Safari for Windows. It is very similar to Google Chrome minus some features.
I'm using Firefox 3 on Ubuntu 9,04 at the moment. Just installed it today.

[yordan 10]

I'm using Firefox 3 on Ubuntu 9,04 at the moment. Just installed it today.

Exactly what I said. Linux+windows people use Firefox. Mac people send you mails with .websave files, and they a surprised learning that safari is not your default browser.