iGuest 3 Report post Posted December 31, 2004 I'm getting this error. Whats it all about? I thought you only create robots.txt to tell search crawlers to go away.[error] [client [someones IP]] File does not exist: /home/xcllcx/public_html/robots.txtChris Langsdale Share this post Link to post Share on other sites
iGuest 3 Report post Posted January 2, 2005 A bit more info, please. What exactly do you do, to produce this error? Share this post Link to post Share on other sites
iGuest 3 Report post Posted January 4, 2005 I fixed the error by placing an empty .txt titled "Robots". Thanks anywayChris Share this post Link to post Share on other sites
iGuest 3 Report post Posted January 4, 2005 Yeah I think everyone gets that error. Thats to tell search engines what directories not to crawl. The error doesn't bug me though, so I don't think I'll be doing anything for that anytime soon. Share this post Link to post Share on other sites
iGuest 3 Report post Posted January 7, 2005 Engines can STILL bug you. Google is a powerful hacking tool (believe it or not), and it will also allow users to get access to your robots.txt.Although you will keep the newbies out, the more advanced users will be able to see your robots.txt.There was a guide I read the other day but I forgot link (damnit I never bookmark these sites) to help you completely block access to these "hackers" Share this post Link to post Share on other sites
