17 May 2008 Support Center Home Why do I have references to robots.txt in my web stat programs? Solution Many search engines will look for a robots.txt file before spidering your sites content. The robots.txt file contains specific rules a robot/spider should abide by when capturing data. Article Details Article ID: 88 Created On: 02 Aug 2004 09:53 PM This answer was helpful This answer was not helpful User Comments Add a Comment Sharing is good. If you have a comment about this entry, please feel free to share. The comments might be reviewed by our staff, and may require approval before being posted. Questions posted will not be answered. Please submit a Ticket for support requests. Fullname: Email: (Optional) Comments: Back Client Login Billing Email: Password: Remember Me: Search -- Entire Support Site -- Knowledgebase Downloads Article Options Add Comment Print Article PDF Version Email Article Add to Favorites Home | Submit a Ticket | Knowledgebase | News | Downloads Language: English (U.S.)
