Hi Tiger,
Quote:
|
How do I tell this from my stats package
|
If your stats package works by using a hidden image on your pages then it is unlikely that it will include search engine information as the crawlers do not generally request image URLs. If this is the case, you will have to find your raw log files - and where they might be and what state they might be in depends on how/where your hosting is configured and what acess you have to it - can you provide more details?
Once you have got hold of your raw logs, you can identify a bot through its "User-Agent" string - the part of a log entry that normally tells you what web browser is being used by the client accessing your website.
A normal log entry looks something like this:
1.2.3.4 - - [03/Feb/2005:16:11:29 +0000] "GET / HTTP/1.1" 200 134 "" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) Opera 7.54u1 [en]"
The last entry on this log record being the user-agent; which in this case is Opera. A search engine bot on the other hand will identify itself something like:
1.2.3.4 - - [03/Feb/2005:08:10:38 +0000] "GET / HTTP/1.0" 200 1887 "-" "Googlebot/2.1 (+
http://www.google.com/bot.html)"
If you can open your log files into a text editor you can use the search function to look for these tell-tale signs of the search engine robots.
If you're interested in a particular search engine, have a look at the following page:
http://www.robotstxt.org/wc/active/html/index.html
If you browse the record for the search engine you want to look out for it will show you what user-agent string that search engine's robot uses.
Hope this helps!
Linear Mode
