robots.txt
I'm not bothered about restricting any spider's from accessing my sites but everytime they request robots.txt (which doesn't exist just now) I get an error in the logs too.
Can I just put an empty file up as robots.txt or is there some way I can write "welcome, come on in everybody" in robot language :-)
|