robots.txt and engine rewrite - will both be followed?
I'm trying to stop Google indexing sites that it might construe as a duplicate - I have 'killed' off all the domains that pointed to my main site but there is one other site that is listed quite heavily in all the other search engines - and to save time on continually updating both I use the 'engine rewrite code in the .htaccess file on the second site to send everything to my first site.
so I have placed the robots.txt file
User-agent: *googlebot
Disallow: /
- created it in notepad and then used Aceftp to transfer it into the second site - and uploaded it.
will Google now read the robots.txt file first and ignore the site - or
will it read the htaccess file first and simply move onto my main site.
All I want is Google to only see the main site - will this work.
|