Google spidered my test site!

**m-a-hudson** · 03-02-10

Google spidered my test site despite using a robots.txt disallowing it!

I now have the problem of some links appearing on the test. subdomain, and some on the new site.
Currently, the test. subdomain is dead, so the links go nowhere. Don't want to set up a redirect to the new site, or Google will continue to index the test. subdomain.

I'm assuming, the best course of action, is just to wait until Google pulls the old test. pages out of it's database when it realises they are dead links. Then it should start to index the real ones.

How would you do it?

**bensd1** · 03-02-10

If you add the site to google webmaster control panel you can request for pages to be removed.

**gadget** · 03-02-10

You could password protect the entire site.

**m-a-hudson** · 03-02-10

Thanks to both posts. In the end I went here:
https://www.google.com/webmasters/tools/removals
and entered all the URL's I need ed to remove.

Annoyingly, the hosting company issue a "soft" 404 header for any missing content.
So actually the returning code was 200 (OK), which isn't helpful as far as Google is concerened.
Google requires a 404 code additionally to remove the content from it's index. Had to create a .htaccess file and createa pseudo 404 error code.

Thanks again