Google

I added lots of new pages to one of my sites about 10 days ago and would like Google to crawl them, but looking through my site's logs I see that Google does appear almost every day to ask for my robots.txt file (which I don't have) but then goes away without doing any crawling.

My question is - is that normal and I just have to be patient for it to get around to crawling my site properly?

I expect it is, but thought I'd check in case there is something I could be doing to speed up the crawling. I have submitted one of my pages to Google hoping that would get it back round to my site, but it's not had that effect yet.

I know that expecting it to be crawled within 10 days is optimistic, it's just that finding that Google does sniff around my site pretty much every day without doing doing any crawling that's made me curious!

My advice is to sit back and wait. If googlebot knows your there then your new pages will be crawled sooner or later.

I see googlebot regularly, but normally it will only visit my index page. If i want certain pages to be crawled quickly i add a temp link from my index page.

I'd also add a robots.txt file. With it, it may crawl your home page regularly, rather than just looking for the text file.

However, i am a newbie to SEO, so others may say this is tripe.