I've got a site which I submitted to the se's a little while ago. Luckily I've got listed for the main homepage and index pages of my subcats, but not for invidual pages within the vategory which I mst say I spent ages doing seo on.
AS example
Mysite.com listed fine
Mysite.com/directory/index.html listed fine
mysite.com/directory/page1.html not listed at all.
Does anyone know why this is. I thought the when a bot visited your site it followed all your links within your site to soak up the content from all pages to offer the consumer the most previse results.
I;ve got links on all the index pages that go further within the site but I'm baffled by this. Do I need to submit each page or diretory indivudally to achieve my goal. I don't want to oversubmit incase I get banned for "spamming" the se's.
I shouldn't worry about it, google has been doing strange things recently. I keep quite a close eye on the number of pages i've got in the index for my sites and it seems to change constantly.
From my (limited) experience google seems to make one visit, and might take the front page. It will then keep coming back and you can see in your stats that it's making more and more hits on your robots.txt file, but it doesn't show any more pages up in google. Then it might up the number of pages a bit and after a while it will just grab a whole load in one go... my index results jumped from 1 page... then 100 pages.... then 16,000 go figure! At the moment it's fluctuating between 5,000 and 16,000 and i've given up trying to work out what's going on
Linear Mode
