Google not spidering?

[scifind]

Hi
This month google has only spidered about 25 pages of my main site while it spidered 1,300 (ish) pages of my second site.

Although there arn't asmany pages on many main site there should still be a couple of hundred (scooter spidered 383)

My main site has dynamic pages with the index.php?one=two&three=4 style links and the second uses Mod_rewrite to give tidy pages.(I havent got mod_rewrite on my main host). Is this likely to be the problem?

[Consultant]

that is very much your problem scifind.

You need to think a little harder about how to get google to spider those dynamic pages, its easy, create a static sitemap (*.htm) with links to each of the dynamic pages, google will munch them up with glee, guaranteed.

If you need a sitemap tool, pm me for the url, its free as well...bonus!!

[scifind]

every dynamically generated page is linked to somewhere.
What I am worried about is the
index.php?one=two&three=four
type pages, I know AV finds them OK but google doesn't seem to bother. Is it something I have to live with untill I can get mod_rewrite going or is there a trick to get round it?

Is a site map the answer if there are already links in place?

I can produce a site map with ease, thanks for the offer though.


thanks

[Consultant]

ok, a sitemap, a static page, with links to dynamic pages.

Google won't spider dynamic links from a dynamic page (unless your PR is above 4), so the links you already have in place (which google isn't spidering) are presumably sitting on dynamic pages.

A sitemap is the answer, G only needs to go one level from a static page to spider anything listed on that page, and can 'back out' from the dynamic page, without getting caught in bad programming, bad linking, dbase errors, etc.

Try it, you'll like it!

(btw, the sitemap needs a link from all the main pages on your site, and needs to include all the main pages on your site to be 100% effective)

[watcher]

All I know is that I have pages like yours spidered by Google. I am not using a specific site map or mod_rewrite.

[scifind]

Aha
So spidering of second level dynamic links may be the pronblem, I will try a site map asap.

Will let you know if I get spidered in the subsequent weeks

[Rich]

Google can index from a dynamic url to another - Just see the number of forum topics that are listed, typically the forum index has a dynamic url.
i.e /viewforum.php?f=1 links to /viewtopic.php?t=4 and both get indexed.

The one thing that does seem to be a no-no for Google is something.php?id=27 - as the name id is often used for session identifiers and Google avoids these as much as it can.

Post what you 'one and 'three' is and I'm sure someone will reply if it looks like it might scare Google off.

[1st Stop]

I have had the same problem with google i think it just don't like php

[uklejon]

I don't have problems with dynamic pages.....keep the URL as short as possible and use mod rewrite if you can.


Google does crawl dynamic pages and quickly......i had over 1000 pages indexed in less than a week last month! But i use a short
URL

page.asp?id=whatever

Consultant, could you post the URL for the site map tool...interested in that.......too hard to do it manually!!!

Take Care

Carol

[drivetowin]

Yes please consultant, please can you post the URL of the site map tool

I would have sent you a PM but you don't seem to have enabled PMs on your account!

[Consultant]

Will let you know if I get spidered in the subsequent weeks

Should be more like a week if google is spidering you heavily at the moment.

post the URL for the site map tool

Xenu

Don't forget to exclude external links when using this tool (especially if you're running it against an affiliate site with thousands of outbound links

[searley]

any chance you could send me the mapping tool? only my site is completely dynamic

rgds