XML Sitemaps

[Publisher-For-You]

Tweet

I'm finally catching on to XML sitemaps, and recently coded a script that will create and update one for my site.

I included the path to this site map in my robots.txt file, in hopes all the engines will find it.

It's only been 3-4 days, and so far it looks like none of the engines have found the site map, which isn't surprising I guess.

I've linked to the site from some of my other sites, some of which have PR 3-4.

Do you care to share your experience with XML Sitemaps? Did you find that once the search engine bots find your sitemap, all your pages get indexed? I've got 8,000+ pages, most of which can't be reached via clickable links, so it will be interesting to see if the sitemap does the job of getting them indexed.

Any observations most welcome, thanks.

[mallyord]

have you added it to google sitemaps?

[Publisher-For-You]

have you added it to google sitemaps?

Ah, good point. Yes, I neglected to mention that above, but I did login to the Google Webmaster Tools, and told it about my sitemap. I have no idea how long it will take their crawler to find my sitemap, or what will happen after that. We'll see, a learning experience.

[adam_knight]

This should help: Create your Google Sitemap Online - XML Sitemaps Generator

a number of tools to help.

[Publisher-For-You]

This should help: Create your Google Sitemap Online - XML Sitemaps Generator

I regret to report I did buy that script. It finds only 9 (nine) of my 8,000 pages, and the seller declines to give refunds for any reason. $20 down the drain.

I couldn't recommend this seller or product, but I'd guess the script will work for somebody.

Thanks for the good intention behind your suggestion though.

[adam_knight]

I regret to report I did buy that script. It finds only 9 (nine) of my 8,000 pages, and the seller declines to give refunds for any reason. $20 down the drain.

I couldn't recommend this seller or product, but I'd guess the script will work for somebody.

Thanks for the good intention behind your suggestion though.

Don't know why it hasn't worked for you. I've always used the free inbuilt version in the site and it works perfectly (albeit you have to rank your pages manually) Have you coded your site in .php?

[Publisher-For-You]

Don't know why it hasn't worked for you. I've always used the free inbuilt version in the site and it works perfectly (albeit you have to rank your pages manually) Have you coded your site in .php?

I'm a perl nerd myself, don't know any php. The script crawls the site from the web, like any other bot. This is perfect, and necessary, for the free service, because there's no other way to get the data.

But, the paid script is installed on your server. So it really should have the option to read your file directory. I wrote a script to do this myself, and it takes 1 -2 seconds to find the URLs for a site with 8,000+ pages. None of the crawling problems being reported on the support forum.

So, the script you mentioned may work well for someone with a site that can be easily crawled by a bot. That is, if you already have a sitemap, this script can create a sitemap for you. :-)

I don't think the seller is a scammer, but, he has my 20 bucks, and I have nothing, so my enthusiasm for further purchases is, um, underwhelming. :-)

[adam_knight]

Sorry to hear that.

There is this on the site:

"Already purchased?

Please use the link that was sent to your email box right after purchase to download the recent version of generator script (your link doesn't change in time). "

Maybe you need the latest version? Worth a try

[Publisher-For-You]

Maybe you need the latest version? Worth a try

That's a good idea, but I just bought the script a few days ago.

Anyway, I would up coding my own, which works the way I want, and I learned in the process, so all's well that ends well. :-)

Thanks again for your suggestion, I'm sure it would work for some.

[Pete Taylor]

I regret to report I did buy that script. It finds only 9 (nine) of my 8,000 pages, and the seller declines to give refunds for any reason. $20 down the drain.

I couldn't recommend this seller or product, but I'd guess the script will work for somebody.

Thanks for the good intention behind your suggestion though.

Are the 9 pages it finds, linked from your home page?

[hobbsy]

I don't know if you are using Wordpress, but if you are there is a plugin available to create your XML sitemap:

http://wordpress.org/extend/plugins/google-sitemap-generator/

hopefully this might be of use to someone

Is this plugin as good/better/worse than coding it yourself?

[johncash]

You can submit your sitemap to Yahoo and MSN wewbmaster centers also.

You can also ping Google webmaster tools everytime you update your site/sitemap.

I wouldn't rush though.

If you have submitted your sitemap to Google sitemaps, it will be indexed soon. In fact it probably has already. Don't expect to see your sitemap in the search engine results necessarilly. My average time to get a sitemap accepted by Google is less than 24 hours. Check in webmaster tools.

I try to keep my sitemaps out of Google serps - by not linking to them from the site.

Cheers

[johncash]

> You can submit your sitemap to Yahoo and MSN wewbmaster centers also.

Sorry, webmaster! Haha

[monkeynuts84]

Publisher-For-You - does your sitemap script dynamically update and inform Goog of new pages/posts/etc? This is the best way forward. Admittedly, once you first submit your xml sitemap to Goog is does take some time before it references the schema before crawling.

Tip for anyone using Wordpress and the Google XML sitemap plugin - check your site feed regularly, especially if you're on shared hosting. Many of the HSP's limit the memory of the virtual host which means the Sitemap plugin barfs and starts sending malformed URLs to Google. This means Google won't find your posts - all from very recent and bitter experience!!!

[johncash]

Publisher-For-You - does your sitemap script dynamically update and inform Goog of new pages/posts/etc? This is the best way forward. Admittedly, once you first submit your xml sitemap to Goog is does take some time before it references the schema before crawling.

Tip for anyone using Wordpress and the Google XML sitemap plugin - check your site feed regularly, especially if you're on shared hosting. Many of the HSP's limit the memory of the virtual host which means the Sitemap plugin barfs and starts sending malformed URLs to Google. This means Google won't find your posts - all from very recent and bitter experience!!!

Thanks Moneynut - I just checked one of my WP blogs and it has missed the category bit out the url! 2000 broken links! Eeekamouse!