Google and script based sites

[Tobman]

Tweet

Hello all.

I've got a puzzler I can't find the answer to.

In the google webmaster advice section, you can find this bullet point:

"# Make sure your web server supports the If-Modified-Since HTTP header. This feature allows your web server to tell Google whether your content has changed since we last crawled your site. Supporting this feature saves you bandwidth and overhead."

Is this something you need to worry about if you're using a script to publish pages, and using mod rewrite to make the urls appear static?

Is there somewhere to learn more about it if it's important?

Thanks

Tom

[Sproates]

I don't believe mod rewrite plays a part in this issue; googlebot will treat a given page the same way whether it's rewritten or not.

The issue is whether you're bothered about saving some bytes of bandwidth. In a nutshell, you can do something like following:

- check for the If-Modified-Since HTTP request header at the start of the script
- If the header is present, determine whether the content you're intending to serve is newer than the date specified in the header
- If the content has not changed, send a 304 HTTP response to indicate the content has not changed.
- If the content has change, send the content as normal (should be 200 HTTP response).

You should also be sending the Last-Modified header in your responses, so that clients can determine what to specify in their If-Last-Modified header when the re-request the page.

This assumes you can actually determine whether the content has changed; you'd need to be storing date and time information alongside your content, whether that's a column in a table holding the content, or a file timestamp, in order to do that.

You can see some example code of how to do this stuff here. If you're not a PHP man, I'd go for googling if-modified-since and last-modified.

Hope that helps.

[Tobman]

Hi Ian,

Thanks very much for the reply. Exactly what I wanted to know. My only concern is with Google really. As long as google can see that the website is being updated that's all I'm worried about. I was just worried that because the pages were created with a script and didn't really 'exist' as such, I was missing something that google needed to see.

Thanks again!

Tom