New Google Tag - Expires After - Usefull for Promotions etc

[tbp]

Read this morning that Google have introduced a new Meta tag, telling it to stop indexing content after a certain date. Usefull for discount sites for example, so you can stop pages from being indexed once a discount code has expired.

The new tag has the format:

<META NAME="GOOGLEBOT"
CONTENT="unavailable_after: 31-Dec-2007 23:59:59 EST">

Full details on the Google Webmaster Blog at:

Official Google Webmaster Central Blog: New robots.txt feature and REP Meta Tags

Also offers support for non HTML pages such as PDF's, spreadsheets etc

[Brendon]

Don't - please. Yet another ill-conceived way to trash the WWW, courtesy of f*ckwit SE engineers who should know better. After all, nofollow has proved a stunning success hasn't it?

[tbp]

Whats the problem with it?

I think it would be usefull to stop pages being included in search results after a certain date.

[hooperman]

Originally Posted by Brendon Don't - please. Yet another ill-conceived way to trash the WWW, courtesy of f*ckwit SE engineers who should know better.

Any chance you could explain why you think it's a bad idea? On the face of it it seems like a useful tag, but maybe I'm missing something.

[Brendon]

>> I think it would be usefull to stop pages being included in search results after a certain date

Yup, that's exactly the problem. Remarkable as the idea may be, Google do NOT own teh Intarwebs. The old refrain used to be "Do things as though SEs didn't exist", because when you start playing with the structure of the WWW, you break the principles on which their carefully crafted algos rest (go find a copy of the "The Anatomy of a Large-Scale Hypertextual Web Search Engine", Page and Brin, the seminal "BackRub" paper).

The original paper (what's there now has been altered - originally, it queried the impartiality of SEs that sold ads...) and in fact pretty much all work on IR assumes that doucments are created SOLELY FOR THE INFORMATION CONTAINED THEREIN, and the "best" info will therefore always, naturally, rise to the top.

Enter SEO.

Now, some documents are designed with malice aforethought, specifically to rank because a reasonable approximation of the rankings criteria can be derived, and that knowledge applied to the creation process. This goes directly against the underlying assumptions the whole WWW is based on

[hooperman]

Originally Posted by Brendon Remarkable as the idea may be, Google do NOT own teh Intarwebs.

But they do own their own data. It is up to Google what data they collect and how.

If using this meta tag stops old - and false (in the case of expired offers, for example) - data from being returned then I think this is a good idea.

I'm still unclear why you think this meta tag is a bad idea. Can you see its use having any negative consequences?

[tbp]

I still think its a good idea.

The internet is full of complete crap, thats out of date, but has always been a job to actually get removed. Even taking down the page doesn't necessarily remove it from the results, I`ve seen a lot recently where the web site has gone, but the cached version is still there, and still shows in the results.

For me, it will be really usefull, removing data from the index when I want it removed.

I think thats the thing that goes against your argument. Its not like their removing pages themselves, they only remove it if you tell them to, when you no longer want the page displayed. Should cut down on a lot of "pollution" with old out of date pages if its adopted.

[Brendon]

>> I'm still unclear why you think this meta tag is a bad idea

Because it interferes with the structure of the WWW. That is a bad idea on SOOOO many levels

>> For me, it will be really usefull, removing data from the index when I want it removed.

O rly? What about removing your data when I want to remove it? The bottom line is that SE engineers aren't smart enough to make this work, PhD or no. Look at what happened with nofollow - utter disaster. Users are also generally as sharp as a marble. This will get misunderstood, misused and basically make everyones life a little bit harder.

If SEs want a higher quality index, they are going to have to work out how to determine relevance for themselves, not rely on self-reporting.

>> Can you see its use having any negative consequences?

I can think of 3 distinct ways this can be abused and in fact I've heard of examples of it's abuse already. I hope for your sake you never find out what they are