Help me with a serious Google problem!

[moredial]

Tweet

I've had the same quirk with Webgains tracking URLs getting indexed by Google. It happened when I used an .HTACCESS redirect to mask an affiliate link but forgot to explicitly specify a 301 header.

So if the affiliate is using something like...

Redirect /MyMaskedLink hxxp://track.webgains.com

...they ought to use:

Redirect 301 /MyMaskedLink hxxp://track.webgains.com

You can also set a 301 header with a PHP redirect, if that's how the affiliate is masking affiliate links.

(Be sure to double check your HTACCESS file, as even minor typos will bring your site tumbling down).

Sorry, but 301 is bad.

301 gives the PR of the sending page to the receiving page. Good if you rename a high PR page on your own site but does nothing for the PR ranking of your own site if you are redirecting to an external site.

302 is bad because the sending url is given the PR of the redirected page. Good if you are renaming within your own site but black hat SEO that is often used to knock a competitor out of the SERPs.

If you just say Redirect /x http://y - then the default is to treat it as a 302 redirect.

Yahoo treats a 303 the same as a 302. I don't know what Google does with a 303 as I had Google banned from my site at the time in an attempt to get Google to remove all the rubbish it was still caching for the site.

[QualityNonsense]

Sorry, but 301 is bad.

301 is not 'bad', nor are 302 directs 'bad'. Each has legitimate uses (permanent & temporary redirects, respectively). An affiliate link is a permanent redirect, ergo a 301 makes perfect sense from a user experience perspective and for search engine spiders.

If you're losing sleep over 'leaking PageRank', NOFOLLOW the affiliate links. The question was how to resolve the problem of affiliate URLs getting indexed by Google - and 301ing the redirects fixes it.

then the default is to treat it as a 302 redirect.

Exactly my point. Google chokes on 302 redirects at the moment. Ergo, removing the 302s ought to fix the problem pronto next time the affiliate site is indexed (all other things being equal).

[km8]

If you don't like the way Google displays the results then stop supporting Google. Stop using gmail, adwords, adsense, google alerts, google analytics, etc and stop using Google as a search engine. Ban Google from your sites via the robots.txt. And tell everyone else to do so also.

If you feel you have to use Google with its 'free webmaster tools' that give Google all the marketing research they need to succeed as an advertiser, then it is you personally that is helping Google to succeed as a bad search engine and a very good advertiser.

And all of that is relevant to the current discussion how?
How exactly does any of that help the merchant, Webgains or the affiliate, right now with the current problem?

[moredial]

And all of that is relevant to the current discussion how?
How exactly does any of that help the merchant, Webgains or the affiliate, right now with the current problem?

I really did not want to say any more on the subject and wanted to leave the followers of this thread to read between the lines. But, as you ask:

Just now bedstars are running an adwords ad which is displayed when a search is made with only 'bedstar' and the ad displays text which is saying that they are the official site.

Where is there any incentive for the people who work at Google or their shareholders to get these problem fixed if merchants are happy to give Google more money when Google removes their URL?

And, even though they are only appearing in the #1 spot, just displaying the SERPs has my browser being asked to accept a cookie from webgains (I think it is the bedstar cookie via the redirect that Google is offering me even though webgains's url is being offered as the source of the cookie).

When I click on the link, bedstar is using google-analytics.

Summary: bedstar have their URL replaced with that from another domain and reward the cause of their distress by paying for advertising and giving away highly valuable marketing information about people who are interested in their site. That is like returning to a restaurant where the waiter left a fly in your soup and was rude to you and leaving twice the tip you left the last time.

Let us look at the bigger picture, the bit that makes me as an affiliate a little biased in my opinion.

There is a growing press which is revisiting the whole concept of affiliate marketing and the cost of affiliate commissions compared with the cost of the many PPC options which are now available to merchants. Why pay super affiliates who are using PPC these large commissions when the merchant can do their own PPC campaign and pocket the commissions?

Affiliate urls appearing in the SERPs is an old problem. If it is now starting to affect more merchants and at the same time their own URL disappears from the SERPs it won't take long before they begin to re-examine their marketing avenues. Both 301 and 302 redirects are a problem for Google and the other search engines. How long will it be before merchants start to drop any affiliate network where either the network or their affiliates are using 3xx redirects?

Since affiliate marketing began, affiliates have always been the first to be left to the wolves.
Small merchant starts using affiliates. Affiliates are successful and merchant becomes a household name. Affiliate commissions reach a stage where an extra zero suddenly becomes noticeable on the profit and loss statement. Merchant realises that they are at a happy balance of repeat custom and new customers from their own marketing efforts and that dropping the affiliates will save them money and hold the business growth within more manageable levels.

What is the level of affiliate URLs in the SERPs that will trigger dropping the networks in favour of PPC to maintain the hard won brand awareness of the merchant? How many little businesses are being threatened by this one gremlin? There is only one winner.

If you are happy with that, fine. Stop complaining, accept the gremlin and work your marketing around it.
If not, it may be many years since the gremlin first appeared but it may not be too late to tell a business that you no longer wish to give, give, give in return for a request to give more, this time in hard cash. If Google wants to keep taking our cash and also have us continue to give them all our (personal/business) marketing information in exchange for a few 'free' tools, maybe they should fix the gremlin first.

[km8]

All of that may be true Moredial but, so long as Google holds 80% of search traffic it may not be commercially wise to ignore them.

[befuddle]

Affiliate urls appearing in the SERPs is an old problem. If it is now starting to affect more merchants and at the same time their own URL disappears from the SERPs it won't take long before they begin to re-examine their marketing avenues.

On 16th August, someone on the German google performed a search for "benfica away shirt 07/08". They found the site that sold the shirt and bought it. The link to the merchants site had my tracking code included. I got paid a commission.

Why are affiliate links in SERPS "an old problem"? To me, those links are valid affiliate links.

The only difference to that example and the BedStar one, is that the BedStar home page has been replaced with an affiliate link.

Yes, in the case of the merchants own url disappearing, the merchant will get upset and may close down the affiliate channel, to save themselves 6% affiliate commission and the further 30% network costs. However, surely these merchants have affiliate schemes in the first place because affiliates provide more exposure and revenue than they can currently generate on their own within the search engines?

[moredial]

All of that may be true Moredial but, so long as Google holds 80% of search traffic it may not be commercially wise to ignore them.

Sounds like you are one of the folks who is happy to arrange your marketing around the gremlins.

Know enough about those gremlins and how to use them to your advantage and you can have a very successful site in the natural results.

[moredial]

Why are affiliate links in SERPS "an old problem"? To me, those links are valid affiliate links.

By "an old problem" I mean that affiliate URLs replacing the merchant's URL has been around for a long time.

The only difference to that example and the BedStar one, is that the BedStar home page has been replaced with an affiliate link.

That is only because bedstar have the affiliate links pointing to their home page and the 3xx redirect gives the URL for the site sending the 302 redirect and not the URL of the final destination. The URL for the final destination is removed from the index. But, the 302 is not redirecting to the home page but to home-page?affiliate info which is a different URL so that is not the cause.

Just now, Google knows 3 pages with the same TITLE text as bedstar's home page:
http: // track. webgains.com/click.html?wgcampaignid=1761&wgprogramid=485
http: // www. awin1.com/cread.php?s=29356&v=962&q=18525&r=66549
http: // www. bedstar.co.uk/shop/catalog/Bed_category_Mattresses_5FT_Eurolux-p-1-c-557.html

There is a bedstar URL in there. But it does not have as much rank as the webgains URL which is currently being seen as the authoritive site nor the awin URL which is currently the clone and is not displayed. Even though the bedstar URL has more content than the other 2 URLs, it does not have sufficiently unique content from the other 2 to warrant being displayed as a result - the curse of duplicating content within the tags used to evaluate relevance on one's own site and directory depth.
Wise Google has removed the bedstar home URL because someone forgot to teach the script the correct way to display URLs where 302 and 301 redirects are used. (I have not followed the awin link to see what that does, but even if it uses a similar method to webgains it should not have resulted in the bedstar home page disappearing.)

If you do a 301 redirect on your own site, you want the new URL included in the SERPs and that is what Yahoo does and Google may or may not do, depending on whatever it is Google depends on.
If you do a 302 redirect on your own site, you do not want the new URL displayed and the old URL continues to be displayed in the SERPs, which is what Yahoo does and Google may or may not do depending on some URL-text valuation factor.

On your own site, you are happy.

When some other site messes up your site's URL by doing a 301 or 302 redirect, you are going to have problems with your own site's ranking and you are not going to be happy.

[km8]

Sounds like you are one of the folks who is happy to arrange your marketing around the gremlins.

No, it doesn't sound like that.

The last time I drove to London I used the DoT's roads.
That doesn't necessarily mean I was "happy" with them.

[Dynamoo]

Time for my two cents worth:

On the technical issue:

1. I'd say the robots.txt file definitely looks wrong. Why allow googlebot or any other crawler access to anything on the track.webgains.com domain anyway? (A quick bit of searching indicates that most other networks don't use robots.txt in this way, but that might require more digging). I would suggest that you review your robots.txt and then look at Webmaster Help Center - How can I prevent content from being indexed or remove content from Google's index?
2. You could go one step further and ban the Googlebot IP. That's trickier though - see Official Google Webmaster Central Blog: How to verify Googlebot
3. It's a moot point, but I believe that the fault is with Webgains for allowing the Googlebot to spider via the affiliate URL, Google is simply seeing the affiliate link as more authoritative.
4. The merchant should also use Google's Webmaster tools - Google Webmaster Central - to ensure that there are no penalties on their site (to be honest, it doesn't look like an SEO penalty from here).
5. Playing whack-a-mole with individual affiliate URLs is probably going to be fruitless. It looks as if track.webgains.com will always outrank the merchant for this particular content until the indexing issue is fixed.

On the payment issue, you will have to look closely at your T&Cs and merchant/affliate agreements to determine if there's anything that covers this. The fault is not with the affiliate - it's either with the network or Google. Yes, it's a tricky situation that's to be sure, but wholesale reversing of commission is likely to make it worse. The best approach is for all parties involved to come to an agreement before it gets legally unpleasant.. if you can't come to an agreement and if it's a substantial amount of money then taking legal counsel is often a good idea.

[kier]

Firstly can I just say a huge thank you to everyone who's contributed to this thread and who has contacted us to offer advice on this issue, you've all been very helpful and I've got a lot of information to digest!

Pending resolution of the issue I've now persuaded the merchant to remove the cancellation on Ray's transactions and place them in delay status until we've got this all sorted out.

[moredial]

The fault is not with the affiliate - it's either with the network or Google. Yes, it's a tricky situation that's to be sure, but wholesale reversing of commission is likely to make it worse. The best approach is for all parties involved to come to an agreement before it gets legally unpleasant.. if you can't come to an agreement and if it's a substantial amount of money then taking legal counsel is often a good idea.

I agree with most of what you say, except that in this case the fault is not with the network - their server is giving the correct header responses for a user agent and Google is ignoring what it is being told to do. Reading and following Google's advice to webmasters does not help because Google itself does not follow what it tells people to do. It just puts that up there in the hope that the problem will be solved by webmasters rather than Google having to pay programmers to write a new script. Maybe that is a bit harsh. But Google says 'do not spam' and yet Google still rewards spammers with first page ranking because that is part of the original algorithm which still works but gives Google bad press so webmasters get asked to stop.

It is the withdrawing of commissions which is the big issue. One affiliate was unlucky because their affiliate ID was in the offending URL. But it could just as easily be all affiliates with the networks where 3xx redirects are used. I am also a bedstar affiliate; I just don't promote it as aggressively as some affiliates but this does leave me wondering if there will ever be any point in promoting any merchant where the 3xx gremlin is risking commissions.

[befuddle]

Pending resolution of the issue I've now persuaded the merchant to remove the cancellation on Ray's transactions and place them in delay status until we've got this all sorted out.

Many thanks for this Kier. We've just spoken on the phone but I publicly want to thankyou for bringing this issue to the attention of the forum and for your handling of the commission issue.

Hopefully the merchant will still see the affiliate scheme as a positive, once their true url is back in Google.

Cheers,
Ray

[hpops]

This has been one of the most fascinating threads of recent times.
Befuddle - although the problem obviously directly affects you, I'd like to echo your last comments.
This is a great example of all the reasons why this industry rocks sometimes.
All parties openly discussing best route through a complicated commercial and technical environment.
I think that the best route forward is now being pursued by Webgains. Google have been making it easy to pagejack (whether deliberately or, as in this case, through sheer domain weight) for far too long and these issues will start to filter into the mainstream.
I do think that the merchant (and to a certain extent, their agency) have been incorrectly lambasted here - if it was my client I'd be struggling to explain the benefits of their investment thus far. I'd like to think that I could direct the client to this thread and they would feel that the industry has their best interests at heart.

[moredial]

I think that the best route forward is now being pursued by Webgains. ....
I do think that the merchant (and to a certain extent, their agency) have been incorrectly lambasted here - if it was my client I'd be struggling to explain the benefits of their investment thus far. I'd like to think that I could direct the client to this thread and they would feel that the industry has their best interests at heart.

Neither webgains nor affiliatewindow are to blame for what is occurring in the SERPs.

Nor are bedstar doing anything incorrectly.

Everyone is following the theory of 3xx redirects correctly and requesting that the bedstar URL is used by the user agent.

This is just one network and one merchant being highlighted here. I suspect there are a lot more cases but they are not so publicly noticeable because they have not had the effect of removing the merchant's home page and everything that means to the PR of the rest of the site.

The cynic in me says that our favourite ppc advertiser has found an 'innocent' way to remove from the natural results URLs that businesses are trying to promote through affiliate marketing [rather than their ppc programme], at the same time show affiliate marketing in a bad light AND give back top ranking in exchange for some advertising revenue.

Edit
Search for site:bedstar.co.uk on live.com and yahoo.com and neither of them have any problem putting bedstar's home page at the top of the results.
What more proof do you need that neither networks nor merchant has caused this issue.