Help me with a serious Google problem!

[befuddle]

Tweet

We do not run the Google search engine nor are we responsible for its content.

If say BBC online were to publish a story on a "serious Google problem" and they linked to BedStar using my tracking code ... and I then made some sales ... would my commissions be reversed then?

Why are sales valid from forums, emails, rss and websites but not natural search? In fact, sales made via natural search are perfectly valid. Please explain to us affiliates otherwise. It's just in this case, they're not because my tracking link is included in the number one result for certain search terms on the biggest search engine. It's a text link on a site, that we are not responsible for.

The issue for me is twofold.

i) Affiliate is blamed by the merchants SEO "experts". Judging by comments here and the pace in which the reversals were actioned.

ii) Commissions reversed for an ethical link. It's a text link at the end of the day. The reason given for the reversals simply say "natural search". Since when has "natural search" been a valid reason. Well, it appears from about lunch time today.

For the record, the commissions wouldn't even pay for a train ticket from Leeds to London. The amount of money is not a problem for me. I just can't believe reversal of commissions were discussed and actioned.

[Qui Gon Jinn]

Excuse me Mark, Moderators hat is always off on personal opinions, I have had that discussion before on that. So straight back at you & your choo choo train set. I complimented you guys for at least bringing to the forum, then you chuck it straight in my face, if you wish to complain, complain to Matt Wood.

But numero uno in this discussion is befuddle, then ensuring it doesn't set a precedent if it is an unfavourable outcome, then potential cause of the problem & how to rectify in future for something for crying out loud he has not been done wrong.

Let's have a straight answer from you ... off the record in your personal opinion ... should the affiliates commission be reversed? Then put on the corporate hat on for an official stance.

But if you wish to be funny about it lets take a look at this search

www.bedstar.co.uk - Google Search

I am highlighting that the merchant is more concerned & yourself on something an affiliate hasn't done wrong to on that search results what is wrong, which is plain to see. Being prepared to stick up for you guys for at least bringing the subject up, I guess I have misjudged that loyalty. So what have you done about the other little problems? Half hoping you sort those out in the background whilst we stay on track on this discussion. But I see that seems irrelevant. Is this reflective of you as a network & the merchant to where priorities lie.

If that's the way you feel, then that's the way the cookie crumbles, you'll simply have to decide whether it's right to allow the merchant to reverse commissions as easily as that. Which some like myself disagree with & some others may do, there is a difference of opinion both on & off the forum.

It doesn't matter if the amount is irrelevant to the affiliate or not, it's a matter of principle & the standards this sets for future issues where a freer reign is given for reversing commission. They are intertwined.

Have you looked into any of the suggestions made to what could cause it?

I can't see how this can be defined as affiliates being squeezed on SEO. It's not a matter of SEO

Rephrase then, affiliates should not be squeezed on the anomalies thrown up on natural serps results whether via intential / unintentional seo or not .. or some miraculous wonder.

[splashy]

This seems like an amazing long discussion on what is s simple problem ( after a quick scotch )

1. does the affialte control webgains pages -No
2. has the affilate done anything to achieve this listing -Unlikely
3. Does webgains have control over what pages are indexed - YES

Should the affiate be penalised for something outside there control - no.

We had a call recently from a car hire company as www.merchant.com/myid=xxx was outlisting their own page on their merchant name. I simply pointed out unless they block Google if its not us it would be someone else.

Over the last 5 years I think there has been a real drop off in basic web marketing knowhow - yes Ok thats off topic but related.

[Marc Gear]

Clearly if this is to move forward there needs to be communication between Bedstar, Webgains and Befuddle and agreement reached on the future.

What I think is non-negotiable though is that Befuddle should be paid all commissions up until the point that he was notified of the problem regardless of what is agreed going forward.

To unilaterally reverse the commissions as the merchant has done is clearly breach of contract. Those commissions should be reinstated forthwith and if I were Befuddle I would insist on that being done before entering into any negotiation about how to resolve the problem.

Whatever the outcome regarding payment of the commissions that were recorded, Webgains will, as ever, ensure that the outcome is satisfactory to both the merchant and the affiliate. As many affiliates will attest, Webgains history and reputation in this matter speaks for itself.

[Marc Gear]

Excuse me Mark, Moderators hat is always off on personal opinions, I have had that discussion before on that. So straight back at you & your choo choo train set. I complimented you guys for at least bringing to the forum, then you chuck it straight in my face, if you wish to complain, complain to Matt Wood.[/b]

I did not mean to imply that you were not allowed to divulge your personal opinion. I just felt that your comments were driving the thread offtopic. Sorry if this was taken the wrong way.

Let's have a straight answer from you ... off the record in your personal opinion ... should the affiliates commission be reversed? Then put on the corporate hat on for an official stance.

I'm happy to discuss my personal opinion on the matter in PM, MSN or email.

[moredial]

An interesting discussion. And a 'bug' which has been around about as long as Google has been around. It took me a long time to discover why Yahoo had followed a url changed redirect I had set up but was putting the new content onto the old url.

It is not a bug - it is search engines doing what we are asking them to do when we do a 302 redirect. It is a temporary move of the address of the content, not a move of the content to a new address. So the old url is given the new content and the redirect url is ignored as it is only temporary and search engines don't want to fill themselves with indexing pages which may be gone the next time they crawl. We have suffered enough years with Google and Yahoo holding onto the ghosts of old pages long after the domains have ceased to be registered.

Back to the subject in hand.

This is what a search engine bot reports when I send it to analyse the URL
track.webgains.com/click.html?wgcampaignid=9157&wgprogramid=485

"WARNING! The URL you entered sent a redirect Location: HEADER response. Below are the redirect Location: HEADER responses sent by your Web server which the analyzer followed in the order received:

Redirect URL: http: // www. bedstar.co.uk?utm_campaign=Affiliates&utm_medium=d atafeed&utm_source=webgains.com&adnetwork=webgains &siteid=9157
Redirect URL: http: // www. bedstar.co.uk/shop/catalog/index.html?utm_campaign=Affiliates&utm_medium=data feed&utm_source=webgains.com&adnetwork=webgains&si teid=9157

It is recommended you verify the URL before submitting to search engines. Not many search engines will follow redirects.

Note! The redirected URL may not have been checked against the robots.txt file to see if it is accessible by robots. You should verify the above redirected URL is allowed access before submitting.

Note: During processing an unfriendly character was found in the URL. Characters which are sometimes denied in URLs by search engines are:

', ?, #, >, <, (, ), \, ;

This character may have been found during a redirection and not shown in the above URL."

(Had to add spaces into the urls so that you can see the two actual urls rather than just the page title)

With redirects from all sides, it is no wonder that googlebot has lost track of the original url.

A possible solution to the problem is for webgains to add the following to their robots.txt file

User-agent: googlebot
Disallow: /click.html?*

This will only work with googlebot. Unfortunately as it is not valid syntax, as I understand it, Yahoo and MSN will ignore it even if asked to disallow. Which means that they too will attribute the bedstar content to the webgains url.
Yahoo has just brought out a new webmasters' tool whereby you can log-in to your Yahoo account and indicate to Yahoo what url it should index for dynamic urls.
MSN has been fairly good at working out duplicate content and as far as I know has not offered webmasters any way to allow/disallow dynamic urls to be indexed.

Google is well known for ignoring disallow statements in robots.txt files so only Google knows whether or not it will work. Asking Google to remove the url from their index is also not usually a solution in these cases and bedstar could suffer from the ghost of this page for a long time.

Whatever solution webgains did add to their robots.txt is not working as expected - if it was, the blockage of the url would have been reported above.

Google has always had problems with working out who the true owner is of content. Without looking at PR of the sites (here the webgains/bedstar page has a PR0 reported by the tool I was using) it has to be assumed that webgains with all their affiliates has a lot more inbound links than bedstar which makes webgains the more likely to be given ownership in the case of duplicate content.

A secondary effect which has not been mentioned is that bedstar has probably 'lost' its PR and any PR which it did have has now been transferred to webgains. I won't distract from the topic by getting into the whys and wherefors of that subject but it does explain why bedstars has disappeared from the SERPs and been replaced by webgains.

There could also be many other factors affecting all this which I won't comment on here. If it is affecting bedstar with their other top key phrases then it is not too difficult to discover the additional causes and do some SEO, etc to reverse the effects.

With such a long thread, I have rather lost track of which affiliate this is affecting. I really need to see the 'offending' page on the affiliate's site to be able to comment on what part, if any, the affiliate has played in all this and what, if anything, needs to be done on their site. PM me if you want me to look at the affiliate page.

[befuddle]

With such a long thread, I have rather lost track of which affiliate this is affecting. I really need to see the 'offending' page on the affiliate's site to be able to comment on what part, if any, the affiliate has played in all this and what, if anything, needs to be done on their site. PM me if you want me to look at the affiliate page.

I promote BedStar at this page:

» BedStar Coupon Code - £15 Off Coupon Code Exclusive « from Shop Codes UK

Feel free to PM me any personal comments. I won't be changing this or any other page or file on my site though.

[Barry]

I think webgains should :-
User-agent: *
Disallow: /click.html?
would be fine to stop it happening (in theory) in other engines

I say in theory cos AW links and other networks af urls still appear in the ranks even though they are excluded via the robots.txt file in most cases.

It might also be useful if the merchant inserts a no,index meta tag thats dynamically inserted if the incoming traffic is coming from an affiliate link.

I don't see in any way why you they should penalise bufuddle for this. Hey it was only a little while back that one network offered the option to add meta tags and a title to their redirects in the hope that af links (I presume) could be optimised for use in natural rankings. (or is this why they moved away from it as its was a tad controversial?)

In any case I am sure that everything can and will be sorted for all involved and would be very interested and disappointed to hear otherwise.

Just to add - don't you all think this has been an open and fresh discussion on the problem (as it must have happen to loads of merchants over the years)

If it wasn't for the openness of webgains it may have never have been addressed or discussed? Its also nice for moderators to speak their minds too, though as I read it it was all a misunderstanding - ahhh..

[bazza99]

Again, just to come back to this - Bedstar (or their SEO company) have clearly recently made substantial changes to their site (this is evident if you look up Bedstar via a Yahoo search - you can see the old page title for the site, pre-optimisation).

It's pretty clear that some changes have recently taken place on the site, which may well have led to a penalty, hence the current set of circumstances.

If befuddle's link was not at number 1 position then I'd have thought it possible that bedstar simply wouldn't have a top 10 position in Google at all for those keyword terms currently. If anything, the appearance of befuddle's link at number 1 may have saved bedstar a bit of business.

Although I hear what Marc is saying (and think it's good that webgains are willing to be open about this on the forum - many other networks wouldn't and I think webgains are to be commended for their approach). If this has just effectively been caused by a bug in the way that Google is indexing, then I guess the situation will be resolved satisfactorily within a few days.

[KieronD]

I've been reading this post with interest, here are my thoughts:

1. Befuddled has done nothing wrong, neither have Webgains.
2. The fault lies with Google.
3. Befuddled's commissions should indeed have been reversed. Put yourself in the place of a merchant for 1 minute, if you site suddenly shows your affiliates URL and you end up paying them for traffic and sales that shouldn't be theirs then you wouldn't be happy would you?

However, if befuddled's site ranked higher than Bedstar naturally then that would be a different matter, he would be due the commissions. But as I say this is a different matter altogether.

If I were Bedstar I would be speaking to Google ASAP to see what can be done to put this right. I also would not of reversed the commissions immediately without speaking to Befuddle personally about it too. And even though I think the commissions should be reversed I wouldn't have done it. I'd rather keep my affiliates sweet and working with me so I'd say "hey I know you have done nothing wrong, lets work together to get this put right and btw you can keep the commissions for being a loyal affiliate".

[befuddle]

"They know you have done nothing wrong, lets work together to get this put right and btw you can keep the commissions for being a loyal affiliate".

The thing is. They or their representatives don't. I'll quote Kier: "His SEO company of course have implied that this has been done deliberately."

[kier]

Hi Ray,

That's what the SEO company initially were insinuating, I quickly cleared that up with BedStar. They are in no way thinking you did this deliberately and reversing the commissions has nothing to do with punishing you or claiming you have actively caused this situation.

Thanks to everyone for your opinions and the spirited discussion, we're still working on a satisfactory resolution and I'll be sure to keep you informed of the progress.

[John Jupp]

Google quite clearly states that their organic listings are down to how they interpret which unpaid listing should be first or higher than another.

For any merchant to rely as part of their business model on ensuring they rank higher on unpaid organic listings is a dangerous and almost suicidal commercial approach. Yet they commissioned a company to create highly ranked organic listings.

If the changes to the merchant site are recent then Google will have penalised and eventually Google will reindex the merchant site. However it is not a requirement by Google to do so. The only way to ensure listing is through paid search. That is the Google business model.

If the merchant has an issue with their unpaid organic listing they should address this directly with Google.

As for an affiliate's organic listing (determined by Google) being higher than a merchant or for that merchant to have received a penalty (see note on their updating their site) it is a fundamental Breach of Contract for the merchant not to pay the affiliate for sales generated through the affiliate's unpaid organic listing.

The merchant needs to understand the difference between paid and unpaid search.

The merchant needs to communicate with the affiliate directly.

The merchant seriously needs to understand Contract Law.

The merchant should pay all commissions that were generated.

The specific issue is that an affiliate's organic listings is biting the affordability of the merchants affiliate program. I've already stated that reliance on organic search as part of the merchants business model is bad practice.

The merchant needs to review their own business model. They also need to be made to understand that the issue is with Google and not the affiliate. The network is caught between a rock and a hard place. The way in which the network accredited links are trawled has resulted in an issue arising with Google unpaid listings. However the merchant had undertaken site adjustments and Google obviously penalised them.

To me this issue of unpaid Google listings is clear cut. the merchant got their business model wrong. They have to pay up. They have to review their business model.

[befuddle]

This is not a bug and I quote Matt Cutts ...

"Now let’s talk about off-domain 302 redirects ... Google ... return the destination page more than 99% of the time. Why not 100% of the time? Most search engine reserve the right to make exceptions when we think the source page will be better for users, even though we’ll only do that rarely."

So I am an exception and fall into the <1% bracket, which Google has the right to do.

BedStar aren't the only merchant to see this. As I said in my blog, a merchant has started a discussion over at Google Webmaster Help.

[moredial]

I promote BedStar at this page:

That page is squeaky clean.

One way that you could improve its rating with search engines is to add the attribute rel="nofollow" to any affiliate links. This is because search engines do not like being sent through links that contain redirects: for reasons that you see now in the SERPs.

User-agent: *
Disallow: /click.html?
....
It might also be useful if the merchant inserts a no,index meta tag thats dynamically inserted if the incoming traffic is coming from an affiliate link.

Not sure that that will work, it will stop the visits to /click.html? but not to /click.html?something as those are 2 different urls.

The meta robots tag is very useful, if only googlebot obeyed it. To (try to) stop googlebot you need to use

<meta name="robots" content="noindex,nofollow" />
<meta name="googlebot" content="noindex,nofollow" />

The bots are very sensitive to correct syntax in this tag. Most people get it wrong, i.e. content="index,nofollow" will not give the desired result.

Reading RFC 2616 gives the following recommendations:

10.3.3 302 Found

The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.

The temporary URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

If the 302 status code is received in response to a request other than GET or HEAD, the user agent MUST NOT automatically redirect the request unless it can be confirmed by the user, since this might change the conditions under which the request was issued.

Note: RFC 1945 and RFC 2068 specify that the client is not allowed to change the method on the redirected request. However, most existing user agent implementations treat 302 as if it were a 303 response, performing a GET on the Location field-value regardless of the original request method. The status codes 303 and 307 have been added for servers that wish to make unambiguously clear which kind of reaction is expected of the client.

10.3.4 303 See Other

The response to the request can be found under a different URI and SHOULD be retrieved using a GET method on that resource. This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource. The new URI is not a substitute reference for the originally requested resource. The 303 response MUST NOT be cached, but the response to the second (redirected) request might be cacheable.

The different URI SHOULD be given by the Location field in the response. Unless the request method was HEAD, the entity of the response SHOULD contain a short hypertext note with a hyperlink to the new URI(s).

Note: Many pre-HTTP/1.1 user agents do not understand the 303 status. When interoperability with such clients is a concern, the 302 status code may be used instead, since most user agents react to a 302 response as described here for 303.

The way I read that is that either a no cache response should be sent with a 302 redirect or use the 303 redirect with the no cache on the 302 being preferred.

Which brings us back to the robtos meta tag and content="noindex,nofollow,nocache". With a header cache control being sent as well for everyone who ignores the robots meta tag.

If all affiliate networks implemented this correctly, the problem of affiliate urls 'stealing' the merchant content would disappear overnight. I can see merchants rushing to use affiliate networks who get this right and fleeing from those who ignore the issue. This is not a search engine problem. Rather ignorant webmasters using code with implications that they do not understand.

I will ask the question on another forum to see what affect the 302 with no cache is likely to have on the different (redirected) uri - surely search engines will only read that as a 'not to cache in this instance' and correctly cache when they visit the uri in its own right.