Ripping a product feed - unethical?

[GW-Sean]

Tweet

Hi guys,

Question for you (and you'll have to excuse my ignorance in this area ) - I have a potential client on the way...obviously I can't name names at this point but I'm confident that many of you would be interested if all goes to plan.

At the moment, they have no product feed (and limited resources in the whole programming department)

The question I had is ...is there a product available that would allow me crawl through their site to create a product feed? (and would this be classed as unethical?) ...obviously I'd recomend the crawler to the client so they can provide this - rather than just stealing their product list ...but I thought I'd check this out before I start jumping into trouble

Thanks

Sean
sean.carter@mvmediagroup.co.uk
0161 242 7282

[pricethat]

There are a couple of things that will crawl a site but they are less than rubbish, ok for some site, awful for most. you will end up spending all day fixing and manually editing the feeds. If you are going to offer a feed then offer a proper feed that is consisten, the last thing affiliates need is an unreliable feed. I personally would not use your feed if i knew it was generated in this way as i can't be bothered with the hassle.

We have a site crawler that extracts feeds for merchants that do not offer them, we do it because it gives us an advantage over affiliates that don't have it but it is a serious amount of work watching it all the time and uses a lot more processing time than using a straight feed.

To answer your question, i do not see anything unethical about it. They already said they want you to sell their products so all you are doing is collecting a list of those products. You are not there to sell their site, just the products!

[GW-Sean]

Thanks for the feedback

I totally agree with you regarding the quality of the product feed - I would not want to be pushing a feed to the affiliates that's unreliable (like you said - its just not worth the extra work involved with constantly checking)

...looks like I'll just have to turn the thumb screws....see if I can persuade the client to work towards having the product feed set up properly (I should've known that a shortcut wouldn't really work)

I'll announce the client properly when everything is 100% (and I have something useful to report! )

Thanks again

Sean

[pricethat]

If the client has a froogle feed then our importer could take that and export it into any networks standard without even thinking about it. Or even just a database dump.No big deal to me and then you owe me a big favour that i can call in at a really inconvenient time of the day or just when you planned to take the mrs out for the night

[GW-Sean]

looks like you've got me pegged!

I doubt they've got a froogle product feed either.... but the database dump may be an option...

I'll come back to you if (when) everything goes well and the client falls in line! (and once I know the way their systems are set up) ...leave it with me

Thanks again....you're a useful man to know ....owing you a favour shouldn't be too much of a problem - the missis would be glad to get rid of me anyway!


Sean
sean.carter@mvmediagroup.co.uk
0161 242 7282

[Itchy]

I would not want to be pushing a feed to the affiliates that's unreliable

Wow thats a 1st a merchant that wants to provide a decent feed count us in what ever its for

[pricethat]

Wow thats a 1st a merchant that wants to provide a decent feed count us in what ever its for

Itchy, you are such a slag

[CompareStore]

Sean,

A regular dump from the database would be best however if they have a list of their product information in Excel then you could simply get them to save this in CSV format on a regular basis and as long as they always export using the same criteria then you could simply upload this to a location on their site and affiliates can then download it. Obviously the spreasheet would need to reflect the current pricing, etc.

[GW-Sean]

Thanks guys - I'll look into this line of thought - I'll try to avoid the manual product feed creation if I can (if I'm not around for a week for example...this would prove to be a pain)...I'll work somethig out I'm sure

I can't believe this is the first time a merchant wanted to provide a decent product feed!...whatever happened to "you get out what you put in".... or was that "you get in when you put out" ?? ....eeeerm ...I think I'll leave it there