Finding the right solution / api

[horse]

Tweet

So I have been having a bit of a nightmare finding a solution that works for me.

I am trying to get data for video games price comparison and all the api's that I have found seem to have some hang up or other.

I have 10,000 - 15,000 products that I need to crawl for data, some of them fairly reglarly to be able to see change in prices. Due to the volume of my products any form of manual processing is a no go, and automated post processing needs to be as streamlined as possible, eg no sorting through 1000's of results just to get back 2 or 3 usable prices.

My needs:
- To be able to query using a product title, and only get back results for that product title. the buy.at api splits your query by the space character and then searches for all the words using OR. So querying 'battlefield 3' will search for anything with 'battlefield' or '3' in the title which makes it work hugely inefficiently
- To be able to filter by platform / console. The buy.at api has the platforms as sub categories, so makes it fairly easy to get stuff just for say playstation 3, however the affiliate window api cant do this, and you have to rely on the product title having the platform in which only a small percent do.
- To be able to specify currency / or locale of the merchant. the AW api just returns a price number. No where in the API documentation can I find what this number is, USD / GBP / etc. And I also cant seem to specify I want merchants from a particular country
- Deeplinks that actually work. The deeplinks on AW seem to for some reason take 60 seconds+ to load, or just dont load at all. I'm not sure if this is because my account is still in development mode or not, but theres no actual error other than the browser saying the connection timed out etc.

I did some experimentation using google product data and could get prices and results as I wanted, but obviously that would not return any form of affiliates solution / monetization.

Starting to get annoyed that these systems that have taglines describing how their API lets you create bespoke, highly adaptable solutions, are actually pretty rubbish when you are trying to do anything specific and tailored.

Any ideas / help / experiences anyone can offer would be very useful

[Cheapdvds]

So I have been having a bit of a nightmare finding a solution that works for me.

I am trying to get data for video games price comparison and all the api's that I have found seem to have some hang up or other.

I have 10,000 - 15,000 products that I need to crawl for data, some of them fairly reglarly to be able to see change in prices. Due to the volume of my products any form of manual processing is a no go, and automated post processing needs to be as streamlined as possible, eg no sorting through 1000's of results just to get back 2 or 3 usable prices.

My needs:
- To be able to query using a product title, and only get back results for that product title. the buy.at api splits your query by the space character and then searches for all the words using OR. So querying 'battlefield 3' will search for anything with 'battlefield' or '3' in the title which makes it work hugely inefficiently
- To be able to filter by platform / console. The buy.at api has the platforms as sub categories, so makes it fairly easy to get stuff just for say playstation 3, however the affiliate window api cant do this, and you have to rely on the product title having the platform in which only a small percent do.
- To be able to specify currency / or locale of the merchant. the AW api just returns a price number. No where in the API documentation can I find what this number is, USD / GBP / etc. And I also cant seem to specify I want merchants from a particular country
- Deeplinks that actually work. The deeplinks on AW seem to for some reason take 60 seconds+ to load, or just dont load at all. I'm not sure if this is because my account is still in development mode or not, but theres no actual error other than the browser saying the connection timed out etc.

I did some experimentation using google product data and could get prices and results as I wanted, but obviously that would not return any form of affiliates solution / monetization.

Starting to get annoyed that these systems that have taglines describing how their API lets you create bespoke, highly adaptable solutions, are actually pretty rubbish when you are trying to do anything specific and tailored.

Any ideas / help / experiences anyone can offer would be very useful

either use something like pricetapestry to use datafeeds but will require setting up etc, or the easiest option would be medifusion that run find-game.co.uk and find-dvd.co.uk, they have a feed that can be used (most good games site use it)

[lowndsy]

MediFusion is very good. It is a live XML service rather than a downloadable data feed, but they have sample scripts available to help you get started. It works well with the Magicparser script that powers PriceTapestry.

Or you could plug skimlinks into your Google experiment to monetise it where an affiliate programme is available.

[enjoymarcus]

Why not just look at Price Comparison affiliate programmes like PriceRunner or Kelkoo (I'm sure there are more) - I know Kelkoo has an extensive API that allows you to direct traffic straight to the merchant (rather than via Kelkoo) - obviously the payout is different (CPC) but you are achieving the same theoretical outcome.

I guess if you get really smart, you potentially include more results that Kelkoo doesn't have.