+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 21

 

Thread: Product Data Feed System

  1. #1
    Registered User

    Status
    Offline
    Join Date
    Aug 2003
    Posts
    461
    Thanks
    0
    Thanked 7 Times in 5 Posts


    More and more people have been asking when will the Paid On Results product data feed system go live. We are looking at 1st week in December for it going live, ideally we would like to have had it in place before Christmas but most important to us was that we brought out something that was good and that worked, this after all is a long term project.

    We listened to all the feedback we have been given both directly and on this forum, as it’s getting closer to launch we thought we would share some of the details here. The system it’s self is working great, we finally picked a name for it today, and all what’s really left is the demo site of how it works and putting together some support information.

    With the new system we are able to have ready made content units that constantly keep themselves up to date without any merchant input needed. A new one stop location for people who just like their feeds the old fashioned way but most exciting you will be able to do lots of new neat things that you may not have had the technical know how to do before.

    Our product feed will require no technical experience or knowledge, any affiliate can customise a feed to fit into any web site, or enable you to create a whole web site around the product feeds. You will be able to use our feeds though JavaScript or an IFrame, or call the feed from any scripting language.

    If you are using Perl, PHP or ASP, no non-standard modules or add-ons are required. With PHP you can integrate your feed with just 1 line of code. Our feeds work though a simple URL, just fetch the URL from a script and output the contents without any other work if you’re using a template.

    Our template system allows total control over the design, your not limited to any layout or options, just use your own HTML to create the design and layout you want for your site.
    Or if you just want a CSV or XML feed then you can. Any number of templates can be added, and more than one template can be used per feed.

    If using templates, your feed is given to you in a instantly usable format, just output the contents!

    You can sort, search and query the feed just by adding options to the URL - this works whatever method you use to fetch the feed. Each product image is available in 6 different sizes, and you can select which image size you want.

    Our "SQL Like Queries" allows you to have more control over the products, if you want to only show products which are between £10 and £50 in price, contain the word "Football" in the product name and appear in the category "toys" then you can, just enter the query in the URL along with any other options, like sort options.

    You can search your feed; just add the keyword to the URL. If you’re using a template then the results are formatted as per your template and ready to use.

    Let’s say you wanted to create a web site which used products from a merchant, you wanted to show 10 products per page, using a short description, and the visitor could then click a link to go to a page with a longer description. This may sound complex, but with our product feeds it would just require you to design 2 templates, one for the listing page and one for the "more information" page. You would need to create a small script in your chosen language to fetch the right pages or use our IFrame or JavaScript method.

    - Sort method, you can sort on any field within the feed
    - You can show only products added after a time/date you give
    - Use products from selected category or multiple categories, across different merchants
    - Search products and output the results as per your template or CSV/XML
    - Complete template control, no limiting features or restrictions

    Coming soon.
    Last edited by Supercod; 18-11-04 at 10:18 PM.

  2. #2
    Typing with both fingers.

    Status
    Offline
    Join Date
    Aug 2003
    Location
    Allt Y Coed Farm, North Pembrokeshire
    Posts
    4,122
    Thanks
    246
    Thanked 197 Times in 156 Posts
    It good to see a network thats committed to giving us a descent quality datafeed system.

    keep up the good work!

    looking forward to it

  3. #3
    Mogga's Avatar
    Chocaholic

    Status
    Offline
    Join Date
    Aug 2003
    Location
    Oldham
    Posts
    8,405
    Thanks
    498
    Thanked 225 Times in 173 Posts
    Sounds great!

  4. #4
    Bone Idol

    Status
    Offline
    Join Date
    Jun 2004
    Location
    Sussex, UK
    Posts
    55
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Sounds very good indeed

    I look forward to seeing it in action. Does it let affiliates know about a merchant refreshing the data? Or will merchants still have to let affiliates know by the POR message system / email?

    Can't wait to try it out!

    Cheers

    AJ
    Bone Clothing (POR Network)
    www.boneclothing.com

    affiliates at boneclothing.com

  5. #5
    graeme's Avatar
    Registered User

    Status
    Online
    Join Date
    Aug 2003
    Posts
    321
    Thanks
    2
    Thanked 20 Times in 16 Posts
    Originally posted by Bone-Clothing
    Sounds very good indeed

    I look forward to seeing it in action. Does it let affiliates know about a merchant refreshing the data? Or will merchants still have to let affiliates know by the POR message system / email?
    Once a merchant updates their feed with us, all affiliates who are using the feed on their site will have the latest product information without doing anything. Affiliates who have downloaded an CSV or XML copy of the feed will need to download it again or download an update.

    Within the Affiliate Control Panel will be a "status" page which will list the time/date each merchant last updated their feed.

  6. #6
    Bone Idol

    Status
    Offline
    Join Date
    Jun 2004
    Location
    Sussex, UK
    Posts
    55
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Thanks for that graeme. Look forward to using it.

    Cheers

    AJ
    Bone Clothing (POR Network)
    www.boneclothing.com

    affiliates at boneclothing.com

  7. #7
    Member

    Status
    Offline
    Join Date
    Apr 2004
    Posts
    51
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Sounds great! Look forward to it!

    Gary
    Tie Warehouse

    Shopzilla Circle of Excellence Platinum Award Winner 2008
    Affiliate Program Information

  8. #8
    CompareStorePrices.co.uk

    Status
    Offline
    Join Date
    Aug 2003
    Location
    Sussex
    Posts
    649
    Thanks
    4
    Thanked 5 Times in 4 Posts
    How many servers have you got to supply the feeds?
    I'm sure you've thought of this but if you are allowing things like SQL "like" statements you could find the database and servers under quite heavy load so make sure you have over estimated and have enough servers with load balancing set up to handle all the requests.

    I can't wait to get hold of all that lovely data!
    Chris Young


  9. #9
    graeme's Avatar
    Registered User

    Status
    Online
    Join Date
    Aug 2003
    Posts
    321
    Thanks
    2
    Thanked 20 Times in 16 Posts
    Originally posted by CompareStore
    How many servers have you got to supply the feeds?
    I'm sure you've thought of this but if you are allowing things like SQL "like" statements you could find the database and servers under quite heavy load so make sure you have over estimated and have enough servers with load balancing set up to handle all the requests.

    I can't wait to get hold of all that lovely data!
    SQL Like queries is basically a way of querying data in a way you would do using SQL but they aren't real SQL queries. For example lets say you wanted all products which were between £10 and £50, plus the name contained the word football. The query you would use is

    Price >= 10 and Price <= 50 and ProductName =~ football
    So its like SQL, but it isn't, although over time we will extend what you can do with these queries. As for server load, we will keep a close eye on load and before its needed add another server to cope with the load and so on.

  10. #10
    aka Antony

    Status
    Offline
    Join Date
    Aug 2003
    Location
    Bristol
    Posts
    966
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Sounds great, but like CompareStore it scares me how slow this might be unless you are really building a huge server farm to deal with this.

  11. #11
    Registered User

    Status
    Offline
    Join Date
    Aug 2003
    Posts
    2,448
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Sounds good, I seem to have missed this thread with the get together and stuff.

    With the right database structures and query & output caches, a massive amount of requests could be handled without too much hardware and it should scale quite well, particually since the queries shouldn't be that complicated.

    I'm looking forward to giving it a try.

  12. #12
    CompareStorePrices.co.uk

    Status
    Offline
    Join Date
    Aug 2003
    Location
    Sussex
    Posts
    649
    Thanks
    4
    Thanked 5 Times in 4 Posts
    I assume =~ is the equivalent of the mySQL "Like" statement such as "where Product like '%toy%'" ?
    If so this can be quite intensive on databases.

    I have had to write a caching system as the minute my main site got really busy the LIKE statements were killing it!
    This might be a deficiency in mySQL but if you have a 100 like request executing at the same time you could find the database grinding to a halt.
    All I'm saying is load test (if you haven't done so already) before you release it otherwise you will be panic buying servers!

    Another suggestion I would have is for you to use different servers for the standard CSV & XML downloads and the "Shop Window" type functionality. The later will be much more demanding on the machines as the requests will come in everytime a page on an affiliates web site is displayed.
    Chris Young


  13. #13
    Registered User

    Status
    Offline
    Join Date
    Aug 2003
    Posts
    2,448
    Thanks
    0
    Thanked 0 Times in 0 Posts
    I took "SQL Like Queries" to mean queries like SQL rather than using queries with LIKE conditions - as LIKE "%something%" is an absolute database killer.

    Of topic, but you could create your own index tables to try to avoid avoid having to do a LIKE "%something%" condition. At a minimum, if your product id 1 has name 'big blue widgets' you could create another table that has: -
    1 | big blue widgets
    1 | blue widgets
    1 | widgets

    Then you could create an index on this new table and do a LIKE "something%" query - LIKE checks that don't start with a % can use an index. I haven't actually needed to do this, but I expect that, even with the extra index keys to check, this would be an improvement over a %..% search. Fulltext indexes can also sometimes work out quite well.

  14. #14
    Supercod's Avatar
    Super Moderator

    Status
    Offline
    Join Date
    Jul 2003
    Location
    Scotland, UK.
    Posts
    3,658
    Thanks
    14
    Thanked 26 Times in 12 Posts
    We are not going to release it until we are happy we can fully cope with the demand, I personally expect this system to generate a lot of extra income for everyone, and would be daft not to reinvest in more servers as we need them.

    The systems we have in place means the product feed functionality can work over multiple servers and even different hosting locations, and the plan would be to add more capacity as we need it, and yes panic buying is something we will want to avoid.

    This is a big project and one we hope will give us an edge.
    Clarke - On Twitter @ClarkeDuncan

    Check out my Blog at www.affiliatemarketingblog.co.uk

  15. #15
    Driving to win

    Status
    Offline
    Join Date
    Aug 2003
    Location
    If I'm not at home, I'm in hospital
    Posts
    7,370
    Thanks
    7
    Thanked 16 Times in 11 Posts
    Agreed that like '%x%' will put a large load on the server

    I find using mysql that fulltext searches are considerably quicker than like '%x%' but the downside is that adding records in to the database is much slower - which needs some careful thinking around so you dont lock the whole table out to others doing select statements while you are updating....
    Never argue with idiots. They just drag you down to their level and then beat you with their experience.

    If ignorance is bliss then some of the people I know must be orgasmic.

+ Reply to Thread
Page 1 of 2 1 2 LastLast


Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
To Top

Content Relevant URLs by vBSEO 3.5.0 RC2