Product Feed Collector

Turn the product feeds from your affiliate networks and shops into one clean product list. Daisycon, Awin, TradeTracker, bol partner feeds, Google Shopping feeds and any other CSV, XML or JSON feed come out with the same fields, so a comparison site, a shop or a spreadsheet can use them without per-network code.

Why this one

• Reads affiliate feeds from Awin, Daisycon and TradeTracker as well as Google Shopping and bol partner files. The bol scrapers with the most users on the Store scrape bol pages instead and do not read these networks' feeds.
• $0.001 per product, and with outputMode: changes unchanged products are free. The three most used bol scrapers charge $0.0015 to $0.003 per product, and two add a start fee.
• Dedupes by EAN across shops, maps categories with your own rules and writes PRODUCTS.csv, PRODUCTS.json and an import file in every run.
• Feed keys in URLs are masked in logs, errors and output.

Screenshots

From a real Apify run on a test product feed (made-up sample products), not a live affiliate network feed.

What you get per product

Field	Meaning
title, description	Cleaned of HTML tags and extra spaces
price, currency	Lowest of price and sale price; "1.299,00", "1,299.00" and "12.99 EUR" are all read correctly
gtin, gtin14	EAN/GTIN with the check digit verified. Spreadsheet damage such as 8.71E+12 and in-store codes are rejected and kept in gtinRejected
brand, images, category	Up to 10 image links; category path as in the feed
categoryMapped	Your own category, from the mapping rules you give
inStock, stockQuantity	From any stock style: 0/1 flags, numbers, "in_stock", "op voorraad", schema.org values
deeplink	The affiliate link from the feed
merchant, merchantKey	Shop name and a stable key, so "Coolblue" and "coolblue.nl" match
shippingCost, lastUpdated	When the feed has them
sourceNetwork, sourceType, licenceStatus	Where the row came from; feed rows are affiliate-feed
changeType, key	new, priceChanged, stockChanged, priceAndStockChanged, changed, unchanged or removed, with a stable key per product
changeFields, previousPrice, previousCurrency, priceDelta, priceDeltaPct, previousInStock	What changed since the last run, the old price and stock, and the price difference (empty when not applicable)
low30, high30, low90, high90	Lowest and highest price of this product in the last 30 and 90 days, from your own earlier runs (same currency only; empty when there is no history yet)
dropFlag	true when today's price is below the lowest price of the previous 30 days
historyPoints, inStockDays30	How many daily price points are stored for the product, and on how many of the last 30 days it was in stock

Features

• Every common feed format. CSV and TSV with any delimiter and encoding, XML (also Google Shopping RSS/Atom, TradeTracker properties and Daisycon nesting), JSON and JSON lines, each also as gzip or zip. Columns are recognised automatically; fieldMap covers unusual names. Tradedoubler, CJ, Admitad and impact.com feeds are accepted as network values (or detected from the feed host with auto) once you have a publisher account; use fieldMap for any column the auto-detection misses.
• Dedupe by EAN. perMerchant keeps one row per EAN per shop. onePerGtin keeps the best offer (in stock first, then the lowest price including shipping, then the most recent) and lists all shop offers in offers, ready for a price comparison table.
• Category mapping. Rules such as {"match": "kettle", "category": "Kitchen > Kettles"} or a /regex/, first match wins.
• Delta updates. With outputMode: changes you only get new and changed products. Unchanged products are skipped and not charged. Products that left a feed come as free removed rows. When a feed fails, its products are never marked removed. Price and stock changes get their own types: priceChanged (price or currency only), stockChanged (availability only), priceAndStockChanged (both) and changed (any other field, such as title or image). changeFields lists the changed fields (e.g. price|inStock), and previousPrice, priceDelta, priceDeltaPct and previousInStock show the old values. No price difference is calculated when the currency changed. The first run after upgrading from 1.0.7 marks changed products as changed once, because the old state has no stored price and stock.
• Price history (free). Every run remembers each product's price and stock per day, keyed like delta mode (per product key / EAN + merchant), for the last 90 days (max 60 points per product). Rows get low30, high30, low90, high90, dropFlag, historyPoints and inStockDays30. History is built only from your own runs of your own feeds; nothing is crawled. It is stored next to the delta state (HISTORY-<stateKey>) and kept under 4 MB; when full, the products not seen for the longest time are dropped. The run summary adds priceDrops and history {keys, droppedKeys}.
• Cross-shop match by EAN (optional, free). Set crossShopMatch: true (default off) to group this run's products by valid EAN/GTIN across shops and save MATCHES.json and MATCHES.csv in the run's key-value store: gtin, title, offerCount, cheapestPrice (incl. shipping), cheapestMerchant, cheapestTie, spreadCents, spreadPct and all offers sorted by price (ties: in stock first). Invalid and restricted (in-store/coupon/internal) barcodes are left out; one best offer per shop. No extra dataset rows and no extra charge.
• Exports. Besides the dataset, every run writes PRODUCTS.csv (opens in Excel and Google Sheets), PRODUCTS.json, CHANGES.csv (all new, changed and removed rows plus one summary row with the counts) and EDGE-PORTAL.ndjson, an import file (format 1.1) with upsert and discontinue lines, prices in minor units and stable product ids.
• Feed keys stay private. Feed URLs often contain your account key. Logs, errors and output only show a masked form such as productdata.awin.com/datafeed/download/apikey/…/fid/….
• Optional direct mode. For a shop without a feed, productUrls reads only the schema.org Product data (JSON-LD) the shop publishes on public product pages. Only when robots.txt allows it, at most one request per 2 seconds per site, no logins. Seller names of private persons and review authors are never taken. These rows are marked needs licence or affiliate approval: check that you may reuse the data before you publish it.

Input example

{
  "feeds": [
    {"url": "https://productdata.awin.com/datafeed/download/apikey/YOURKEY/...", "name": "Awin kitchen", "network": "awin"},
    {"url": "https://daisycon.io/datafeed/?...", "network": "daisycon"},
    {"url": "https://pf.tradetracker.net/?...", "network": "tradetracker", "merchant": "Garden Shop"}
  ],
  "dedupeMode": "onePerGtin",
  "outputMode": "changes",
  "stateKey": "kitchen-site",
  "crossShopMatch": true,
  "categoryMap": [{"match": "kettle", "category": "Kitchen > Kettles"}]
}

Schedule the same input daily and each run only returns what changed since the last run.

Example output (real run, 2026-10-26 UTC)

From a real run of this actor on a small test Google Merchant feed with made-up sample products (5 products in, 5 rows from an earlier test state returned as free removed rows). removed rows are free; only the 5 delivered products are billed. Three records max, long fields shortened:

[
  {
    "id": "PF-1",
    "title": "Nordic Kettle 1.7L",
    "description": "Nordic Kettle 1.7L",
    "price": 24,
    "currency": "EUR",
    "gtin": "8712345000011",
    "gtin14": "08712345000011",
    "brand": "Sample",
    "images": [
      "https://sample-shop.example/img/pf-1.jpg"
    ],
    "category": "Home > Kitchen",
    "categoryMapped": null,
    "inStock": true,
    "stockQuantity": null,
    "deeplink": "https://sample-shop.example/p/pf-1",
    "merchant": "Sample Shop",
    "merchantKey": "sampleshop",
    "shippingCost": null,
    "lastUpdated": null,
    "sourceName": "Sample Google feed",
    "sourceNetwork": "google",
    "sourceType": "feed",
    "licenceStatus": "affiliate-feed",
    "externalId": "PF-1",
    "sku": "PF-1",
    "listPrice": 26,
    "salePrice": 24,
    "mpn": null,
    "condition": "new",
    "language": "nl-NL",
    "country": "NL",
    "key": "eeaddb36a5c5fdcd1a5f",
    "hash": "085eca133111103e",
    "changeType": "new",
    "collectedAt": "2026-10-26T12:17:14+00:00"
  },
  {
    "id": "PF-2",
    "title": "Desk Lamp LED",
    "description": "Desk Lamp LED",
    "price": 39.95,
    "currency": "EUR",
    "gtin": "8712345000028",
    "gtin14": "08712345000028",
    "brand": "Sample",
    "images": [
      "https://sample-shop.example/img/pf-2.jpg"
    ],
    "category": "Home > Kitchen",
    "categoryMapped": null,
    "inStock": true,
    "stockQuantity": null,
    "deeplink": "https://sample-shop.example/p/pf-2",
    "merchant": "Sample Shop",
    "merchantKey": "sampleshop",
    "shippingCost": null,
    "lastUpdated": null,
    "sourceName": "Sample Google feed",
    "sourceNetwork": "google",
    "sourceType": "feed",
    "licenceStatus": "affiliate-feed",
    "externalId": "PF-2",
    "sku": "PF-2",
    "listPrice": 39.95,
    "salePrice": null,
    "mpn": null,
    "condition": "new",
    "language": "nl-NL",
    "country": "NL",
    "key": "ad3977f1ff8a020928d8",
    "hash": "0cb0fac74894a4f3",
    "changeType": "new",
    "collectedAt": "2026-10-26T12:17:14+00:00"
  },
  {
    "id": "PF-3",
    "title": "Steel Water Bottle 750ml",
    "description": "Steel Water Bottle 750ml",
    "price": 14.5,
    "currency": "EUR",
    "gtin": "8712345000035",
    "gtin14": "08712345000035",
    "brand": "Sample",
    "images": [
      "https://sample-shop.example/img/pf-3.jpg"
    ],
    "category": "Home > Kitchen",
    "categoryMapped": null,
    "inStock": false,
    "stockQuantity": null,
    "deeplink": "https://sample-shop.example/p/pf-3",
    "merchant": "Sample Shop",
    "merchantKey": "sampleshop",
    "shippingCost": null,
    "lastUpdated": null,
    "sourceName": "Sample Google feed",
    "sourceNetwork": "google",
    "sourceType": "feed",
    "licenceStatus": "affiliate-feed",
    "externalId": "PF-3",
    "sku": "PF-3",
    "listPrice": 14.5,
    "salePrice": null,
    "mpn": null,
    "condition": "new",
    "language": "nl-NL",
    "country": "NL",
    "key": "4fa6daecb48c5cf618ef",
    "hash": "3ea44e69a7ff57f8",
    "changeType": "new",
    "collectedAt": "2026-10-26T12:17:14+00:00"
  }
]

Full sample from the same run (≤50 rows): JSON · CSV

Try it: run with {"feeds":[{"url":"https://www.yourshop.nl/google-feed.xml","network":"google"}],"outputMode":"all"}. That is 5 products × $0.001 (product event) = $0.0050 (about $0.01). Apify's free plan includes $5 of prepaid usage every month, so this run fits in it many times over.

Pricing

$0.001 per product delivered ($1 per 1,000 products). Try it: start a free run in Apify with your own feed URL or any sample feed URL. No start fee. Only product rows in the dataset are charged. Not charged: the summary row in CHANGES.csv, the price history fields, MATCHES.json / MATCHES.csv from crossShopMatch, rows skipped as unchanged in delta mode, removed rows, rows dropped by your filters, rows without a title or price, and failed feeds or pages. The run stops delivering when your maximum cost per run is reached and tells you how many products were left out.

Limits

• You need your own feed URLs from the networks. This actor does not log in to Awin, Daisycon, TradeTracker or bol and does not create affiliate links.
• The bol Marketing Catalog API (OAuth) is not called; use the bol partner product feed file instead.
• One feed download is capped at 500 MB and 1,000,000 rows (maxItemsPerFeed). Very large XML feeds need more memory; use 2 GB or more for feeds over 100 MB.
• Direct mode reads JSON-LD only. Pages that build their product data with scripts after loading, or that block automated access, are reported and skipped. Access blocks are never bypassed.
• Delta state is kept in a named key-value store in your account (product-feed-collector-state), one record per stateKey.
• Currency is not converted.

Support

Questions or a feed that is not read correctly: https://digivibetech.com/support/