# Product Feed Collector - Affiliate Feeds to One Schema (`digivibetech/product-feed-collector`) Actor Merge Daisycon, Awin, TradeTracker, bol partner and Google Shopping feeds (CSV, XML, JSON) into one schema: EAN dedupe, category mapping, price/stock change detection, free 30/90-day price history with drop flag, optional free cross-shop EAN match export, CSV/JSON exports. - **URL**: https://apify.com/digivibetech/product-feed-collector.md - **Developed by:** [DigiVibeTech](https://apify.com/digivibetech) (community) - **Categories:** E-commerce, Developer tools - **Security:** Limited permissions - **Stats:** 2 total users, 1 monthly users, 55.6% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing $1.00 / 1,000 product delivereds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Product Feed Collector Turn the product feeds from your affiliate networks and shops into one clean product list. Daisycon, Awin, TradeTracker, bol partner feeds, Google Shopping feeds and any other CSV, XML or JSON feed come out with the same fields, so a comparison site, a shop or a spreadsheet can use them without per-network code. ### Why this one - Reads affiliate feeds from Awin, Daisycon and TradeTracker as well as Google Shopping and bol partner files. The bol scrapers with the most users on the Store scrape bol pages instead and do not read these networks' feeds. - $0.001 per product, and with `outputMode: changes` unchanged products are free. The three most used bol scrapers charge $0.0015 to $0.003 per product, and two add a start fee. - Dedupes by EAN across shops, maps categories with your own rules and writes `PRODUCTS.csv`, `PRODUCTS.json` and an import file in every run. - Feed keys in URLs are masked in logs, errors and output. ### Screenshots From a real Apify run on a test product feed (made-up sample products), not a live affiliate network feed. ![input](https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/product-feed-collector-01-input.png?signature=1gA2xuFJHkylbrO30c4dE) ![results table](https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/product-feed-collector-02-results-table.png?signature=ePT77YCACY2WppCzGzR) ![result item](https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/product-feed-collector-03-result-item.png?signature=15KjXAY9l3yxSCOybEWmh) ![run cost](https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/product-feed-collector-04-run-cost.png?signature=1Mm6OGpZYI6t3GKfTVymT) ### What you get per product | Field | Meaning | |---|---| | `title`, `description` | Cleaned of HTML tags and extra spaces | | `price`, `currency` | Lowest of price and sale price; "1.299,00", "1,299.00" and "12.99 EUR" are all read correctly | | `gtin`, `gtin14` | EAN/GTIN with the check digit verified. Spreadsheet damage such as `8.71E+12` and in-store codes are rejected and kept in `gtinRejected` | | `brand`, `images`, `category` | Up to 10 image links; category path as in the feed | | `categoryMapped` | Your own category, from the mapping rules you give | | `inStock`, `stockQuantity` | From any stock style: 0/1 flags, numbers, "in_stock", "op voorraad", schema.org values | | `deeplink` | The affiliate link from the feed | | `merchant`, `merchantKey` | Shop name and a stable key, so "Coolblue" and "coolblue.nl" match | | `shippingCost`, `lastUpdated` | When the feed has them | | `sourceNetwork`, `sourceType`, `licenceStatus` | Where the row came from; feed rows are `affiliate-feed` | | `changeType`, `key` | `new`, `priceChanged`, `stockChanged`, `priceAndStockChanged`, `changed`, `unchanged` or `removed`, with a stable key per product | | `changeFields`, `previousPrice`, `previousCurrency`, `priceDelta`, `priceDeltaPct`, `previousInStock` | What changed since the last run, the old price and stock, and the price difference (empty when not applicable) | | `low30`, `high30`, `low90`, `high90` | Lowest and highest price of this product in the last 30 and 90 days, from your own earlier runs (same currency only; empty when there is no history yet) | | `dropFlag` | `true` when today's price is below the lowest price of the previous 30 days | | `historyPoints`, `inStockDays30` | How many daily price points are stored for the product, and on how many of the last 30 days it was in stock | ### Features - **Every common feed format.** CSV and TSV with any delimiter and encoding, XML (also Google Shopping RSS/Atom, TradeTracker properties and Daisycon nesting), JSON and JSON lines, each also as gzip or zip. Columns are recognised automatically; `fieldMap` covers unusual names. Tradedoubler, CJ, Admitad and impact.com feeds are accepted as `network` values (or detected from the feed host with `auto`) once you have a publisher account; use `fieldMap` for any column the auto-detection misses. - **Dedupe by EAN.** `perMerchant` keeps one row per EAN per shop. `onePerGtin` keeps the best offer (in stock first, then the lowest price including shipping, then the most recent) and lists all shop offers in `offers`, ready for a price comparison table. - **Category mapping.** Rules such as `{"match": "kettle", "category": "Kitchen > Kettles"}` or a `/regex/`, first match wins. - **Delta updates.** With `outputMode: changes` you only get new and changed products. Unchanged products are skipped and not charged. Products that left a feed come as free `removed` rows. When a feed fails, its products are never marked removed. Price and stock changes get their own types: `priceChanged` (price or currency only), `stockChanged` (availability only), `priceAndStockChanged` (both) and `changed` (any other field, such as title or image). `changeFields` lists the changed fields (e.g. `price|inStock`), and `previousPrice`, `priceDelta`, `priceDeltaPct` and `previousInStock` show the old values. No price difference is calculated when the currency changed. The first run after upgrading from 1.0.7 marks changed products as `changed` once, because the old state has no stored price and stock. - **Price history (free).** Every run remembers each product's price and stock per day, keyed like delta mode (per product key / EAN + merchant), for the last 90 days (max 60 points per product). Rows get `low30`, `high30`, `low90`, `high90`, `dropFlag`, `historyPoints` and `inStockDays30`. History is built only from your own runs of your own feeds; nothing is crawled. It is stored next to the delta state (`HISTORY-`) and kept under 4 MB; when full, the products not seen for the longest time are dropped. The run summary adds `priceDrops` and `history {keys, droppedKeys}`. - **Cross-shop match by EAN (optional, free).** Set `crossShopMatch: true` (default off) to group this run's products by valid EAN/GTIN across shops and save `MATCHES.json` and `MATCHES.csv` in the run's key-value store: gtin, title, offerCount, cheapestPrice (incl. shipping), cheapestMerchant, cheapestTie, spreadCents, spreadPct and all offers sorted by price (ties: in stock first). Invalid and restricted (in-store/coupon/internal) barcodes are left out; one best offer per shop. No extra dataset rows and no extra charge. - **Exports.** Besides the dataset, every run writes `PRODUCTS.csv` (opens in Excel and Google Sheets), `PRODUCTS.json`, `CHANGES.csv` (all new, changed and removed rows plus one summary row with the counts) and `EDGE-PORTAL.ndjson`, an import file (format 1.1) with upsert and discontinue lines, prices in minor units and stable product ids. - **Feed keys stay private.** Feed URLs often contain your account key. Logs, errors and output only show a masked form such as `productdata.awin.com/datafeed/download/apikey/…/fid/…`. - **Optional direct mode.** For a shop without a feed, `productUrls` reads only the schema.org Product data (JSON-LD) the shop publishes on public product pages. Only when robots.txt allows it, at most one request per 2 seconds per site, no logins. Seller names of private persons and review authors are never taken. These rows are marked `needs licence or affiliate approval`: check that you may reuse the data before you publish it. ### Input example ```json { "feeds": [ {"url": "https://productdata.awin.com/datafeed/download/apikey/YOURKEY/...", "name": "Awin kitchen", "network": "awin"}, {"url": "https://daisycon.io/datafeed/?...", "network": "daisycon"}, {"url": "https://pf.tradetracker.net/?...", "network": "tradetracker", "merchant": "Garden Shop"} ], "dedupeMode": "onePerGtin", "outputMode": "changes", "stateKey": "kitchen-site", "crossShopMatch": true, "categoryMap": [{"match": "kettle", "category": "Kitchen > Kettles"}] } ``` Schedule the same input daily and each run only returns what changed since the last run. ### Example output (real run, 2026-10-26 UTC) From a real run of this actor on a small test Google Merchant feed with made-up sample products (5 products in, 5 rows from an earlier test state returned as free `removed` rows). `removed` rows are free; only the 5 delivered products are billed. Three records max, long fields shortened: ```json [ { "id": "PF-1", "title": "Nordic Kettle 1.7L", "description": "Nordic Kettle 1.7L", "price": 24, "currency": "EUR", "gtin": "8712345000011", "gtin14": "08712345000011", "brand": "Sample", "images": [ "https://sample-shop.example/img/pf-1.jpg" ], "category": "Home > Kitchen", "categoryMapped": null, "inStock": true, "stockQuantity": null, "deeplink": "https://sample-shop.example/p/pf-1", "merchant": "Sample Shop", "merchantKey": "sampleshop", "shippingCost": null, "lastUpdated": null, "sourceName": "Sample Google feed", "sourceNetwork": "google", "sourceType": "feed", "licenceStatus": "affiliate-feed", "externalId": "PF-1", "sku": "PF-1", "listPrice": 26, "salePrice": 24, "mpn": null, "condition": "new", "language": "nl-NL", "country": "NL", "key": "eeaddb36a5c5fdcd1a5f", "hash": "085eca133111103e", "changeType": "new", "collectedAt": "2026-10-26T12:17:14+00:00" }, { "id": "PF-2", "title": "Desk Lamp LED", "description": "Desk Lamp LED", "price": 39.95, "currency": "EUR", "gtin": "8712345000028", "gtin14": "08712345000028", "brand": "Sample", "images": [ "https://sample-shop.example/img/pf-2.jpg" ], "category": "Home > Kitchen", "categoryMapped": null, "inStock": true, "stockQuantity": null, "deeplink": "https://sample-shop.example/p/pf-2", "merchant": "Sample Shop", "merchantKey": "sampleshop", "shippingCost": null, "lastUpdated": null, "sourceName": "Sample Google feed", "sourceNetwork": "google", "sourceType": "feed", "licenceStatus": "affiliate-feed", "externalId": "PF-2", "sku": "PF-2", "listPrice": 39.95, "salePrice": null, "mpn": null, "condition": "new", "language": "nl-NL", "country": "NL", "key": "ad3977f1ff8a020928d8", "hash": "0cb0fac74894a4f3", "changeType": "new", "collectedAt": "2026-10-26T12:17:14+00:00" }, { "id": "PF-3", "title": "Steel Water Bottle 750ml", "description": "Steel Water Bottle 750ml", "price": 14.5, "currency": "EUR", "gtin": "8712345000035", "gtin14": "08712345000035", "brand": "Sample", "images": [ "https://sample-shop.example/img/pf-3.jpg" ], "category": "Home > Kitchen", "categoryMapped": null, "inStock": false, "stockQuantity": null, "deeplink": "https://sample-shop.example/p/pf-3", "merchant": "Sample Shop", "merchantKey": "sampleshop", "shippingCost": null, "lastUpdated": null, "sourceName": "Sample Google feed", "sourceNetwork": "google", "sourceType": "feed", "licenceStatus": "affiliate-feed", "externalId": "PF-3", "sku": "PF-3", "listPrice": 14.5, "salePrice": null, "mpn": null, "condition": "new", "language": "nl-NL", "country": "NL", "key": "4fa6daecb48c5cf618ef", "hash": "3ea44e69a7ff57f8", "changeType": "new", "collectedAt": "2026-10-26T12:17:14+00:00" } ] ``` Full sample from the same run (≤50 rows): [JSON](https://digivibetech.com/samples/product-feed-collector/sample.json) · [CSV](https://digivibetech.com/samples/product-feed-collector/sample.csv) **Try it:** run with `{"feeds":[{"url":"https://www.yourshop.nl/google-feed.xml","network":"google"}],"outputMode":"all"}`. That is 5 products × $0.001 (`product` event) = **$0.0050** (about $0.01). Apify's free plan includes $5 of prepaid usage every month, so this run fits in it many times over. ### Pricing **$0.001 per product delivered** ($1 per 1,000 products). Try it: start a free run in Apify with your own feed URL or any sample feed URL. No start fee. Only product rows in the dataset are charged. Not charged: the summary row in `CHANGES.csv`, the price history fields, `MATCHES.json` / `MATCHES.csv` from `crossShopMatch`, rows skipped as unchanged in delta mode, `removed` rows, rows dropped by your filters, rows without a title or price, and failed feeds or pages. The run stops delivering when your maximum cost per run is reached and tells you how many products were left out. ### Limits - You need your own feed URLs from the networks. This actor does not log in to Awin, Daisycon, TradeTracker or bol and does not create affiliate links. - The bol Marketing Catalog API (OAuth) is not called; use the bol partner product feed file instead. - One feed download is capped at 500 MB and 1,000,000 rows (`maxItemsPerFeed`). Very large XML feeds need more memory; use 2 GB or more for feeds over 100 MB. - Direct mode reads JSON-LD only. Pages that build their product data with scripts after loading, or that block automated access, are reported and skipped. Access blocks are never bypassed. - Delta state is kept in a named key-value store in your account (`product-feed-collector-state`), one record per `stateKey`. - Currency is not converted. ### Support Questions or a feed that is not read correctly: https://digivibetech.com/support/ # Actor input Schema ## `feeds` (type: `array`): One object per feed: {"url": "...", "name": "My Awin feed", "network": "auto|daisycon|awin|tradetracker|bol|google|tradedoubler|cj|admitad|impact|generic", "format": "auto|csv|xml|json", "merchant": "Shop name if the feed has none", "currency": "EUR", "fieldMap": {"gtin": "EAN code"}}. CSV/TSV, XML (incl. Google Shopping RSS/Atom), JSON and JSON lines, also gzip or zip. Feed URLs are treated as secrets: only masked forms appear in logs and output. ## `productUrls` (type: `array`): Public product pages. Only the schema.org Product data (JSON-LD) the shop publishes is read, only when robots.txt allows it, at most one request per 2 seconds per site. Such rows are marked licenceStatus 'needs licence or affiliate approval'. ## `dedupeMode` (type: `string`): perMerchant keeps the best offer per shop per EAN; onePerGtin keeps the best offer overall (in stock first, then lowest price incl. shipping) and lists the other shops in 'offers'. Rows without a valid EAN are always kept. ## `outputMode` (type: `string`): In delta mode, unchanged products are skipped and not charged, and products that left the feed are listed as 'removed' for free. ## `stateKey` (type: `string`): Name for the stored product fingerprints used by delta mode. Use a different name per job. Letters, digits, - and \_. ## `crossShopMatch` (type: `boolean`): Group this run's products by valid EAN/GTIN across merchants and save MATCHES.json and MATCHES.csv in the run's key-value store: offer count, cheapest price and merchant, price spread and all offers sorted by price. Invalid or restricted (in-store/coupon) barcodes are left out. Not charged: no extra dataset rows. ## `categoryMap` (type: `array`): Rules applied in order, first match wins: \[{"match": "kettle", "category": "Kitchen > Kettles"}, {"match": "/lamp|verlichting/", "category": "Home > Lighting"}]. Plain text matches anywhere (case-insensitive); /.../ is a regular expression. The result is in 'categoryMapped'. ## `fieldMap` (type: `object`): Only needed for unusual column names, e.g. {"gtin": "streepjescode", "price": "verkoopprijs"}. Keys: id, title, description, price, sale_price, currency, gtin, brand, images, category, stock, deeplink, merchant, updated, shipping. ## `requireGtin` (type: `boolean`): Drop rows without a valid EAN/GTIN (they are not charged). ## `onlyInStock` (type: `boolean`): Drop rows the feed marks as out of stock (they are not charged). ## `emitRemoved` (type: `boolean`): In later runs, add a free 'removed' row for every product that is no longer in the feeds. Skipped automatically when a feed failed. ## `defaultCurrency` (type: `string`): Used when neither the feed nor the price says a currency. ## `maxItemsPerFeed` (type: `integer`): Stop reading a feed after this many rows. ## `shopId` (type: `string`): Optional. Written as shopId on every line of EDGE-PORTAL.ndjson so the import lands in the right shop. ## `defaultLanguage` (type: `string`): BCP-47 language of titles and descriptions when a feed does not set one, e.g. nl-NL. ## `defaultCountry` (type: `string`): Two-letter country for prices and shipping when a feed does not set one, e.g. NL. ## Actor input object example ```json { "feeds": [ { "url": "https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/canary-product-feed.xml?signature=3YOo0lhW1M4iBc3ePb6R", "name": "Sample Google feed", "network": "google", "merchant": "Sample Shop" } ], "productUrls": [], "dedupeMode": "none", "outputMode": "all", "stateKey": "default", "crossShopMatch": false, "categoryMap": [], "fieldMap": {}, "requireGtin": false, "onlyInStock": false, "emitRemoved": true, "defaultCurrency": "EUR", "maxItemsPerFeed": 100000, "defaultLanguage": "nl-NL", "defaultCountry": "NL" } ``` # Actor output Schema ## `products` (type: `string`): No description ## `csv` (type: `string`): No description ## `json` (type: `string`): No description ## `changesCsv` (type: `string`): No description ## `edge` (type: `string`): No description ## `matchesJson` (type: `string`): No description ## `matchesCsv` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "feeds": [ { "url": "https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/canary-product-feed.xml?signature=3YOo0lhW1M4iBc3ePb6R", "name": "Sample Google feed", "network": "google", "merchant": "Sample Shop" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("digivibetech/product-feed-collector").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "feeds": [{ "url": "https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/canary-product-feed.xml?signature=3YOo0lhW1M4iBc3ePb6R", "name": "Sample Google feed", "network": "google", "merchant": "Sample Shop", }] } # Run the Actor and wait for it to finish run = client.actor("digivibetech/product-feed-collector").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "feeds": [ { "url": "https://api.apify.com/v2/key-value-stores/8dx7mDMr71NbvtKdp/records/canary-product-feed.xml?signature=3YOo0lhW1M4iBc3ePb6R", "name": "Sample Google feed", "network": "google", "merchant": "Sample Shop" } ] }' | apify call digivibetech/product-feed-collector --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,digivibetech/product-feed-collector" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/b3TDtj2yhIng0UAQi/builds/ObfNj6lrPeVI0xt5K/openapi.json