# Shift4Shop Scraper - 3dcart Product Catalog (`clintsa/shift4shop-product-catalog-extractor`) Actor

Export product catalogs from public Shift4Shop and legacy 3dcart stores. Export JSON or CSV and monitor changes. $0.75 per 1,000 rows plus $0.001 per run.

- **URL**: https://apify.com/clintsa/shift4shop-product-catalog-extractor.md
- **Developed by:** [Andy Besos](https://apify.com/clintsa) (community)
- **Categories:** E-commerce, Automation
- **Stats:** 2 total users, 1 monthly users, 95.2% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.75 / 1,000 shift4shop catalog rows

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

Shift4Shop Scraper - 3dcart Product Catalog helps you export product catalogs from public Shift4Shop and legacy 3dcart stores. Collect product IDs, names, prices, currency and availability; optional RSS rows show on-sale, new-release and bestseller feed membership.

Enter a product, category or storefront URL. The Actor reads public pages, sitemaps and optional RSS feeds without a store account or merchant API key. Actor event charges at the default 256 MB memory are **$0.001 per run plus $0.00075 per exported row**: **$0.751 for 1,000 rows**.

Selected fields from a recent successful run; source values can change:

```json
{
  "id": "19",
  "name": "Hope Sunglasses",
  "price": 3500,
  "currency": "USD",
  "availability": "in_stock",
  "source": "product"
}
```

### Why use this Shift4Shop catalog extractor?

- **Product details:** capture `id`, `catalogId`, `price`, `currency`, `availability` and the public `inventoryItem` object from product pages. Standard page microdata provides a fallback for key fields.
- **Catalog discovery:** follow category links from the home page up to the configured depth and add product links found in the store sitemap.
- **Feed context:** optional RSS requests add separate `source: "rss"` rows. Product rows carry `onSale`, `newRelease` and `bestSeller` flags when feeds are checked.
- **Change history:** reuse a Monitor ID in scheduled runs to see new, updated and removed rows, including changes in public price or availability.
- **Small example run:** paste one product URL to inspect a single page before crawling a whole storefront.

Transient HTTP errors are retried. A failed product page is logged, and the run can still return other products; incomplete crawls do not emit removal events for missing pages.

### Pricing

The price is **$0.001 per run plus $0.00075 per exported row** at the default 256 MB memory. A product row and an RSS row each count as a row. The maximum-row setting limits billed dataset rows, although catalog discovery can still make requests to build a monitoring snapshot.

| Exported rows | Estimated charge |
| ---: | ---: |
| 1 | $0.00175 |
| 100 | $0.076 |
| 1,000 | $0.751 |
| 10,000 | $7.501 |

These figures use one start charge and exclude any unrelated account costs. A run with a different memory setting can have a different start charge.

### How to use this Actor

For a quick product check, use a direct product page:

```json
{
  "targets": [
    "https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html"
  ],
  "maxItemsPerTarget": 1
}
```

Click **Start**, inspect the Dataset, then export JSON or CSV. Start with RSS disabled to inspect one product row. When you enable RSS feeds, source and rssType distinguish feed records from product records; both exported row types are billable.

For the demo storefront catalog and its three RSS feeds:

```json
{
  "targets": ["https://standard-preview-com.3dcartstores.com/"],
  "categoryDepth": 2,
  "includeRss": true,
  "maxItemsPerTarget": 0
}
```

You can also enter a category link such as `https://standard-preview-com.3dcartstores.com/Men_c_1.html` or a bare store hostname. The Actor normalizes those inputs to a public target URL and only follows links on the same host.

### Monitor new, updated, and removed products

Use at most 18 characters in `monitorId` to keep independent schedules distinct. Longer values are shortened for the monitor snapshot.

Set a stable `monitorId` and schedule the same store with the same crawl settings. Each row has a `changeType`:

| Value | Meaning |
| --- | --- |
| `new` | First observation of this product or feed entry |
| `updated` | A monitored field changed since the last run |
| `unchanged` | No monitored field changed |
| `removed` | A record disappeared during a complete crawl |

Use `onlyChangesSince` with an ISO 8601 timestamp to export just changes after that time. Keep the same target and `includeRss` setting across scheduled runs. Product and RSS rows have distinct stable `itemId` values. The Actor keeps unseen prior records in its snapshot when a page fails or a crawl cap is reached, so a partial crawl does not invent removals.

### API example

```bash
curl "https://api.apify.com/v2/acts/Clintsa~shift4shop-product-catalog-extractor/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"targets":["https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html"],"includeRss":false,"maxItemsPerTarget":1}'
```

The synchronous endpoint is best for a one-product run. For larger storefronts, start an asynchronous Actor run and fetch its dataset after completion.

### Input options

| Option | What it controls |
| --- | --- |
| `targets` | One or more public store, category or product URLs |
| `maxItemsPerTarget` | Export cap per target; `0` exports every observed row |
| `includeRss` | Read on-sale, new-release and bestseller feeds and emit RSS rows |
| `categoryDepth` | Number of category link levels to follow; default `2` |
| `maxProductPages` | Product page request cap; `0` means no cap |
| `maxCategoryPages` | Category/listing request cap; `0` means no cap |
| `requestDelayMillis` | Pause between requests to a storefront; default `200` ms |
| `monitorId` | Snapshot namespace shared by repeated runs |
| `onlyChangesSince` | Optional timestamp filter for new, updated and removed rows |

### Output

A product row from the public Shift4Shop demo looks like this (timestamps vary):

```json
{
  "target": "https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html",
  "itemId": "product:19",
  "id": "19",
  "catalogId": "19",
  "name": "Hope Sunglasses",
  "price": 3500,
  "currency": "USD",
  "availability": "in_stock",
  "inventoryItem": {"catalogid": 19, "id": "19", "name": "Hope Sunglasses", "price": 3500, "availability": 1, "currency": "USD"},
  "source": "product",
  "rssType": null,
  "productUrl": "https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html",
  "changeType": "new"
}
```

RSS rows have `source: "rss"` and an `rssType` of `onsale`, `newreleases` or `bestsellers`. If their product page was also found, they include its price and inventory object. The `SUMMARY` key-value record reports requested and successful targets, product and RSS counts, exported rows and whether a crawl was incomplete.

### Use cases

- Compare public Shift4Shop prices and stock across merchant storefronts.
- Build a recurring catalog snapshot for wholesale or dropship sourcing.
- Watch products entering or leaving a store's public RSS promotion feeds.
- Audit whether visible product pages show the expected catalog IDs and prices.
- Feed price and availability changes into a dashboard or alert workflow.

### FAQ

**Does it need a Shift4Shop account?** No. It reads public storefront pages and feeds only.

**Will it work on every Shift4Shop theme?** It handles the `_3d_item` pattern and common product microdata, but a customized theme may expose different markup or no product links.

**What if a store does not offer RSS?** Product rows can still be returned. The summary marks a failed feed request as an incomplete crawl; disable `includeRss` if the store has no feeds.

**Will the Actor discover every product?** It follows visible category links and a sitemap when available. Hidden, unlinked or access-restricted products cannot be guaranteed.

**Does availability mean an exact inventory count?** No. It reflects the public availability indicator, not the merchant's private stock ledger.

**Can it use a proxy or bypass a block?** No. It does not use a browser, proxy or access-bypass step. It respects the storefront's `robots.txt` rules.

### Limitations and responsible use

The Actor only reads publicly served data and cannot access private inventory, customer details, admin APIs or checkout pages. RSS flags show membership in the feed at observation time; they do not establish historical promotions. Each store can have its own robots rules and terms. [Shift4Shop publishes service terms](https://www.shift4.com/pdf/Shift4Shop-Terms-and-Conditions.pdf) that restrict automated crawling or scraping of its Services; check which terms and permissions apply before using this Actor on a store.

### Support

Use the Actor's Issues tab and include the run ID, storefront URL and the product page or feed that behaved unexpectedly. Do not include account credentials.

Related tools: [Sitemap Scraper - XML URL Extractor & Monitor](https://apify.com/clintsa/sitemap-scraper).

# Actor input Schema

## `targets` (type: `array`):

Public Shift4Shop/3dcart store home, category or product URLs; hostnames also work. A product URL makes the daily example a one-product run.

## `maxItemsPerTarget` (type: `integer`):

Limit exported dataset rows per storefront; 0 returns all. Monitoring still compares every fetched product. The daily example exports one row.

## `includeRss` (type: `boolean`):

Read on-sale, new-release and bestseller RSS feeds and emit separate rows for feed entries; also add feed flags to product rows.

## `categoryDepth` (type: `integer`):

Follow nested category links from the storefront home page up to this depth; the sitemap is also checked for product URLs.

## `maxProductPages` (type: `integer`):

Cap product detail requests per target; 0 visits every discovered product. Capped runs suppress removal events.

## `maxCategoryPages` (type: `integer`):

Cap category and listing page requests; 0 has no category cap. Capped runs suppress removal events.

## `requestDelayMillis` (type: `integer`):

Pause between storefront requests to reduce load on the merchant site.

## `monitorId` (type: `string`):

Persistent snapshot namespace for comparing scheduled runs of the same storefront and settings.

## `onlyChangesSince` (type: `string`):

Optional ISO 8601 timestamp; return only new, updated or removed records detected since that time.

## Actor input object example

```json
{
  "targets": [
    "https://standard-preview-com.3dcartstores.com/"
  ],
  "maxItemsPerTarget": 1,
  "includeRss": true,
  "categoryDepth": 2,
  "maxProductPages": 0,
  "maxCategoryPages": 0,
  "requestDelayMillis": 200,
  "monitorId": "default"
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html"
    ],
    "maxItemsPerTarget": 1
};

// Run the Actor and wait for it to finish
const run = await client.actor("clintsa/shift4shop-product-catalog-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "targets": ["https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html"],
    "maxItemsPerTarget": 1,
}

# Run the Actor and wait for it to finish
run = client.actor("clintsa/shift4shop-product-catalog-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "https://standard-preview-com.3dcartstores.com/Hope-Sunglasses_p_19.html"
  ],
  "maxItemsPerTarget": 1
}' |
apify call clintsa/shift4shop-product-catalog-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,clintsa/shift4shop-product-catalog-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/leiTT3UmqVu05Utza/builds/DSsFEKgK3Ol0JWz6b/openapi.json