Replace $250 manual agency fees with $0.05 automated cloud execution

Shopify & WooCommerce Catalog + 301 Redirect Extractor

Extract public product catalogs from accessible Shopify and WooCommerce stores into standardized 32-column Shopify import CSVs, deduplicated 301 URL redirect maps, and structured rejected-record audit reports.

---

⚡ Overview & The Migration Challenge

Re-platforming an e-commerce store between WooCommerce and Shopify or auditing competitor product lines presents significant data handling risks:

• Unmapped URLs & 404 Traffic Drops: Modifying store URL structures without proper redirect mappings causes legacy product URLs to return HTTP 404 errors, breaking customer bookmarks and degrading organic search equity.
• Spreadsheet Formula Execution Risks: Unsanitized product titles or descriptions starting with =, +, -, or @ can trigger dynamic data exchange (DDE) formula execution or syntax errors when imported into spreadsheet programs or e-commerce admin panels.
• Silent Price Zeroing: Comma-separated European decimals or missing variant pricing can silently corrupt product values if parsers coerce unverified strings to 0.00.

How this Actor assists your migration workflow:

1. Connects to publicly exposed store endpoints (/products.json for Shopify, /wp-json/wc/store/v1/products for WooCommerce) without requiring private API credentials or merchant secrets.
2. Normalizes product records into the official 32-column Shopify import CSV specification, structuring variants, options, and gallery images.
3. Automatically maps source URL paths (/product/slug, /?p=123) to canonical Shopify destinations (/products/{handle}) in an RFC 4180 SEO_REDIRECTS.csv file ready for import into Shopify Admin.
4. Quarantines ambiguous or malformed pricing records into REJECTED_RECORDS.json with machine-readable audit reasons rather than guessing or silently zeroing prices.

---

💰 Pricing & Monetization Structure

This Actor is configured under Apify's Pay-Per-Event (PPE) model on the Apify Store.

Platform-Configured Event Catalog

Event Name	Event Identifier	Configured Rate	Platform Role	Charging Mechanism & Status
Store Catalog & 301 Redirect Extraction	store-catalog-extraction	$0.05	Primary Event (isPrimaryEvent: true)	Assessed upon Actor run initialization by the platform.
Exported Product & 301 Redirect Record	product-record-exported	$0.001	Secondary Event (isPrimaryEvent: false)	Configured in PPE settings; not currently invoked by Actor code.

Verified Code Charging Behavior vs. Configured Estimates

An audit of the Actor source code (main.py) confirms:

• Pushes to dataset occur via push_data(batch) without specifying charged_event_name="product-record-exported".
• The Actor contains zero calls to Actor.charge().
• Under Apify's PPE model, secondary/custom events are never charged automatically by the platform without explicit code invocation.
• Consequently, runs on the current Actor build are billed only the $0.05 primary event entry fee upon start, regardless of output record volume.

Product Volume	Verified Actual Event Charge (Current Code)	Conditional Configured Estimate (If Per-Record Charging Is Activated)
100 Products	$0.05 ($0.05 base + $0.00 uncharged records)	$0.15 ($0.05 base + 100 × $0.001) (conditional)
500 Products	$0.05 ($0.05 base + $0.00 uncharged records)	$0.55 ($0.05 base + 500 × $0.001) (conditional)
1,000 Products	$0.05 ($0.05 base + $0.00 uncharged records)	$1.05 ($0.05 base + 1,000 × $0.001) (conditional)
2,500 Products	$0.05 ($0.05 base + $0.00 uncharged records)	$2.55 ($0.05 base + 2,500 × $0.001) (conditional)

[!NOTE] Platform Usage Costs: In addition to Actor event fees, standard Apify platform infrastructure costs (Actor Compute Units at ~$0.20/ACU hour) apply. This lightweight Python Actor typically finishes standard runs in 5–20 seconds with minimal memory consumption (~0.01 ACU / ~$0.002), which is deducted from your monthly platform usage credits ($5.00/month included on the Free plan).

---

🚀 Key Technical Features

• Public Endpoint Extraction: Queries standard publicly exposed catalog feeds (/products.json or /wp-json/wc/store/v1/products) via pure Python HTTP streaming with SSRF protection, bounded redirects, and exponential backoff on HTTP 429/5xx responses.
• 32-Column Shopify Product Hierarchy: Correctly generates parent rows with full titles, HTML descriptions, and tags, followed by indented secondary variant rows containing variant SKUs, option values (Option1 Value, Option2 Value, Option3 Value), and gallery image URLs.
• Automated 301 Redirect Generation: Extracts legacy URLs, decodes nested URL escapes, eliminates traversal segments (..), discards self-redirects, and outputs SEO_REDIRECTS.csv (Redirect from -> Redirect to) formatted for Shopify Admin -> Online Store -> Navigation -> URL Redirects.
• Formula Neutralization Modes:
  • shopify_safe (default): Strips non-printable control characters and prepends a space before formula triggers (=, +, -, @) so titles import cleanly without apostrophe prefixes.
  • spreadsheet_strict: Adheres to OWASP formula injection mitigation by prefixing formula triggers with a single quote (').
• Deterministic Handle Disambiguation: Resolves slug collisions by deterministically appending incremental counter suffixes (-1, -2) to ensure valid handle uniqueness across multi-page catalogs.
• Quarantine Audit Logging: Captures unparseable prices, unhandled currencies, or corrupt schemas into REJECTED_RECORDS.json with the exact record ID, field name, raw value, and error description.

---

⚠️ Limitations & Migration Realities

To maintain realistic expectations, understand the boundary conditions and operational limits of this scraper:

1. Accessible Public Endpoints Required:
   • Shopify: The store must have public /products.json enabled. Stores requiring customer login, password-protected development stores, or custom headless frontends with disabled standard endpoints cannot be scraped.
   • WooCommerce: The store must have the public WooCommerce Store API v1 (/wp-json/wc/store/v1/products) active and accessible. Stores with custom authentication, disabled REST APIs, or non-standard routing cannot be extracted.
2. Anti-Bot & WAF Protections: Stores deployed behind aggressive Web Application Firewalls (such as Cloudflare Under-Attack mode, reCAPTCHA challenges, or restrictive IP rate-limit blocks) may reject automated requests.
3. No Client-Side JavaScript Rendering: Extraction operates via pure HTTP streaming requests without a headless browser runtime. Single-page applications (SPAs) or dynamically injected JavaScript catalogs that do not expose public JSON feeds will yield zero products.
4. Pagination & Catalog Caps:
   • Shopify: Supports up to 250 products per page; bounded to maxPages=50 (theoretical ceiling of 12,500 products per run).
   • WooCommerce Store API v1: Hard-capped by WooCommerce to 100 products per page; bounded to maxPages=50 (theoretical ceiling of 5,000 products per run).
5. SEO & Search Ranking Disclaimer:
   • Generated SEO_REDIRECTS.csv files map legacy product URLs to standard /products/{handle} destinations.
   • Important: Implementing 301 redirects assists search engines in discovering replacement URLs and helps pass link authority, but no tool can guarantee 100% search ranking preservation or zero traffic loss. Search rankings depend on multiple broader domain factors, including domain authority, content equivalence, page speed, technical site health, mobile responsiveness, and search engine re-indexing schedules.
6. Pre-Publishing Review Mandatory: Merchants must review the generated SHOPIFY_PRODUCTS.csv and SEO_REDIRECTS.csv in a staging environment and verify that target product handles exist before publishing redirects to a live production store.
7. Accessible Public Endpoints Required (Anti-Bot Limitations): Target stores must have publicly accessible product catalog endpoints (/products.json for Shopify or /wp-json/wc/store/v1/products for WooCommerce). If a merchant protects their store with Cloudflare Turnstile, Bot Fight Mode, or private IP allowlists returning HTTP 403, this Actor cannot bypass the firewall without authenticated merchant session credentials.

---

📥 Input Parameters

Parameter	Type	Required	Default	Description
platform	String	Yes	woocommerce	Source store platform: shopify or woocommerce.
storeUrl	String	Yes	https://example-store.com	Base public HTTP/HTTPS URL of the target store. Local paths, non-HTTP schemes, and private IP addresses are blocked.
maxPages	Integer	No	5	Maximum number of pagination pages to crawl (1 to 50).
pageSize	Integer	No	100	Products per request page (1 to 250 for Shopify; auto-capped at 100 for WooCommerce Store API).
csvMode	String	No	shopify_safe	Formula neutralization mode: shopify_safe (recommended for clean Shopify Admin imports) or spreadsheet_strict (OWASP single-quote prefix).

---

📤 Output Artifacts

Upon completion, the run generates the following storage items:

Artifact	Location	Format	Description
32-Column Product Rows	Default Dataset	JSON / CSV / Excel	Structured rows conforming to the official 32-column Shopify import schema.
SHOPIFY_PRODUCTS.csv	Key-Value Store	CSV (RFC 4180)	Ready-to-upload product import CSV for Shopify Admin -> Products -> Import.
SEO_REDIRECTS.csv	Key-Value Store	CSV (RFC 4180)	Deduplicated 301 redirect map for Shopify Admin -> Online Store -> Navigation -> URL Redirects.
REJECTED_RECORDS.json	Key-Value Store	JSON	Audit trail of skipped or malformed items detailing record IDs, fields, and failure reasons.
OUTPUT	Key-Value Store	JSON	Execution summary containing status, product counts, redirect counts, and artifact references.

---

🔌 API Integrations & Automation

1. Model Context Protocol (MCP Server)

Integrate catalog extraction directly into Claude Desktop, Cursor, or any MCP-compatible AI agent:

{
  "mcpServers": {
    "store-catalog-extractor": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-fetch",
        "https://mcp.apify.com/?tools=deepaksheoran172/store-catalog-redirect-extractor"
      ]
    }
  }
}

2. cURL Example (Secure Authentication Header)

Never pass API tokens in URL query strings. Use standard Authorization headers:

curl -X POST "https://api.apify.com/v2/acts/deepaksheoran172~store-catalog-redirect-extractor/runs" \
  -H "Authorization: Bearer YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "platform": "shopify",
    "storeUrl": "https://example-fashion-store.myshopify.com",
    "maxPages": 3,
    "pageSize": 50,
    "csvMode": "shopify_safe"
  }'

3. Python SDK Example

Run programmatically using the official apify-client library:

import os
from apify_client import ApifyClient

# Securely read token from environment variable
apify_token = os.environ.get("APIFY_TOKEN")
if not apify_token:
    raise ValueError("APIFY_TOKEN environment variable is not set.")

client = ApifyClient(apify_token)

actor_input = {
    "platform": "shopify",
    "storeUrl": "https://example-fashion-store.myshopify.com",
    "maxPages": 2,
    "pageSize": 50,
    "csvMode": "shopify_safe",
}

# Start the Actor and wait for completion
run = client.actor("deepaksheoran172/store-catalog-redirect-extractor").call(run_input=actor_input)

# Fetch structured dataset items
dataset_items = client.dataset(run["defaultDatasetId"]).list_items().items
print(f"Extracted {len(dataset_items)} dataset rows.")

# Download 301 redirects CSV from Key-Value Store
kvs = client.key_value_store(run["defaultKeyValueStoreId"])
redirects_csv = kvs.get_record("SEO_REDIRECTS.csv")["value"]
print(f"Generated 301 Redirects CSV ({len(redirects_csv)} bytes).")

---

🔒 Security & Credential Best Practices

1. Protect Your API Token: Never commit Apify tokens to version control, public repositories, or client-side code. Use environment variables (APIFY_TOKEN) or a secret manager.
2. Data Verification: Always review REJECTED_RECORDS.json in the Key-Value Store after a run to verify whether any products were quarantined due to pricing formatting discrepancies.
3. SSRF Safeguards: This Actor automatically resolves destination hostnames, blocking requests targeting localhost, link-local addresses, loopback ranges, internal cloud metadata endpoints (e.g., 169.254.169.254), and non-routable private networks.

---

❓ Frequently Asked Questions (FAQs)

Q: Do I need Shopify private API keys or WooCommerce REST secrets?

A: No. This Actor is designed for public catalog extraction using standard exposed JSON feeds (/products.json and WooCommerce Store API v1). No private merchant credentials or admin privileges are required.

Q: How does this help prevent broken links during a store migration?

A: It maps every extracted legacy product URL (/product/slug, /?p=123) to the canonical Shopify format (/products/{handle}). Importing the generated SEO_REDIRECTS.csv into Shopify Admin sets up server-side 301 permanent redirects so legacy URLs point to the new location.

Q: Can this guarantee that our search engine rankings will not drop?

A: No tool can make that guarantee. While 301 redirects are an industry-standard best practice for transferring link equity, overall search rankings depend on many site-level factors including content parity, page load performance, internal linking, and search engine crawl cycles.

Q: How does the Actor format multi-variant products?

A: It adheres to Shopify's CSV structure: the parent product row contains the primary title, vendor, tags, and category, while subsequent rows represent individual variants (Option1 Value, Option2 Value, Option3 Value) with dedicated SKUs, prices, inventory flags, and image positions.

---

🚀 Ready to Extract Catalogs & 301 Redirects?

Generate Shopify-ready product CSVs and SEO redirect maps in seconds:

1. Click Try for free at the top right of this page.
2. Enter your source store URL (storeUrl) and select the platform (shopify or woocommerce).
3. Download SHOPIFY_PRODUCTS.csv and SEO_REDIRECTS.csv for immediate import.