E-commerce Store Intelligence Actor

Extract comprehensive store and product intelligence from Shopify and WooCommerce stores with automated change tracking and monitoring.

Features

🏪 Store Analysis

• Platform Detection: Automatically detect Shopify, WooCommerce, and other platforms
• Store Metadata: Extract store name, description, logo, and branding
• Contact Signals: Find email, phone, WhatsApp, and chat widgets
• Policies: Extract shipping and return policy URLs
• Promotions: Detect promo banners, coupons, and free shipping offers

📦 Product Intelligence

• Comprehensive Extraction: Title, price, variants, availability, images
• Multiple Discovery Methods: Sitemap, platform APIs, category crawl, search
• Smart Filtering: Filter by collection, keywords, price range
• Variant Support: Extract all product variants with individual pricing
• Stock Status: Track in-stock vs. out-of-stock products

📊 Change Tracking

• Price Monitoring: Detect price changes and calculate percentages
• Availability Tracking: Monitor stock status changes
• New Products: Identify newly added products
• Removed Products: Detect products removed from catalog
• Historical Comparison: Compare against previous runs or specific snapshots

🔔 Delivery & Alerts

• Webhook Integration: POST results to custom webhooks
• Conditional Alerts: Only notify when changes are detected
• Multiple Output Formats: Dataset, Key-Value Store, Webhook

Use Cases

1. Competitor Price Monitoring: Track competitor pricing and promotions
2. Product Research: Analyze product catalogs and trends
3. Market Intelligence: Monitor new product launches
4. Inventory Tracking: Track stock availability
5. Pricing Strategy: Analyze pricing patterns and changes

Input Configuration

{
  "stores": [
    {
      "storeUrl": "https://example-store.myshopify.com",
      "label": "Example Store",
      "country": "IN",
      "currency": "INR"
    }
  ],
  "crawl": {
    "render": "playwright",
    "maxProducts": 200,
    "maxCollections": 50,
    "productDiscovery": ["sitemap", "platform_api", "category_crawl"],
    "includeOutOfStock": true,
    "respectRobots": true,
    "timeoutMs": 45000
  },
  "extract": {
    "storeMeta": true,
    "collections": true,
    "products": true,
    "reviewsSummary": false,
    "contactSignals": true,
    "policies": true,
    "promotions": true
  },
  "filters": {
    "onlyCollections": [],
    "onlyKeywords": [],
    "minPrice": null,
    "maxPrice": null
  },
  "changeTracking": {
    "enabled": true,
    "compareWith": "previousRun",
    "track": ["price", "availability", "new_products", "removed_products"]
  },
  "delivery": {
    "webhookUrl": "",
    "onlyWhenChanged": true
  }
}

Output Format

Store Summary

{
  "type": "store_summary",
  "storeUrl": "https://example-store.com",
  "label": "Example Store",
  "platform": {
    "name": "shopify",
    "confidence": 0.92,
    "evidence": ["window.Shopify object found", "Shopify CDN detected"]
  },
  "currency": "INR",
  "storeMeta": {
    "name": "Example Store",
    "description": "...",
    "socials": ["instagram.com/..."],
    "logo": "https://.../logo.png"
  },
  "signals": {
    "hasWhatsapp": true,
    "hasChatWidget": true,
    "emailFound": "support@example-store.com",
    "phoneFound": "+91..."
  },
  "promotions": {
    "freeShipping": true,
    "saleBannerText": "Up to 40% off",
    "couponHints": ["WELCOME10"]
  },
  "policies": {
    "shippingUrl": ".../shipping-policy",
    "returnsUrl": ".../refund-policy"
  },
  "catalogStats": {
    "collectionsCount": 18,
    "productsCollected": 200,
    "inStockCount": 164
  },
  "runMeta": {
    "runId": "xxx",
    "timestamp": "2025-12-23T10:30:00Z"
  }
}

Product Item

{
  "type": "product",
  "storeUrl": "https://example-store.com",
  "productUrl": "https://example-store.com/products/red-tshirt",
  "handle": "red-tshirt",
  "title": "Red T-Shirt",
  "vendor": "Example",
  "productType": "Apparel",
  "tags": ["cotton", "new"],
  "images": ["https://..."],
  "price": {
    "min": 499,
    "max": 699,
    "currency": "INR"
  },
  "compareAtPrice": {
    "min": 799,
    "max": 999,
    "currency": "INR"
  },
  "availability": "in_stock",
  "variants": [
    {
      "title": "S",
      "price": 499,
      "available": true,
      "sku": "RTS-S"
    }
  ],
  "collectionHints": ["tshirts", "new-arrivals"],
  "extractedFrom": "platform_api"
}

Change Event

{
  "type": "change_event",
  "storeUrl": "https://example-store.com",
  "productUrl": "https://example-store.com/products/red-tshirt",
  "changeType": "price",
  "before": {
    "min": 549,
    "max": 749
  },
  "after": {
    "min": 499,
    "max": 699
  },
  "changePercent": -9.1,
  "detectedAt": "2025-12-23T10:30:00Z"
}

Pricing

Pay-Per-Event Model

Event 1: apify-actor-start (Synthetic)

• Price: $0.00005 per run
• Benefit: First 5 seconds FREE
• Charged: Automatically by Apify

Event 2: store-analyzed

• Price: $0.01 per store
• Includes: Platform detection, signals, catalog discovery

Event 3: product-processed

• Price: $0.001 per product
• Includes: Full product extraction and processing

Pricing Examples

10 products from 1 store:

• Actor start: $0.00005
• Store analyzed: $0.01
• Products: 10 × $0.001 = $0.01
• Total: $0.02005

200 products from 5 stores:

• Actor start: $0.00005
• Stores: 5 × $0.01 = $0.05
• Products: 200 × $0.001 = $0.20
• Total: $0.25005

Daily monitoring (100 products, 24 runs):

• Per run: ~$0.11
• Per day: $2.64
• Per month: ~$79

Architecture

Module Structure

src/
├── main.js                      # Main actor logic
├── detectors/
│   └── platformDetector.js      # Platform detection
├── discovery/
│   ├── sitemapDiscovery.js      # Sitemap-based discovery
│   ├── apiDiscovery.js          # Platform API discovery
│   └── categoryDiscovery.js     # Category crawl discovery
├── extractors/
│   ├── storeExtractor.js        # Store metadata extraction
│   ├── productExtractor.js      # Product data extraction
│   └── signalsExtractor.js      # Contact/promo signals
├── tracking/
│   └── changeTracker.js         # Change detection & tracking
└── utils/
    ├── helpers.js               # Utility functions
    └── webhooks.js              # Webhook delivery

Product Discovery Flow

1. Sitemap Discovery (Fast, high recall)

   • Parse /sitemap.xml
   • Extract product URLs
   • Priority: High

2. Platform API (Clean, structured)

   • Shopify: /products.json
   • WooCommerce: /wp-json/wc/v3/products
   • Priority: High

3. Category Crawl (Fallback)

   • Crawl collection/category pages
   • Extract product links
   • Priority: Medium

4. Search (Last resort)

   • Use on-site search
   • Extract results
   • Priority: Low

Best Practices

For Accurate Results

1. Use playwright render for JavaScript-heavy stores
2. Set appropriate maxProducts to avoid runaway crawls
3. Enable respectRobots to be a good citizen
4. Use filters to focus on relevant products

For Cost Optimization

1. Use cheerio render when possible (faster, cheaper)
2. Set maxProducts limit based on needs
3. Filter by collections or keywords to reduce scope
4. Schedule runs during off-peak hours

For Change Tracking

1. Enable changeTracking for monitoring use cases
2. Run on a schedule (hourly, daily, weekly)
3. Use onlyWhenChanged webhooks to reduce noise
4. Track only relevant changes (price, availability)

Limitations

• Rate Limiting: Respects robots.txt and implements delays
• JavaScript Required: Some stores require Playwright rendering
• API Access: Not all stores expose public APIs
• Authentication: Does not handle login-required stores
• Dynamic Pricing: May not capture personalized pricing

Development

Local Setup

cd ecommerce-store-intelligence
npm install
npm start

Testing

# Test with example input
apify run --input-file examples/shopify-store.json

# Test specific store
apify run --input '{"stores":[{"storeUrl":"https://example.myshopify.com"}]}'

Deployment

# Build and push to Apify
apify push

# Or use Apify Console to build from GitHub

Support

For issues, questions, or feature requests:

• GitHub Issues: [Link to repo]
• Apify Console: Actor page
• Email: support@example.com

License

Apache-2.0

---

Built with ❤️ for e-commerce intelligence