# Shopify App Store Analyzer (`gaplens/shopify-app-store-analyzer`) Actor

Crawl Shopify App Store categories and get per-app listings plus niche-opportunity metrics (reviews per app, top-3 concentration, share of apps under 100 reviews).

- **URL**: https://apify.com/gaplens/shopify-app-store-analyzer.md
- **Developed by:** [GapLens](https://apify.com/gaplens) (community)
- **Categories:** E-commerce, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Shopify App Store Analyzer

**Find the under-served niches in the Shopify App Store — with numbers, not vibes.**

This Actor crawls [apps.shopify.com](https://apps.shopify.com) category pages and returns two things:

1. **One row per app, per category** — name, URL/handle, star rating, review count, pricing label, one-line description, its rank in the category, the category's total app count, and the page it came from.
2. **One opportunity summary per category** — how much demand there is per competing app, how concentrated the reviews are at the top, and how much of the long tail is still tiny.

The summary metrics are the interesting part. A category with 400 apps where the top 3 hold 70% of all reviews is a very different business proposition from one with 90 apps where the median app has 300 reviews.

***

### What you get

#### Per-app dataset item

```json
{
  "name": "Judge.me Product Reviews",
  "handle": "judgeme",
  "url": "https://apps.shopify.com/judgeme",
  "rating": 4.9,
  "reviews": 44073,
  "pricing": "Free plan available",
  "description": "Collect and display product reviews, photo reviews and Q&A to build trust",
  "position": 1,
  "positionOnPage": 1,
  "page": 1,
  "categoryPath": "/categories/marketing-and-conversion-social-trust-product-reviews",
  "categoryUrl": "https://apps.shopify.com/categories/marketing-and-conversion-social-trust-product-reviews",
  "categoryName": "Product reviews",
  "categoryBreadcrumb": "Marketing and conversion > Social trust > Product reviews",
  "totalApps": 428,
  "keywordClusters": ["gdpr_cookie"],
  "parseStrategy": "data-attributes",
  "scrapedAt": "2026-08-18T12:00:00.000Z",
  "detail": null
}
```

With **Include app detail pages** enabled, `detail` is filled in:

```json
{
  "detail": {
    "handle": "judgeme",
    "developer": "Judge.me",
    "developerUrl": "https://apps.shopify.com/partners/judgeme",
    "launchedAt": "2015-06-24",
    "categoryTags": ["Product reviews", "Social trust"],
    "pricingPlans": [
      { "name": "Forever Free", "price": "0 USD", "features": [] },
      { "name": "Awesome", "price": "15 USD", "features": [] }
    ],
    "strategy": "json-ld"
  }
}
```

#### Per-category summary

Written to the **`OUTPUT` record of the default key-value store** and to a **named dataset** (`category-summaries` by default):

```json
{
  "path": "/categories/marketing-and-conversion-social-trust-product-reviews",
  "name": "Product reviews",
  "breadcrumb": "Marketing and conversion > Social trust > Product reviews",
  "total_apps": 428,
  "sampled": 72,
  "sampled_reviews": 85815,
  "top24_reviews": 81785,
  "reviews_per_app": 200.5,
  "top24_avg": 3408,
  "median_sampled": 135,
  "share_under100": 0.42,
  "top3_share": 0.66,
  "free_only_share": 0.19,
  "reg_kw": { "gdpr_cookie": 3, "tax_vat": 1 }
}
```

| Metric | Meaning | How to read it |
| --- | --- | --- |
| `total_apps` | Apps Shopify reports in the category | Competition |
| `sampled` | Apps actually crawled (24 per page) | Sample size |
| `reviews_per_app` | `sampled_reviews / total_apps` | **Demand per competitor.** The headline number — high means merchants are buying and there is room |
| `top24_avg` | Mean reviews of the 24 most-reviewed apps | How big the winners get |
| `median_sampled` | Median review count | Is the middle of the market alive? |
| `share_under100` | Fraction of sampled apps with < 100 reviews | Size of the dead long tail |
| `top3_share` | Share of reviews held by the top 3 apps | **Concentration.** > 0.6 means an entrenched incumbent |
| `free_only_share` | Fraction priced exactly "Free" | Willingness to pay |
| `reg_kw` | Keyword-cluster hit counts | Thematic sub-niches inside the category |

The `OUTPUT` record also contains a cross-category `keywordClusters` roll-up (apps deduped by name, ranked by total reviews) and a `stats` block listing which parser strategy was used per category plus any `DEBUG_` keys that were written.

All three destinations are declared in `.actor/output_schema.json` (linked from `actor.json` as `"output"`), so the Console's Output tab links straight to the app dataset, the `OUTPUT` record, the `category-summaries` dataset and any `DEBUG_` snapshots. The shape of a dataset item is described in `.actor/dataset_schema.json`.

***

### Use cases

**App developers picking a niche.** Sort categories by `reviews_per_app` descending, then filter to `top3_share < 0.5` and `share_under100 > 0.6`. That combination — real demand, no entrenched winner, a long tail nobody has consolidated — is where a new app has room. Add `keywordClusters` to hunt for regulatory sub-niches (EU withdrawal buttons, GPSR, accessibility, e-invoicing) that spike in demand when legislation lands.

**Agencies and consultancies.** Build a defensible "which app should we recommend" matrix per category for client stacks: rating, review volume, pricing model and launch date in one table, refreshed on a schedule. Also useful for spotting when a client's incumbent app is losing ground.

**Investors and M\&A.** Size a category (`total_apps`, `sampled_reviews`), measure how winner-take-all it is (`top3_share`), and track those numbers over time by scheduling the Actor monthly and diffing the summary dataset. Review counts are a decent public proxy for installed base growth.

**Competitive monitoring.** Run one category weekly and watch your own `position`, `reviews` and `rating` against the field.

***

### Input

| Field | Type | Default | Description |
| --- | --- | --- | --- |
| `categories` | string\[] | `[]` → auto-discover | Slugs (`store-management-security-legal`), paths (`/categories/store-design`) or full URLs. Empty means: discover every top-level category from `/categories`. |
| `maxPagesPerCategory` | integer | `3` | Result pages per category, 24 apps each. 3 pages ≈ 72 apps, enough for stable concentration metrics. |
| `followSubcategories` | boolean | `true` | Descend into subcategories when a page lists them instead of apps. |
| `maxSubcategoryDepth` | integer | `2` | How many levels to descend. 2 reaches the leaf level of Shopify's taxonomy. |
| `includeAppDetails` | boolean | `false` | Also visit each app page for launch date, developer, pricing plans and category tags. Roughly 25× the requests. |
| `keywordClusters` | object\[] | built-in | `[{ "name": "gdpr_cookie", "regex": "gdpr\|cookie\|consent" }]`, matched case-insensitively against name + description. |
| `useBrowser` | boolean | `false` | Switch to Playwright. Only needed if pages start arriving empty — see *Limitations*. |
| `proxyConfiguration` | object | Apify Proxy | Standard Apify proxy input. |
| `maxConcurrency` | integer | `5` | Deliberately polite. |
| `maxRequestRetries` | integer | `5` | Each retry rotates the session and proxy. |
| `summaryDatasetName` | string | `category-summaries` | Named dataset for the per-category rows. |
| `saveDebugHtml` | boolean | `true` | Dump raw HTML to `DEBUG_<category>_<page>.html` when a page yields zero apps. |
| `enablePpeCharging` | boolean | `false` | Advanced, leave off. Enables the optional custom charge events — see *Pricing*. Billing normally comes from Apify's platform-managed events. |

#### Example input

```json
{
  "categories": [
    "store-management-security-legal",
    "marketing-and-conversion-social-trust-product-reviews"
  ],
  "maxPagesPerCategory": 3,
  "includeAppDetails": false,
  "keywordClusters": [
    { "name": "gdpr_cookie", "regex": "gdpr|cookie|\\bconsent|ccpa|privacy" },
    { "name": "accessibility", "regex": "accessib|wcag|\\bada\\b|\\beaa\\b|bfsg|aoda" }
  ],
  "proxyConfiguration": { "useApifyProxy": true }
}
```

Whole-store scan (slower, much broader):

```json
{ "categories": [], "maxPagesPerCategory": 3, "maxSubcategoryDepth": 2 }
```

***

### Pricing

This Actor uses **pay-per-event** billing with Apify's two **platform-managed** events:

| Event | When it fires | Price |
| --- | --- | --- |
| `apify-actor-start` | Once per run — covers category discovery, pagination and metric computation | $0.05 |
| `apify-default-dataset-item` | Once per app-in-category item pushed to the default dataset | $0.005 |

A typical two-category run at 3 pages each pushes ~144 app records: **$0.05 + 144 × $0.005 ≈ $0.77**.

Both events are metered by the Apify platform itself — the Actor code never calls `Actor.charge()` for them, and **no dollar amount is hardcoded anywhere in `src/`**. Prices are configured in the Apify Console (Monetization tab); `.actor/pay_per_event.json` is a documentation-only mirror of that setup.

<details>
<summary>Optional: custom charge events (off by default)</summary>

The Actor also ships an unused custom-event code path (`analyzer-run-start`, `app-record`) for the case where this Actor is re-monetized with custom pay-per-event events instead of the platform-managed ones. It is **off by default** (`enablePpeCharging` defaults to `false`) and only fires when explicitly enabled, either through that input or by setting `ACTOR_PPE_ENABLED=1` locally.

Do not enable it while the platform-managed `apify-default-dataset-item` price is active: the custom `app-record` event covers exactly the same unit of work, so every run would be billed twice. Neither event carries a price in code — the platform applies whatever is configured in the Console.

</details>

***

### How the scraping works (and why it keeps working)

Shopify changes its markup. Rather than betting on one selector, the Actor runs a **ladder of four independent extraction strategies** per page and takes the first that produces results:

1. **`json-ld`** — `<script type="application/ld+json">` `ItemList` of `SoftwareApplication` entries.
2. **`embedded-json`** — any `<script type="application/json">` / `window.__STATE__` blob containing an array of app-shaped objects.
3. **`data-attributes`** — elements carrying `data-controller="app-card"` and `data-app-card-*-value` attributes.
4. **`heuristic`** — anchors matching `https://apps.shopify.com/<handle>`, with rating, review count (`4.9 out of 5 stars (2,190)`) and pricing mined from the surrounding card node.

The strategy that won is logged and recorded on every dataset item as `parseStrategy`, so you can see at a glance whether the site changed shape. If **zero** app cards are found on a page that looks like a listing, the raw HTML is stored in the default key-value store as `DEBUG_<category-slug>_<page>.html` — enough to fix selectors in minutes instead of guessing.

Total counts come from `/(\d[\d,]*)\s+apps/i`; pagination follows `rel="next"` and `?page=N` links.

***

### Reproducing the analysis offline

Export the dataset as JSON and re-run the exact same metric code locally:

```bash
npm ci
npm run analyze:local -- ./dataset.json --out ./out --top 20
```

This prints the ranked category table plus keyword clusters, and writes `leaf_metrics.json`, `leaf_metrics.csv` and `keyword_clusters.json` to `./out`. It accepts an Apify dataset export (`[…]` or `{"items": […]}`).

***

### Limitations and known risks

- **Selector drift.** Shopify's App Store DOM is not a public API. The four-strategy ladder plus the `DEBUG_` HTML dumps make a break fast to diagnose, but a break is always possible. Check `stats.parseStrategies` and `stats.debugKeys` in `OUTPUT` after each run.
- **`total_apps` may be capped.** Shopify sometimes rounds or caps the displayed count on large categories, which shifts `reviews_per_app` proportionally. Compare categories, don't treat the absolute value as exact.
- **Sampling bias by design.** With the default 3 pages you see the top ~72 apps by Shopify's own ranking, not a random sample. `sampled_reviews`, `top3_share` and `median_sampled` describe *the head of the category*. Raise `maxPagesPerCategory` for a deeper (and more expensive) view.
- **Review counts ≠ installs.** They are a proxy, biased toward apps that ask for reviews aggressively and toward older apps.
- **`useBrowser` needs a different image.** Category pages are server-rendered, so the default HTTP crawler is correct and much cheaper. Playwright mode requires building from `Dockerfile.playwright` (`apify/actor-node-playwright-chrome:20`) with `playwright` added to dependencies.
- **Rate limiting.** Concurrency is capped at 10 and defaults to 5. Use Apify Proxy; switch to residential groups if you see 403s.
- **App detail parsing is best-effort.** Launch dates and pricing plans come from several fallbacks and may be `null` on pages with unusual layouts.
- **`description` is best-effort too.** Shopify does not always expose the card tagline in a data attribute. The parser reads it from the description element in the card and, if a whole page still comes back without descriptions, back-fills them from the heuristic strategy (matched on handle/URL). A card that genuinely renders no tagline keeps `description: null`.
- **Localization.** Only the default `apps.shopify.com` locale is crawled; localized paths are ignored.

***

### Development

```bash
npm ci
npm run build      # tsc -> dist/
npm test           # vitest: parsers, metrics, and an HTTP integration test
npm run typecheck  # type-checks src + tests + scripts
```

Tests run entirely offline against hand-written fixtures in `tests/fixtures/` that mimic each DOM shape, plus a golden dataset captured from the original Python reference implementation — the metrics port is asserted to match it field for field.

# Actor input Schema

## `categories` (type: `array`):

Shopify App Store categories to analyze. Accepts a slug (`store-management-security-legal`), a path (`/categories/store-management`) or a full URL (`https://apps.shopify.com/categories/store-design`). Leave empty to auto-discover every top-level category from https://apps.shopify.com/categories.

## `maxPagesPerCategory` (type: `integer`):

How many result pages (24 apps each) to sample per category. 3 pages ≈ 72 apps, which is enough for stable concentration metrics. Higher values cost more and take longer.

## `followSubcategories` (type: `boolean`):

When a category page lists subcategories instead of apps (non-leaf page), crawl into those subcategories. Turn this off to analyze only the exact categories you supplied.

## `maxSubcategoryDepth` (type: `integer`):

How many levels of subcategories to descend from each starting category. 0 = never descend, 2 = reach the leaf level of the Shopify taxonomy (e.g. Marketing and conversion → Social trust → Product reviews).

## `includeAppDetails` (type: `boolean`):

Also visit each app's own page to collect launch date, developer, pricing plans and category tags. This roughly multiplies the number of requests by 25, so expect a much longer run.

## `keywordClusters` (type: `array`):

Optional named regular expressions used to bucket apps by theme. Each entry is `{ "name": "gdpr_cookie", "regex": "gdpr|cookie|consent" }` and is matched case-insensitively against the app name + description. Leave empty to use the built-in regulatory/compliance clusters.

## `useBrowser` (type: `boolean`):

Category pages are server-rendered HTML, so the fast HTTP crawler is the default and is what you want in almost every case. Enable this only if pages start arriving empty or blocked — it requires a build from Dockerfile.playwright (apify/actor-node-playwright-chrome).

## `proxyConfiguration` (type: `object`):

Apify Proxy or your own proxy servers. Datacenter proxies are usually sufficient; switch to residential if you see blocks.

## `maxConcurrency` (type: `integer`):

Maximum parallel requests. Kept low on purpose to stay polite to apps.shopify.com.

## `maxRequestRetries` (type: `integer`):

How many times to retry a failing page before giving up (each retry rotates the session/proxy).

## `summaryDatasetName` (type: `string`):

Name of the named dataset that receives one row of metrics per category. The same data is always written to the OUTPUT record of the default key-value store as well.

## `saveDebugHtml` (type: `boolean`):

If a listing page produces no app cards (usually a Shopify DOM change), store the raw HTML in the default key-value store under DEBUG\_<category>\_<page>.html so selectors can be fixed quickly.

## `enablePpeCharging` (type: `boolean`):

Leave this off. Monetization uses Apify's platform-managed events (`apify-actor-start` and `apify-default-dataset-item`), which the platform meters on its own. This switch only enables the optional custom `analyzer-run-start` / `app-record` charge calls, which would double-bill the same work while the platform-managed dataset-item price is active. Prices are always configured in the Apify Console, never in the Actor code.

## Actor input object example

```json
{
  "categories": [
    "store-management-security-legal",
    "marketing-and-conversion-social-trust-product-reviews"
  ],
  "maxPagesPerCategory": 3,
  "followSubcategories": true,
  "maxSubcategoryDepth": 2,
  "includeAppDetails": false,
  "keywordClusters": [
    {
      "name": "gdpr_cookie",
      "regex": "gdpr|cookie|\\bconsent|ccpa|privacy"
    },
    {
      "name": "accessibility",
      "regex": "accessib|wcag|\\bada\\b|\\beaa\\b|bfsg|aoda"
    },
    {
      "name": "invoice",
      "regex": "invoice|e-invoic|peppol|xrechnung|zugferd|factur"
    }
  ],
  "useBrowser": false,
  "proxyConfiguration": {
    "useApifyProxy": true
  },
  "maxConcurrency": 5,
  "maxRequestRetries": 5,
  "summaryDatasetName": "category-summaries",
  "saveDebugHtml": true,
  "enablePpeCharging": false
}
```

# Actor output Schema

## `apps` (type: `string`):

One item per app per category: name, handle, canonical URL, star rating, review count, pricing label, one-line description, rank in the category, page number, category path/name/breadcrumb, the category's total app count, matched keyword clusters and (optionally) app-detail fields.

## `appsTable` (type: `string`):

The same app records rendered through the 'overview' view of the dataset schema.

## `summary` (type: `string`):

The OUTPUT record of the default key-value store: per-category opportunity metrics (reviews\_per\_app, top24\_avg, median\_sampled, share\_under100, top3\_share, free\_only\_share, reg\_kw), the cross-category keyword-cluster roll-up, the echoed input and run stats (categories analysed, app records pushed, parse strategy per category, debug keys).

## `debugHtml` (type: `string`):

Raw HTML of listing pages that yielded zero app cards, stored as DEBUG\_<category>\_<page>.html in the default key-value store. Empty on a healthy run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "categories": [
        "store-management-security-legal",
        "marketing-and-conversion-social-trust-product-reviews"
    ],
    "keywordClusters": [
        {
            "name": "gdpr_cookie",
            "regex": "gdpr|cookie|\\bconsent|ccpa|privacy"
        },
        {
            "name": "accessibility",
            "regex": "accessib|wcag|\\bada\\b|\\beaa\\b|bfsg|aoda"
        },
        {
            "name": "invoice",
            "regex": "invoice|e-invoic|peppol|xrechnung|zugferd|factur"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("gaplens/shopify-app-store-analyzer").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "categories": [
        "store-management-security-legal",
        "marketing-and-conversion-social-trust-product-reviews",
    ],
    "keywordClusters": [
        {
            "name": "gdpr_cookie",
            "regex": "gdpr|cookie|\\bconsent|ccpa|privacy",
        },
        {
            "name": "accessibility",
            "regex": "accessib|wcag|\\bada\\b|\\beaa\\b|bfsg|aoda",
        },
        {
            "name": "invoice",
            "regex": "invoice|e-invoic|peppol|xrechnung|zugferd|factur",
        },
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("gaplens/shopify-app-store-analyzer").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "categories": [
    "store-management-security-legal",
    "marketing-and-conversion-social-trust-product-reviews"
  ],
  "keywordClusters": [
    {
      "name": "gdpr_cookie",
      "regex": "gdpr|cookie|\\\\bconsent|ccpa|privacy"
    },
    {
      "name": "accessibility",
      "regex": "accessib|wcag|\\\\bada\\\\b|\\\\beaa\\\\b|bfsg|aoda"
    },
    {
      "name": "invoice",
      "regex": "invoice|e-invoic|peppol|xrechnung|zugferd|factur"
    }
  ]
}' |
apify call gaplens/shopify-app-store-analyzer --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,gaplens/shopify-app-store-analyzer"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/7B5nhOE8ajsVizfN6/builds/BRqa4mUSWUuriqzKs/openapi.json