Apify Market Gap Finder

Compare two to twenty buyer-declared Actor niches against separate public Apify Store cohorts, then receive a ranked, evidence-linked validation queue.

Built for: Actor builders, product studios, automation agencies, marketplace researchers, and portfolio owners deciding which narrowly defined product idea deserves the next discovery sprint.

Commercial unit: one useful market-gap report. Price: $0.10 per delivered result-found report plus the configured start event (starting at $0.005). Invalid input, source failure, insufficient evidence, cache hits, and true replays are not presented as newly delivered paid reports.

Decide what to research next — without pretending public metadata is a business forecast

Choosing an Actor niche, portfolio move, or price is usually slowed down by fragmented evidence. Store search results are easy to browse but difficult to compare consistently: queries change, visible usage windows are confused with revenue, pricing events represent different units, and attractive numbers are copied into spreadsheets without their coverage limits. Apify Market Gap Finder turns that messy first research step into one bounded Dataset row with source context, deterministic calculations, confidence, explicit gaps, a human-review action, and an auditable billing receipt.

The Actor is designed to answer a narrow decision question, not to manufacture certainty. Two to twenty seedQueries; each query is searched and scored independently rather than merged into a misleading global result set. The result helps a product owner decide where to spend the next hour of qualitative validation. It does not claim to know private revenue, conversion, retention, buyer intent, market size, profitability, or future demand.

The result in plain language

• A normalized, timestamped report tied to the exact input and public Store cohort.
• Deterministic ranking or benchmarking fields that can be reproduced and reviewed.
• Visible source coverage and dataGaps, so missing evidence does not disappear behind a score.
• A separate confidence axis; confidence describes evidence sufficiency, not commercial attractiveness.
• A conservative next action: VALIDATE_TOP_GAP when at least one sufficiently covered opportunity exists; otherwise REFINE_NICHE_QUERIES.
• A Dataset row for analysis plus a KVS OUTPUT envelope for terminal workflow truth.
• Replay-safe PPE accounting: one genuinely delivered report can create one result-found charge; the same request cannot create a second result charge.

When this Actor is a good fit

• An independent publisher has ten possible Actor ideas and needs a reproducible shortlist before interviewing buyers.
• An agency wants to compare vertical-specific ideas such as clinic intake, tender monitoring, and catalog change detection with the same evidence frame.
• A portfolio owner wants a weekly research artifact that can be reviewed by a human product lead instead of a spreadsheet full of undocumented hunches.
• A consultant needs transparent competitor evidence to explain why an idea should be validated, refined, or parked.

Use it near the beginning of a product-discovery workflow: after a human has written a specific decision question, but before a team builds, reprices, or reallocates a portfolio slot. The strongest workflow combines this report with buyer interviews, support-ticket themes, search demand, sales objections, delivery-cost estimates, and a paid landing-page or concierge test.

When not to use it

Do not use this Actor as a substitute for customer discovery, financial due diligence, a private Apify account export, an autonomous investment decision, or permission to contact anyone. Do not use a score to make irreversible product changes without reading the underlying cohort and gaps. If the decision requires exact current Store facts, run a fresh observation rather than relying on an old cached report.

Evidence-to-action workflow

1. Define the decision in buyer language. Narrow queries make interpretation easier than broad category labels.
2. Submit a unique requestId with the bounded input. The Actor validates types, lengths, unknown fields, and URI-like/control-character payloads before source access.
3. Fetch the declared cohort from the public Apify Store API. No private account token or hidden marketplace dataset is used for research.
4. Normalize visible fields and calculate Actor-specific metrics deterministically.
5. Keep the commercial signal separate from evidence confidence and sample coverage.
6. Write one useful report to the default Dataset, then charge its result-found event and record the delivery receipt.
7. Write the authoritative terminal envelope to default KVS key OUTPUT, including replay, cache, source, failure, and billing truth.
8. Route recommendedAction to a human product review. safeToAutomate remains false.

How the calculation works

For every declared niche, the Actor builds a bounded public Store cohort, ranks the returned Actors by deterministic query similarity, and derives the shared opportunity factors from visible listing metadata. It then sorts niche reports by opportunity score, breaking score ties lexically so identical inputs remain stable. A niche is useful only when its score exists, at least three relevant Actors were observed, and coverage is at least 0.50. Thin evidence remains in the report but is not silently upgraded into a confident recommendation.

The method intentionally favors auditability over false sophistication. No LLM invents comparators or rewrites the decision after seeing a result. Stable canonical input creates a digest used for replay and conflict handling. Sorting rules make ties deterministic. Display rounding happens only after calculations that require raw precision. If the public source or state layer cannot support a trustworthy result, the Actor fails closed or returns an unbilled insufficient-evidence terminal state rather than manufacturing a positive row.

Read score and confidence separately

Axis	What it answers	What it does not answer
Signal	What the deterministic method observed for this declared cohort	Whether to ship the product
Confidence	Whether enough expected public fields were available	Whether a high-scoring niche is strategically correct
Coverage	How complete the expected source metrics were	Whether the public Store represents the whole market
Data gaps	What a human should verify next	A reason to silently discard inconvenient evidence
Recommended action	Which review step follows from the evidence state	Permission for autonomous build, pricing, outreach, or deletion

Input

Start with Try for free in Apify Console, or use the same contract through an API client. public-task.json, examples/input.json, and the Input schema prefill are intentionally identical. The published Task input is therefore a real contract example rather than decorative documentation.

{
  "schemaVersion": "1.0",
  "requestId": "gap-daily-prefill-001",
  "maxResults": 10,
  "freshnessMinutes": 60,
  "detailLevel": "compact",
  "seedQueries": ["market gap", "pricing gap"],
  "categories": ["LEAD_GENERATION"]
}

Common input controls

Field	Purpose
schemaVersion	Must be 1.0; prevents silent interpretation of an unknown contract.
requestId	Stable idempotency key, 1–128 safe characters. Use a new value for a genuinely new observation.
maxResults	Bounds detailed Store comparator retention from 1 to 100. It does not increase what the source publishes.
freshnessMinutes	Allows compatible cached report reuse; set 0 when the workflow requires a new source observation.
detailLevel	compact for routing or evidence for review-heavy workflows.

Input validation is strict. Unknown properties, coercible strings such as "10" for an integer, control characters, and embedded URI-like strings in query fields are rejected. This prevents ambiguous automation inputs and keeps research queries separate from network destinations.

Output: Dataset report and KVS terminal envelope

The Actor exposes two result stores because they serve different jobs:

• REPORTS points to default Dataset items. A row exists only for a useful delivered business report and is convenient for tables, exports, integrations, and downstream analysis.
• OUTPUT points to the default Key-Value Store record named OUTPUT. It is the authoritative terminal envelope for success, replay, cache behavior, source failure, validation failure, delivery failure, and billing reconciliation.

Never infer terminal success from process exit alone. For automation, read OUTPUT, verify its status and billing fields, and then read the Dataset row it references. This prevents a workflow from treating an empty Dataset, failed charge, or replay as a fresh paid result.

Dataset field dictionary

Field	Meaning
schemaVersion	Contract version; currently 1.0.
requestId	Buyer-controlled idempotency key used to distinguish a replay from a new commercial request.
reportType	Stable Actor-specific report discriminator.
generatedAt / observedAt	UTC timestamps for report construction and source observation.
dedupeKey / entityId	Stable normalized identity for storage joins, deduplication, and review queues.
detailLevel	compact or evidence, as requested.
sampleConfidence	Compatibility label derived from sample size and source coverage.
confidenceScore / confidenceBand	Separate confidence axis; never substitute this for opportunity or priority.
dataGaps	Explicit reasons the report should be interpreted cautiously or researched further.
recommendedAction	Human-facing next step. For this Actor: VALIDATE_TOP_GAP when at least one sufficiently covered opportunity exists; otherwise REFINE_NICHE_QUERIES.
safeToAutomate	Always false for the business decision; downstream systems should route to review.
failureType / retryable	Successful Dataset rows use null and false; terminal failures are recorded in OUTPUT instead of billed as results.
billing	Event name, delivery status, and PPE receipt facts for the row.
evaluatedQueryCount	How many submitted niche queries were evaluated.
opportunities	Ranked per-query reports with cohort size, opportunity factors, coverage, confidence, and visible comparator samples.
opportunities[].niche	The normalized buyer-declared query represented by this result.
opportunities[].opportunity.score	Deterministic ranking score; useful for ordering within this run, not a market-size estimate.
opportunities[].actorCount	Number of observed public Store Actors in the bounded cohort.
opportunities[].coverage	Share of expected source metrics actually available for scoring.

Decision-envelope guarantees

Every successful Dataset row includes recordType, entityId, observedAt, confidenceScore, confidenceBand, dataGaps, recommendedAction, safeToAutomate, failureType, retryable, and billing. These fields make the report usable as an enrichment object instead of a loose analytics blob. They also make uncertainty queryable: a reviewer can filter low-confidence rows, inspect gaps, group stable entities, and separate business evidence from delivery mechanics.

safeToAutomate is deliberately false. The report may automate research collection and queue construction, but it should not autonomously build an Actor, change a public price, delete a product, or represent a market claim to a customer.

Pricing and billing truth

This Actor uses pay per event (PPE): $0.10 per delivered result-found report plus the configured start event (starting at $0.005). Pricing tiers may reduce configured event prices at higher platform usage, and the live Apify pricing panel remains the source of truth for the current account charge.

The runtime treats billing as part of delivery integrity:

1. It validates the request and reserves the idempotency state.
2. It constructs the report and verifies the minimum usefulness boundary.
3. It pushes the Dataset row with the result-found event.
4. It checks the returned aggregate charge receipt. A positive chargedCount proves the current row was delivered even when that event also reaches the configured event limit.
5. It records receipt truth in the report state and OUTPUT.
6. A compatible replay writes one free copy of the prior report to the new run's Dataset, with billing.billable:false; it cannot create another result-found charge.

Validation errors, public-source failures, insufficient evidence, and internal delivery contradictions are not relabeled as paid insights. If delivery truth cannot be established, the terminal envelope exposes the failure and retryability instead of claiming success.

Run with the Apify API

Replace APIFY_TOKEN with a secret stored in your environment. Do not place a token in source control, screenshots, or README examples.

cURL

$curl -X POST   "https://api.apify.com/v2/acts/zinin~apify-market-gap-finder/run-sync-get-dataset-items?token=$APIFY_TOKEN"   -H "Content-Type: application/json"   -d @examples/input.json

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const input = {
  "schemaVersion": "1.0",
  "requestId": "gap-daily-prefill-001",
  "maxResults": 10,
  "freshnessMinutes": 60,
  "detailLevel": "compact",
  "seedQueries": ["market gap", "pricing gap"],
  "categories": ["LEAD_GENERATION"]
};

const run = await client.actor('zinin/apify-market-gap-finder').call(input);
const output = await client.keyValueStore(run.defaultKeyValueStoreId).getRecord('OUTPUT');
if (output?.value?.status !== 'ok') {
    throw new Error(`Actor terminal status: ${output?.value?.status ?? 'missing OUTPUT'}`);
}
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items[0]);

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
actor_input = {
  "schemaVersion": "1.0",
  "requestId": "gap-daily-prefill-001",
  "maxResults": 10,
  "freshnessMinutes": 60,
  "detailLevel": "compact",
  "seedQueries": ["market gap", "pricing gap"],
  "categories": ["LEAD_GENERATION"]
}

run = client.actor('zinin/apify-market-gap-finder').call(run_input=actor_input)
output = client.key_value_store(run['defaultKeyValueStoreId']).get_record('OUTPUT')
if not output or output['value'].get('status') != 'ok':
    raise RuntimeError('Actor did not produce a successful terminal OUTPUT envelope')
items = list(client.dataset(run['defaultDatasetId']).iterate_items())
print(items[0])

The Python example mirrors the exact JSON shape; if your codebase uses a generated model, preserve string enums and integer types exactly. For scheduled Tasks, generate a new business requestId per intended observation, or keep a stable request only when replay suppression is the desired behavior.

Automation patterns

Product-research review queue

Run the Actor on a fixed cadence, append useful Dataset rows to a warehouse, and create a review ticket only when recommendedAction changes or confidence crosses your team’s threshold. Include the source timestamp, confidence, gaps, and exact query in the ticket. A changing score without a changing cohort or coverage should not trigger an irreversible roadmap action.

Spreadsheet or BI export

Export Dataset items as JSON, CSV, or Excel through Apify Dataset endpoints. Keep nested evidence fields in the raw table even if the dashboard displays only rank, score, confidence, and action. That preserves the ability to explain a decision months later.

n8n, Make, Zapier, and webhooks

Trigger the Actor from the automation platform, wait for terminal completion, read KVS OUTPUT, and branch on explicit status. Send successful report rows to a human review queue. Send retryable source failures to bounded retry handling. Send non-retryable validation failures to input repair. Never branch only on “run finished.”

Agent and MCP workflows

An agent can call the Actor as a research tool, but the prompt should require it to cite dataGaps, confidenceBand, and the observed source timestamp whenever it summarizes the result. The agent must not convert an opportunity score into a revenue forecast or omit the “not safe to automate” boundary.

Reliability, idempotency, and replay behavior

The Actor canonicalizes the validated business input and excludes trusted run-specific transport additions from the buyer digest. requestId identifies the commercial request; the digest detects incompatible reuse. A compatible completed request can return existing state without creating a second paid result. Reusing one requestId with materially different business input is treated as a conflict, not as permission to overwrite history.

On Apify, the published daily-prefill request ID is extended with the trusted run ID so repeated clicks on the public Task remain distinct intentional test runs. In your own automation, choose the semantics explicitly:

• Use a new requestId for each new observation you want to purchase and store.
• Reuse the same requestId when retry/replay suppression is required.
• Do not reuse an ID with changed queries, categories, windows, portfolio context, or selectors.
• Read OUTPUT to distinguish cache reuse, replay, terminal failure, and delivered success.

Data provenance, security, and privacy

Research data comes from the public Apify Store API endpoint used by the runtime. The Actor does not log into a private Store account, scrape customer dashboards, infer private revenue, or fetch submitted portfolio objects from an account. Buyer input remains the caller’s declared research context.

The runtime uses limited Actor permissions, bounds input and result sizes, rejects network-looking query payloads, and avoids LLM processing. Public source metadata can still change or be incomplete. Store the observation timestamp and source diagnostics with any derived decision.

Do not include personal data, credentials, private URLs, tokens, or confidential strategy text in query fields. Although these Actors are aimed at marketplace-product research rather than personal-data enrichment, your organization remains responsible for input governance, retention, and access controls.

Honest limitations

• A small or empty public Store cohort is not proof that a niche has demand or that no competitor exists.
• Visible Store user metrics are not revenue, retention, paid conversion, margin, or willingness to pay.
• Search relevance depends on the wording of buyer-declared queries; the Actor does not autonomously invent niches.
• The score ranks evidence inside this method. It is not a probability of commercial success.

Additional boundaries apply to every Actor in this family:

• Public search results may be ranked, capped, delayed, renamed, or removed by the source.
• Missing visible fields lower evidence quality; they are not safely interpreted as zero.
• A comparator may target a different buyer, output unit, geography, freshness requirement, or service level.
• The Actor observes marketplace metadata at a point in time and does not establish causality.
• Recommendations are research-routing labels, not legal, financial, investment, or autonomous operating advice.

A practical validation playbook

After receiving a report, keep the next step small and falsifiable:

1. Read the top result and every dataGap; reject any interpretation that requires a missing field to be zero.
2. Open the relevant public comparator listings and compare the actual buyer promise, output unit, freshness, price, and limitations.
3. Interview at least a small set of target users about the workflow, current workaround, frequency, budget owner, and cost of delay.
4. Define one paid outcome unit and estimate source, compute, support, refund, and failure costs.
5. Test positioning with a landing page, concierge delivery, waitlist, or paid pilot.
6. Record conversion and retention evidence separately from Store metadata.
7. Re-run the Actor only when a fresh marketplace observation would change the decision.

This sequence prevents the report from becoming ceremonial analytics. Its job is to make the first research decision faster and more explainable, then hand off to stronger evidence.

Troubleshooting

The Dataset is empty

Read KVS OUTPUT. A compatible useful replay now has one free Dataset row and requestReplay:true; an empty Dataset instead points to invalid input, insufficient evidence, source failure, or delivery failure and is not automatically a successful zero-result report.

Confidence is low

Inspect dataGaps, source coverage, and observed cohort size. Narrow or rephrase the query, select more comparable categories, or wait for a later observation. Do not simply override confidence because the score supports the preferred roadmap answer.

A repeated run did not create a new row

Check whether the same canonical input and requestId were already completed. That is expected replay protection. Supply a new requestId only if you intend to purchase a new observation.

A run reached an event limit

Read the billing receipt in OUTPUT. A positive chargedCount means the current result event was delivered even if the same event reaches the configured limit. Zero charged events with a limit flag is not a delivered paid result.

Store results look unrelated

Tighten buyer-declared queries and categories. The Actor preserves search limitations rather than using an LLM to invent relevance. Always review visible comparator samples before acting.

FAQ

Is this an AI forecast? No. The workflow is deterministic and evidence-first. It structures public Store metadata and buyer-supplied context; it does not ask an LLM to predict success.

Does it access private Apify analytics? No. It uses public Store observations only.

Does a high score prove demand? No. It prioritizes validation under the documented method.

Can I use the output in a dashboard? Yes. Use Dataset REPORTS for rows and keep KVS OUTPUT for terminal and billing truth.

Can I automate a roadmap or price change from recommendedAction? No. safeToAutomate is false because the public evidence cannot support that irreversible decision by itself.

What should I cite internally? Cite the exact query, source timestamp, observed cohort, coverage, confidence, and data gaps—not only the headline score or label.

Does the highest score mean “build it now”? No. It means “validate this candidate first under the declared queries and observed evidence.” Interview buyers, test a landing page, and model delivery cost before committing engineering capacity.

Why must I submit at least two niches? The product is designed for comparative prioritization. With one idea, use Apify Actor Idea Validator; with multiple declared candidates, this Actor makes the ranking decision explicit.

Related Apify marketplace research tools

Actor	Best used for
Apify Niche Demand Radar	Measure the current direction of visible Apify Store user activity for one declared niche across comparable 7/30 or 30/90-day windows.
Apify Portfolio Expansion Planner	Rank buyer-supplied candidate niches by visible Store opportunity, textual adjacency to an existing Actor portfolio, and explicit novelty.
Apify Pricing Benchmark Advisor	Build an observed price distribution from visible pay-per-event prices in a declared public Apify Store cohort.

Support

When reporting a reproducible issue, include the run ID, Actor version, sanitized input, terminal OUTPUT status, and whether the Dataset contains a row. Never post an API token or signed storage URL. Feature requests are most useful when they name the buyer, decision, required public evidence, and acceptable false-positive or missing-data behavior.

Apify Market Gap Finder is intentionally a decision-support product: it collects bounded evidence, explains the calculation, preserves uncertainty, and tells a human what to validate next. That is more useful than a confident number whose source, coverage, and commercial meaning cannot be defended.