# Apify Dataset QA Gate (`leadops_lab/dataset-quality-auditor`) Actor Pass, warn, or stop Apify datasets before CRM import, enrichment, client delivery, or webhook automation. - **URL**: https://apify.com/leadops\_lab/dataset-quality-auditor.md - **Developed by:** [jiaxun mao](https://apify.com/leadops_lab) (community) - **Categories:** Automation, Developer tools, Lead generation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Apify Dataset QA Gate Pass, warn, or stop Apify datasets before CRM import, enrichment, client delivery, or webhook automation. This Actor is for teams that run scrapers repeatedly and need a quality gate before bad data flows into expensive or visible downstream steps. Use it after a scraper and before: - CRM import - enrichment APIs - Google Sheets exports - client lead-list delivery - n8n, Make, Zapier, or Apify webhook workflows ### Why use a QA gate? Deduplication tools clean rows. Enrichment tools add data. This Actor answers the earlier question: Should this dataset continue through the workflow at all? It returns: - `qaStatus`: `PASS`, `WARN`, or `FAIL` - `automationAction`: `continue`, `review`, or `stop` - failed quality checks with actual vs expected values - CRM-ready record count and percentage - duplicate count and percentage - field coverage for company, domain, email, phone, location, and category - sample messy rows for review - sample clean rows that can continue downstream - recommendations for cleanup, enrichment, or scoring ### Input options Use either: - `records`: paste raw records as JSON. - `sourceDatasetId`: select an existing Apify Dataset ID. Example: ```json { "sourceDatasetId": "YOUR_DATASET_ID", "requiredFields": ["companyName", "domain", "email", "phone", "location"], "passThresholds": { "minCrmReadyPercent": 80, "maxDuplicatePercent": 10, "minRequiredFieldCoveragePercent": 70 }, "maxRecords": 10000, "sampleSize": 25 } ```` ### Automation workflow 1. Run a lead, directory, product, review, or listing scraper. 2. Send the scraper Dataset ID into this Actor. 3. If `automationAction` is `continue`, send clean rows to CRM, Sheets, or enrichment. 4. If `automationAction` is `review`, route the dataset to manual review. 5. If `automationAction` is `stop`, block the workflow before wasting enrichment credits or importing bad data. ### Lead workflow For lead lists, run this Actor first. If the dataset passes, run Lead Intelligence Scorer to deduplicate, score, and prioritize the leads. Recommended chain: `scraper -> QA Gate -> Lead Intelligence Scorer -> CRM/export` ### Best fit - agencies validating client lead-list deliverables - operators running scheduled Apify scrapers - founders sending scraper output into Sheets or a CRM - automation builders who need a simple pass/fail signal # Actor input Schema ## `records` (type: `array`): Paste raw dataset records as JSON objects. Use this or sourceDatasetId. ## `sourceDatasetId` (type: `string`): Optional Apify Dataset ID to audit instead of inline records. ## `requiredFields` (type: `array`): Fields that must be present for a row to count as CRM-ready. ## `passThresholds` (type: `object`): Quality thresholds used to return PASS, WARN, or FAIL for downstream automations. ## `maxRecords` (type: `integer`): Maximum records to read and audit. ## `sampleSize` (type: `integer`): Maximum number of rows with missing required fields to include in sampleIssues. ## Actor input object example ```json { "records": [ { "company": "Acme Supply", "website": "https://acme.example", "email": "sales@acme.example", "phone": "+1 555 0100", "city": "Austin", "state": "TX" }, { "company": "Acme Supply", "website": "https://www.acme.example/contact", "city": "Austin", "state": "TX" } ], "requiredFields": [ "companyName", "domain", "email", "phone", "location" ], "passThresholds": { "minCrmReadyPercent": 80, "maxDuplicatePercent": 10, "minRequiredFieldCoveragePercent": 70 }, "maxRecords": 10000, "sampleSize": 25 } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "records": [ { "company": "Acme Supply", "website": "https://acme.example", "email": "sales@acme.example", "phone": "+1 555 0100", "city": "Austin", "state": "TX" }, { "company": "Acme Supply", "website": "https://www.acme.example/contact", "city": "Austin", "state": "TX" } ] }; // Run the Actor and wait for it to finish const run = await client.actor("leadops_lab/dataset-quality-auditor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "records": [ { "company": "Acme Supply", "website": "https://acme.example", "email": "sales@acme.example", "phone": "+1 555 0100", "city": "Austin", "state": "TX", }, { "company": "Acme Supply", "website": "https://www.acme.example/contact", "city": "Austin", "state": "TX", }, ] } # Run the Actor and wait for it to finish run = client.actor("leadops_lab/dataset-quality-auditor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "records": [ { "company": "Acme Supply", "website": "https://acme.example", "email": "sales@acme.example", "phone": "+1 555 0100", "city": "Austin", "state": "TX" }, { "company": "Acme Supply", "website": "https://www.acme.example/contact", "city": "Austin", "state": "TX" } ] }' | apify call leadops_lab/dataset-quality-auditor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=leadops_lab/dataset-quality-auditor", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Apify Dataset QA Gate", "description": "Pass, warn, or stop Apify datasets before CRM import, enrichment, client delivery, or webhook automation.", "version": "0.1", "x-build-id": "nPTTQ6nofRFIqaVTc" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/leadops_lab~dataset-quality-auditor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-leadops_lab-dataset-quality-auditor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/leadops_lab~dataset-quality-auditor/runs": { "post": { "operationId": "runs-sync-leadops_lab-dataset-quality-auditor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/leadops_lab~dataset-quality-auditor/run-sync": { "post": { "operationId": "run-sync-leadops_lab-dataset-quality-auditor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "records": { "title": "Inline records", "type": "array", "description": "Paste raw dataset records as JSON objects. Use this or sourceDatasetId." }, "sourceDatasetId": { "title": "Source Apify dataset", "type": "string", "description": "Optional Apify Dataset ID to audit instead of inline records." }, "requiredFields": { "title": "Required CRM fields", "type": "array", "description": "Fields that must be present for a row to count as CRM-ready.", "default": [ "companyName", "domain", "email", "phone", "location" ], "items": { "type": "string" } }, "passThresholds": { "title": "Pass thresholds", "type": "object", "description": "Quality thresholds used to return PASS, WARN, or FAIL for downstream automations.", "properties": { "minCrmReadyPercent": { "title": "Minimum CRM-ready %", "type": "integer", "description": "Minimum percentage of rows that must contain all required fields for the dataset to pass.", "minimum": 0, "maximum": 100, "default": 80 }, "maxDuplicatePercent": { "title": "Maximum duplicate %", "type": "integer", "description": "Maximum allowed percentage of duplicate rows before the dataset warns or fails.", "minimum": 0, "maximum": 100, "default": 10 }, "minRequiredFieldCoveragePercent": { "title": "Minimum required field coverage %", "type": "integer", "description": "Minimum coverage percentage required for each selected required field.", "minimum": 0, "maximum": 100, "default": 70 } }, "default": { "minCrmReadyPercent": 80, "maxDuplicatePercent": 10, "minRequiredFieldCoveragePercent": 70 } }, "maxRecords": { "title": "Maximum records", "minimum": 1, "maximum": 100000, "type": "integer", "description": "Maximum records to read and audit.", "default": 10000 }, "sampleSize": { "title": "Issue sample size", "minimum": 1, "maximum": 100, "type": "integer", "description": "Maximum number of rows with missing required fields to include in sampleIssues.", "default": 25 } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```