Dataset Schema Guard

Stop malformed scraper output before it reaches automations, APIs, customers, or databases.

Dataset Schema Guard validates every record in an existing Apify dataset—or a pasted inline sample—against explicit field rules. Clean records are copied unchanged into a valid dataset. Failed records go to a separate rejected dataset with the original source index, the untouched record, and an exact list of errors.

What it checks

• Required nested fields using dot paths such as contact.email
• JSON-compatible types: string, number, integer, boolean, array, and object
• Empty strings, arrays, and objects
• Numeric minimum and maximum values
• String and array length limits
• JavaScript regular expressions
• Exact allowed-value lists
• Duplicate values within the source dataset
• Unexpected top-level fields when strict mode is enabled

The source dataset is read-only and is never modified.

Leave Source dataset blank to validate the inline records shown in the input form. When a source dataset is selected, it takes precedence and the inline demo records are ignored.

Example input

{
  "sourceDatasetId": "YOUR_DATASET_ID",
  "rules": [
    {
      "field": "id",
      "required": true,
      "type": "string",
      "allowEmpty": false,
      "unique": true
    },
    {
      "field": "contact.email",
      "required": true,
      "type": "string",
      "pattern": "^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$"
    },
    {
      "field": "score",
      "required": false,
      "type": "number",
      "min": 0,
      "max": 100
    },
    {
      "field": "status",
      "type": "string",
      "allowedValues": ["new", "approved", "rejected"]
    }
  ],
  "rejectUnknownTopLevelFields": false,
  "maxErrorsPerRecord": 20,
  "maxItems": 100000
}

Outputs

Each run produces:

• Valid dataset — original records that passed every rule, unchanged and ready for downstream use.
• Rejected dataset — wrappers containing sourceIndex, errorCount, errors, and the original record.
• Result manifest — dataset IDs, API URLs, and item counts for both result datasets.
• OUTPUT summary — processed, valid, rejected, total-error, completion, and per-error-code totals.

Each validation error includes the field, stable error code, human-readable message, expected value, and actual value when useful. Duplicate errors also point to the first source index where the value appeared.

Uniqueness behavior

For a rule with unique: true, the first appearance of a value is accepted. Later appearances are rejected and reference the first record's zero-based source index. Missing optional values do not participate in uniqueness checks.

Pricing

Pay per source record validated:

• Actor start: Apify's standard low-cost start event
• validated-record: $0.00002 per record ($0.02 per 1,000)

One source record creates at most one validation charge, regardless of the number of field rules or errors. The Actor stops cleanly before processing unpaid records when the run's spending limit is reached.

Limits and safety

• Up to 100 field rules per run
• Up to 1,000,000 source records, controlled by maxItems
• Up to 100 recorded errors per rejected record
• Regular expressions are compiled before dataset processing begins
• No proxy, browser, LLM, or external API is used
• Runs under Apify limited permissions and accesses only the selected source dataset plus its own run storages

Local development

npm install
npm test
npm start

Built by Orange Pin Labs: practical plugins, APIs, and applications for stubborn operational problems.