Scraper monitoring and API monitoring for your whole pipeline: monitor the health of your Actors, scrapers, and API keys. Catches schema drift, empty results, stale data, and quota exhaustion. One alert when something degrades, one notice when it recovers. Learn more: datasourcepulse.com
Enterprise / managed onboarding. Supply only this token; your full connector manifest is loaded from the operator's private KV store. Leave blank for Mode A (self-serve) where you configure checks directly below.
Type:string
Connector checks (Mode A · self-serve)
checks
Optional
Array of connector checks to run; billed $0.02 per item. Each item is ONE of two object shapes.
CREDENTIAL CHECK: { "type": "credential", "id": string (unique key), "label": string (human name), "vendor": one of ["serpapi","apify","hunter","openai","anthropic","twitter","reddit","github","stripe","resend","producthunt","gemini","groq","mistral","cohere","perplexity","together","openrouter","firecrawl","exa","tavily","brightdata","scrapingbee","zenrows","elevenlabs","deepgram","assemblyai","replicate","huggingface","pinecone","supabase","airtable","stackexchange","google_oauth","youtube","serper","brave","generic_http"], + credential fields for that vendor.
ACTOR CHECK: { "type": "actor", "id": string (unique key), "label": string, "actorId": string (e.g. "clockworks~tiktok-scraper"), "apifyToken": string (Actor read+run-read scope), "enableVolumeBaseline": boolean, "enableCostBaseline": boolean, optional "stalenessThresholdDays": int (default 30), "volumeDropPct": int (default 30), "costSpikePct": int (default 50), "zeroYieldRunThreshold": int (default 3; 0 disables) }.
zeroYieldRunThreshold: alert when this many consecutive most-recent SUCCEEDED runs each produced 0 dataset items (silent zero-yield: run status green, dataset empty · run status alone never reveals this). Needs no baseline, so it works from a source's very first runs. Set 0 for sources where an empty result is routinely legitimate.
Set enableVolumeBaseline/enableCostBaseline to false when each run uses different inputs (varying queries), or you will get false alarms. Optional on BOTH shapes: "impact": string (max 200 chars) · what breaks downstream when this connector fails; echoed verbatim in every alert so the operator's first question ("what breaks?") is answered in the alert itself. Optional on CREDENTIAL checks: "quotaSoftCap": number · YOUR OWN budget line in the vendor's native usage unit (searches for serpapi, USD for apify/openrouter, requests for hunter, credits for firecrawl/scrapingbee, characters for elevenlabs, API calls for tavily/stackexchange); quota warnings and the 100%-crossed warning state are evaluated against your cap, with the vendor-account percentage always shown alongside.
See README for per-vendor examples.
Type:array
Recheck only these connector ids (subset recheck)
onlyConnectorIds
Optional
On-demand subset recheck for preflight gates: when set, ONLY the listed connector ids (from your checks array, or from your managed manifest in Mode B) are probed and billed this run · e.g. recheck a single credential immediately before an expensive job instead of paying for the full roster. Unknown ids stop the run before any connector is charged and write a guidance record listing the known ids. Leave empty for a full run. Works in both Mode A and Mode B.
Type:array
Alert email address
emailTo
Optional
Email address to receive state-change alerts. Managed (Mode B) plans get native email delivery; in Mode A this address is stamped into a structured EMAIL_ALERT log event for relaying via an Apify webhook to your email provider (see README, Alerting). Leave blank to skip email alerts.
Type:string
Webhook URL (Slack, Discord, or any HTTP endpoint)
webhookUrl
Optional
Webhook endpoint that receives a JSON POST on every state transition. The default body is a Slack/Discord-compatible notification card; set webhookFormat to "json" for a raw structured event body instead.
Type:string
Webhook signing secret (optional)
webhookSecret
Optional
When set, every webhook POST includes X-DP-Signature: sha256=, so your receiver can verify the alert genuinely came from Datasource Pulse.
Type:string
Webhook body format
webhookFormat
Optional
"slack" (default) posts a Slack/Discord-compatible notification card · pick this when a person reads the alerts in a chat channel. "json" posts the raw structured event object with top-level fields transition, connector_id, connector_label, failure_class, message, last_healthy_at, occurred_at, recommended_action, impact (present when the check declares one), and metadata · pick this when a program consumes your alerts and indexes them by field. X-DP-Signature signs the raw body in both formats.
Type:string
Default:slack
Options:
slackjson
Require {"ok":true} webhook acknowledgement
webhookExpectJsonOk
Optional
When enabled, a 2xx webhook response must carry the JSON body {"ok":true} to count as delivered. Guards against catch-all 200 pages (framework fallback routes) silently swallowing alerts. Leave off for Slack/Discord · they acknowledge with plain "ok".
Type:boolean
Default:false
Quota warning threshold (%)
quotaWarningPct
Optional
Fire a warning alert when a credential's usage exceeds this percentage of its quota limit. Default: 80.
