ClinicalTrials.gov Scraper (Clinical Trial & Study Data)

Extract clinical trial records from the official ClinicalTrials.gov API v2: registry status, phase, conditions, interventions, enrollment, dates, sponsors and locations as structured JSON. Study and sponsor metadata only, with no participant or contact data.

What it does

This Actor queries the official, keyless ClinicalTrials.gov API v2 (https://clinicaltrials.gov/api/v2/studies) and returns one dataset item per clinical study. It is an official-API extractor, not a web scraper: no browser, login, proxy or CAPTCHA is involved.

Choose a last-update window (since, default the last 30 days, and optional until), then narrow the results by condition, intervention, overall status and phase. Pages are requested in LastUpdatePostDate descending order with an opaque pageToken; up to 1,000 studies are requested per page and up to 5,000 per run. Studies are deduplicated by NCT id within the run, so a record repeated across pages is delivered and charged once.

mode: "extractor" (default) delivers every matching study. mode: "monitor" remembers the NCT ids delivered by previous runs that share the same stateKey in the named key-value store clinicaltrials-gov-monitor-state and delivers only new studies. Failed requests, retries, empty pages and studies already emitted are never charged as results.

The data comes from the public registry maintained by the U.S. National Library of Medicine. Per the NLM Terms and Conditions, results carry the attribution "Courtesy of the U.S. National Library of Medicine"; NLM does not endorse this Actor. The Actor returns the studies that match the requested last-update window, so a dataset can lag the live registry and may not reflect the most current data. Use a recent since, or no until, when you need the freshest records.

Only study-level and sponsor-organisation metadata is returned. Participant records, contact persons, investigator names, emails and phone numbers are never requested or mapped.

Input: since (default: last 30 days, on LastUpdatePostDate), until, condition, intervention, status, phase, maxResults (default 1000, maximum 5000), mode (extractor or monitor), stateKey and strictSchema (default true).

Output: one dataset item per study with nct_id, brief_title, official_title, overall_status, study_type, phases, conditions, interventions, enrollment_count, start_date, primary_completion_date, completion_date, lead_sponsor, collaborators, locations, last_update_post_date, source_url and source_attribution, plus the common url, title and scrapedAt envelope. Format: JSON dataset.

Example output

Real record produced from the committed API response for NCT02684435 (fixture captured on 2026-10-20):

{
    "url": "https://clinicaltrials.gov/study/NCT02684435",
    "title": "Contrast-enhanced Ultrasound of the Kidney",
    "scrapedAt": "2026-10-20T08:30:00.000Z",
    "nct_id": "NCT02684435",
    "brief_title": "Contrast-enhanced Ultrasound of the Kidney",
    "official_title": "Contrast-enhanced Ultrasound for Complex Kidney Lesion Diagnosis in Patients With CKD (CEUS CKD)",
    "overall_status": "COMPLETED",
    "study_type": "INTERVENTIONAL",
    "phases": ["PHASE2"],
    "conditions": ["Chronic Kidney Disease", "Cystic Kidney Disease"],
    "interventions": ["DRUG: Perflutren lipid microsphere"],
    "enrollment_count": 63,
    "start_date": "2016-02",
    "primary_completion_date": "2018-07-09",
    "completion_date": "2018-07-09",
    "lead_sponsor": "University of North Carolina, Chapel Hill",
    "collaborators": [
        "Vanderbilt University",
        "North Carolina Translational and Clinical Sciences Institute"
    ],
    "locations": [
        { "country": "United States", "state": "North Carolina", "city": "Chapel Hill" },
        { "country": "United States", "state": "Tennessee", "city": "Nashville" }
    ],
    "last_update_post_date": "2019-09-26",
    "source_url": "https://clinicaltrials.gov/study/NCT02684435",
    "source_attribution": "Courtesy of the U.S. National Library of Medicine"
}

Partial registry dates are preserved exactly as published, so start_date can be 2016-02 as well as 2016-02-18.

Pricing

Pay-per-event pricing (.actor/pay_per_event.json, referenced from actor.json as payPerEvent):

Event	Charged when	Price
apify-actor-start	a run starts (synthetic, automatic)	0.00005 USD per run (Apify-recommended default; includes 5 seconds of free compute)
apify-default-dataset-item	a study is saved to the dataset (synthetic, automatic)	0.003 USD per result = 3.00 USD per 1,000 results

There is no custom per-item event and no Actor.charge() call: the platform already charges apify-default-dataset-item per pushed item, so adding another event would bill users twice. Failed requests, retries, empty pages and studies already emitted by a monitor run are not charged as results. No proxy is required and no proxy cost is added.

The price sits inside the direct-competition band for ClinicalTrials.gov data on the Apify Store (about 0.01-12 USD per 1,000 studies) and inside Steadfetch's 1-10 USD band.

Measured cost basis per 1,000 results (no browser, no proxy): one API page request (page size 1,000) plus JSON parsing and schema validation. The local fixture pipeline processed and validated 1,000 studies in 74 ms of compute, about USD 0.0000011 at 256 MB and Apify's USD 0.20 per GB-hour compute rate. Even a conservative 60 seconds of total runtime would cost about USD 0.00085 per 1,000 results, so the cost basis is below USD 0.01 per 1,000 results and the gross margin is above 99%. The first paid platform run will replace this local measurement with the billed figure.

FAQ

Do I need an API key, account or login? No. The ClinicalTrials.gov API v2 studies endpoint is public and keyless. The Actor needs no credentials.

Does it need a browser? No. The source returns structured JSON, so plain HTTP is roughly 10x cheaper than a browser-based Actor.

Does it scrape the ClinicalTrials.gov website? No. It uses the official API v2 endpoint and never parses the public search pages.

Does it return personal data? No. Only study-level metadata and sponsor-organisation names are mapped. Participant records, contact persons, investigators, emails and phone numbers are never requested.

How does monitor mode avoid charging twice? It stores the NCT ids it has delivered under stateKey in the named key-value store clinicaltrials-gov-monitor-state (the default store is run-scoped, so a named store is required to survive scheduled runs) and skips those ids on later runs. Only new studies reach the dataset and trigger the per-result event.

What happens when no study matches? The run finishes successfully with an empty dataset. Only the start event applies; there is no per-result charge.

Are partial dates supported? Yes. ClinicalTrials.gov publishes some dates as YYYY-MM or YYYY. They are preserved as published instead of being invented or rounded.

Is the output schema stable? Yes. New fields are additive and the required fields in .actor/dataset_schema.json do not change.

Maintenance

Maintenance status: the Actor is registered in the Steadfetch daily catalog QA workflow. The label Tested daily. Last verified: {YYYY-MM-DD}. will be published only after the first successful scheduled run is logged. Until then no verified date is claimed.

The source is versioned by the public ClinicalTrials.gov API v2. Committed API fixtures and schema tests catch response-shape changes in CI, and any break in the daily run is reported as an issue in the Steadfetch Actors repository.

Keywords

clinical trials, ClinicalTrials.gov, clinical study data, trial status, clinical trial API, JSON dataset

Input reference

Minimal input (the specification test case):

{
    "since": "2026-10-01",
    "maxResults": 10
}

Filtered monitor input:

{
    "since": "2026-09-20",
    "condition": "breast cancer",
    "status": "RECRUITING",
    "phase": "PHASE2",
    "maxResults": 1000,
    "mode": "monitor",
    "stateKey": "breast-cancer-phase2"
}

Field	Type	Description
since	string	Optional YYYY-MM-DD lower bound on LastUpdatePostDate. Default: last 30 days.
until	string	Optional YYYY-MM-DD upper bound on LastUpdatePostDate. Default: no upper bound.
condition	string	Filter by condition or disease.
intervention	string	Filter by intervention or treatment.
status	string	One overall status or a comma-separated list, for example RECRUITING,ACTIVE_NOT_RECRUITING.
phase	string	EARLY_PHASE1, PHASE1-PHASE4 or NA. Short forms such as 2 are accepted.
maxResults	integer	Default 1000, maximum 5000.
mode	string	extractor (default) or monitor.
stateKey	string	Key-value store key used to remember emitted NCT ids in monitor mode.
strictSchema	boolean	Default true; fail the run on schema violations.

Development

npm install
npm test                # lint + typecheck + offline Vitest suites (no credit)
npm run publish:check   # Steadfetch publish gate
npm run build && npm run verify:build   # production entrypoint guard
npm run schemas:validate
CLINICALTRIALS_LIVE_TESTS=1 npm run test:live   # optional live API check (no Apify credit)

The default test suite runs against committed API fixtures and never touches the external network or Apify. The live suite is opt-in. For a credit-free end-to-end run, point CLINICALTRIALS_API_BASE_URL at a loopback server that serves test/fixtures/studies-page-*.json and run apify run --input ...; production always defaults to the official API host.

Project structure:

.actor/actor.json          Actor definition (payPerEvent, dataset schema, output schema)
.actor/input_schema.json   INPUT options (dates, condition, intervention, status, phase, maxResults, mode)
.actor/dataset_schema.json Item JSON Schema (fields) + table view for the Store
.actor/pay_per_event.json  PPE pricing (two synthetic platform events)
src/fetch.ts               ClinicalTrials.gov API v2 client: filters, pagination, retry/backoff
src/extract.ts             Pure extraction and normalization (fixture-tested, no network)
src/dedupe.ts              Pure NCT-id dedupe and monitor-state merge helpers
src/monitor-state.ts       Named key-value store persistence for monitor mode
src/main.ts                Entry point: fetch, monitor filter, validate, push
src/validate-output.ts     Ajv validator shared by the run and the tests
test/                      Offline API fixtures, Vitest suites and an optional live test