Clinical Trials & FDA Data Scraper

Extract clinical trial data from ClinicalTrials.gov and drug/device data from the openFDA APIs in one unified actor. Useful for pharma competitive intelligence, clinical research, regulatory monitoring, and biotech due diligence.

All data comes from official public government APIs. No authentication is required to run the actor; optional API keys unlock higher openFDA rate limits and optional AI summaries.

Data Sources

Source	openFDA endpoint	Data type
ClinicalTrials.gov API v2	clinicaltrials.gov/api/v2/studies	Clinical trial studies
openFDA FAERS	api.fda.gov/drug/event.json	Drug adverse event reports
openFDA SPL	api.fda.gov/drug/label.json	Drug labels / prescribing info
openFDA Drug Enforcement	api.fda.gov/drug/enforcement.json	Drug recalls
openFDA Drugs@FDA	api.fda.gov/drug/drugsfda.json	Drug approval history
openFDA MAUDE	api.fda.gov/device/event.json	Device adverse event reports
openFDA Device Enforcement	api.fda.gov/device/recall.json	Device recalls
openFDA 510(k)	api.fda.gov/device/510k.json	Device 510(k) clearances

The underlying databases are large (ClinicalTrials.gov holds 400,000+ studies; FAERS/MAUDE hold millions of reports), but a single run returns only the records that match your input, subject to the per-type caps described under Limitations.

Features

• Multi-source search — query ClinicalTrials.gov and openFDA in a single run.
• Unified output — every record carries a dataType discriminator so mixed results are easy to split.
• Optional AI summaries — per-trial narratives and an aggregate adverse-event analysis, generated with OpenAI (gpt-4o-mini). Requires an OpenAI API key and includeAISummary: true. When no key is supplied (or the OpenAI call fails), a deterministic rule-based summary is produced instead.
• Built-in rate-limit throttling — the actor spaces out requests to respect the source APIs.
• Public-domain data — ClinicalTrials.gov is a U.S. Government work; openFDA is released under CC0 1.0.

Use Cases

• Pharma competitive intelligence — monitor competitor trials and FDA submissions.
• Clinical research — find trials for systematic reviews and identify research gaps.
• Biotech due diligence — track drug pipelines and approval history.
• Regulatory monitoring — watch recalls, adverse events, and clearances.

Example Inputs

Search clinical trials for a condition

{
  "dataSources": ["clinicaltrials"],
  "condition": "breast cancer",
  "phase": ["PHASE3"],
  "status": ["RECRUITING"],
  "maxResults": 100
}

Get FDA data for a specific drug

{
  "dataSources": ["openfda"],
  "drugName": "Keytruda",
  "genericName": "pembrolizumab",
  "includeAdverseEvents": true,
  "includeDrugLabels": true,
  "includeRecalls": true,
  "maxResults": 500
}

Combined trial + FDA search

{
  "dataSources": ["clinicaltrials", "openfda"],
  "intervention": "pembrolizumab",
  "drugName": "Keytruda",
  "genericName": "pembrolizumab",
  "phase": ["PHASE2", "PHASE3"],
  "maxResults": 200
}

Device recalls and clearances

{
  "dataSources": ["openfda"],
  "deviceName": "pacemaker",
  "recallClass": "I",
  "includeRecalls": true,
  "includeDevice510k": true,
  "dateFrom": "2023-01-01"
}

Input Parameters

Data source selection

Parameter	Type	Default	Description
dataSources	array	["clinicaltrials"]	Sources to query: "clinicaltrials", "openfda"

ClinicalTrials.gov parameters

Parameter	Type	Description
condition	string	Disease or condition (query.cond)
intervention	string	Drug, device, or other intervention (query.intr)
searchTerm	string	Free-text term across all fields (query.term)
sponsor	string	Lead sponsor (query.spons); also used for the Drugs@FDA approval search
location	string	Geographic location (query.locn)
status	array	Trial statuses (e.g. RECRUITING, COMPLETED, ACTIVE_NOT_RECRUITING)
phase	array	Trial phases: EARLY_PHASE1, PHASE1, PHASE2, PHASE3, PHASE4, NA
studyType	string	INTERVENTIONAL, OBSERVATIONAL, EXPANDED_ACCESS
nctIds	array	Specific NCT IDs to retrieve

openFDA parameters

Parameter	Type	Description
drugName	string	Drug brand name (used across FAERS, SPL, recalls, approvals)
genericName	string	Generic drug name
deviceName	string	Device name (used across MAUDE, recalls, 510(k))
manufacturerName	string	Drug/device manufacturer
recallClass	string	Recall classification: I, II, III
productDescription	string	Recall product-description keywords

openFDA sections only run when a relevant parameter is supplied: drug adverse events / drug labels require drugName or genericName; drug recalls require drugName, genericName, or productDescription; drug approvals require drugName, genericName, or sponsor; device adverse events / 510(k) require deviceName or manufacturerName; device recalls require deviceName, manufacturerName, or productDescription.

Shared parameters

Parameter	Type	Default	Description
dateFrom	string	–	Start date (YYYY-MM-DD)
dateTo	string	–	End date (YYYY-MM-DD)
maxResults	integer	100	Max records per data type (min 1, max 25000). See per-type caps under Limitations

Feature flags

Parameter	Type	Default	Description
includeAISummary	boolean	false	Generate AI/rule-based summaries (AI requires openaiApiKey)
includeAdverseEvents	boolean	true	Include FAERS/MAUDE adverse events
includeDrugLabels	boolean	true	Include SPL drug labels
includeRecalls	boolean	true	Include drug/device recalls
includeDrugApprovals	boolean	true	Include Drugs@FDA approvals
includeDevice510k	boolean	true	Include 510(k) clearances

API keys (optional)

Parameter	Type	Description
openfdaApiKey	string	openFDA API key for a higher daily request quota. Free at open.fda.gov
openaiApiKey	string	OpenAI API key for AI summaries. Without it, summaries (when enabled) fall back to rule-based text

Output

Every record is pushed to the default dataset with a dataType field:

dataType	Meaning
clinical_trial	ClinicalTrials.gov study (normalized)
drug_adverse_event	FAERS adverse event report
adverse_event_summary	Aggregate AE summary (only when includeAISummary is on and events exist)
drug_label	SPL drug label
drug_recall	Drug enforcement / recall
drug_approval	Drugs@FDA approval
device_adverse_event	MAUDE device event
device_recall	Device enforcement / recall
device_510k	510(k) clearance
run_status	Emitted only when a run matches zero records, so the dataset is never empty

All records also include a dataSource label and an ISO scrapedAt timestamp. The full field list per record type is documented in .actor/dataset_schema.json.

Clinical trial fields (selection)

nctId, briefTitle, officialTitle, overallStatus, phases, studyType, conditions, keywords, interventions, armGroups, primaryOutcomes, secondaryOutcomes, leadSponsor ({name, class}), collaborators, enrollmentCount, enrollmentType, eligibilityCriteria, sex, minimumAge, maximumAge, locations (with geoPoint), locationCount, countries, startDate, completionDate, briefSummary, detailedDescription, isFDARegulatedDrug, isFDARegulatedDevice, url, and (optional) aiSummary.

Drug adverse event fields (selection)

safetyReportId, receiveDate, serious, seriousnessDeath (and other seriousness flags), patientAge, patientSex, patientWeight, drugs (with characterization/route/indication), reactions ({reaction, outcome, outcomeLabel}), occurCountry.

Other FDA record types

Drug labels expose indications, warnings, boxed warnings, and pharmacologic class; recalls expose recallNumber, classification, reasonForRecall, and dates; approvals expose applicationNumber, sponsorName, products, and submission history; 510(k) records expose kNumber, applicant, deviceName, and decision details. See the dataset schema for the complete list.

Key-Value Store

At the end of every run the actor writes a RUN_STATS record to the default key-value store with per-type counts:

{
  "clinicalTrials": 100,
  "drugAdverseEvents": 500,
  "drugLabels": 5,
  "drugRecalls": 2,
  "drugApprovals": 1,
  "deviceAdverseEvents": 0,
  "deviceRecalls": 0,
  "device510k": 0
}

Rate Limiting

The actor throttles its own requests to stay within the source limits:

API	Actor throttle	Notes
ClinicalTrials.gov v2	~50 req/min (1.2 s spacing)	No key required
openFDA (no key)	~40 req/min (1.5 s spacing)	Conservative default
openFDA (with key)	~240 req/min (0.25 s spacing)	Add openfdaApiKey

Limitations

• Per-type result caps. maxResults is applied per data type, not per run. Clinical trials and drug adverse events paginate up to maxResults (adverse events are hard-capped at 25,000 because openFDA does not allow skip beyond 25,000). Drug labels, drug recalls, drug approvals, device adverse events, device recalls, and 510(k) clearances are fetched in a single request and capped at 100 records each, regardless of a higher maxResults.
• openFDA rate/volume limits. openFDA allows roughly 240 requests/minute per IP and caps total volume by day: about 1,000 requests/day without an API key and a much higher daily quota with a free key. The actor throttles more conservatively without a key (see above). Large adverse-event pulls can consume many requests.
• ClinicalTrials.gov limits. The v2 API has no authentication but is rate-limited (the actor spaces requests ~1.2 s apart). Very large pulls are slow by design.
• AI summaries are optional and AI-generated. They only run when includeAISummary is true. With an openaiApiKey they use OpenAI gpt-4o-mini (subject to OpenAI cost and availability); without a key, or if the API call fails, they fall back to deterministic rule-based text. AI-generated summaries may contain errors and should not be treated as medical, regulatory, or investment advice.
• Coverage bounds. Results reflect only what the source APIs return for the given filters at request time. openFDA sections are skipped entirely unless a relevant drug/device parameter is provided (see the openFDA parameters note above).
• Static fetch. The actor performs direct REST calls to the public APIs — it does not render JavaScript, log in, or crawl the ClinicalTrials.gov / FDA websites.
• Empty runs. If no records match, the actor does not fail; it emits a single run_status record instead so the dataset is never empty. Individual source errors are logged and skipped rather than aborting the run.
• Data freshness & accuracy. Data is only as current and complete as the upstream government databases. FAERS/MAUDE are spontaneous-reporting systems: counts reflect reports, not verified incidence or causation.

Legal & Data License

• ClinicalTrials.gov — U.S. Government work, public domain.
• openFDA — CC0 1.0 Universal (public domain dedication).

Data is legal to use, modify, and redistribute commercially. Not medical advice.

Pricing

This actor runs on the Apify platform and consumes platform compute units. Cost scales with the number of records requested, the data types enabled, and whether AI summaries are turned on. See the actor's pricing on Apify and your plan's compute-unit rate for exact figures. If you enable AI summaries with your own OpenAI key, OpenAI usage is billed separately by OpenAI.

Support

For issues or feature requests, contact the author through the Apify platform.

---

Built by JCD | Data from ClinicalTrials.gov & openFDA