# ClinicalTrials.gov Scraper (`datalayer/clinical-trials-failure-intel`) Actor

Trial records with the whyStopped free text classified into safety, futility, slow accrual, funding and the rest — unreadable at scale across 34,000+ terminated studies until now. Plus enrolment failure, overdue records and how many stopped trials never posted results.

- **URL**: https://apify.com/datalayer/clinical-trials-failure-intel.md
- **Developed by:** [Datalayer](https://apify.com/datalayer) (community)
- **Categories:** Other, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 studies

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## ClinicalTrials.gov Scraper — Why Trials Stop

ClinicalTrials.gov is free, official and has a good search UI. Scraping the
study record and handing it back is not a product — the registry already does
that better.

The field nobody uses is **`whyStopped`**. When a sponsor halts a trial they
must say why, in free text. Two real examples:

> "Initial PI departed institution."

> "Study was prematurely terminated due to difficulties experienced in
> recruiting patients in a reasonable timeframe."

Completely different events. The first is administrative and says nothing about
the drug. The second is an **accrual failure** — the most common cause of death
in clinical research and a direct read on how hard that indication is to run.

As free text it is unusable at scale, which is why it sits unread across
**34,000+ terminated studies**. This Actor classifies it.

### Measured on 300 live melanoma trials

| Reason | Count |
|---|---|
| Slow accrual | 85 |
| Business decision | 38 |
| Safety | 37 |
| **Sponsor decision, no rationale given** | 21 |
| Futility / lack of efficacy | 18 |
| Funding | 18 |
| Administrative | 12 |
| Investigator or site | 9 |
| No reason given at all | 9 |
| COVID / regulatory / drug supply | 8 |
| Unclassified | 45 |

**197 of the 300 never posted results.** A trial that stops and publishes
nothing is how a negative finding disappears, and `resultsWithheld` counts them.

`sponsor_decision_undisclosed` is its own category on purpose. "Sponsor
decision" was the single most common thing sponsors wrote — a reason in form
only. Lumping it in with genuine strategic rationale would hide that.

### The other signals, all computed from the record

**`enrolledNobody`** — a stopped trial whose *actual* enrolment is one or zero.
It never got going, whatever reason was given.

**`isOverdue`** — the primary completion date is still marked `ESTIMATED` and
has already passed. The record was never updated to say what happened. Only
detectable by comparing the date's **type** against today, which is why nobody
reports it.

**`isLatePhase`** — a halted Phase 2 or 3 is a material event; a halted `NA`
study usually is not.

**`durationMonths`** — how long a trial survived before it was killed.

### Input

```json
{
  "condition": "melanoma",
  "stoppedOnly": true,
  "phases": ["PHASE2", "PHASE3"],
  "maxStudies": 500
}
```

| Field | Default | Notes |
|---|---|---|
| `condition` | `melanoma` | Disease or condition. |
| `intervention` | — | Drug, device or procedure. |
| `sponsor` | — | Lead sponsor only, not collaborators. |
| `searchTerm` | — | Extra registry search terms. |
| `stoppedOnly` | `true` | Applied registry-side, not after downloading. |
| `statuses` | — | Specific statuses; overrides `stoppedOnly`. |
| `phases` | — | `PHASE1`…`PHASE4`, `NA`. |
| `maxStudies` | `500` | Real ceiling — the registry is large. |
| `pageSize` | `100` | Up to 1000. |

An unfiltered query is refused rather than quietly downloading the whole
registry.

### Output

**`study`** — one row per trial:

```json
{
  "type": "study",
  "nctId": "NCT06185088",
  "status": "TERMINATED",
  "whyStopped": "Initial PI departed institution.",
  "stopReason": "investigator_or_site",
  "stopReasonGiven": true,
  "phase": "NA",
  "isLatePhase": false,
  "leadSponsor": "University of Michigan",
  "isIndustrySponsored": false,
  "enrollment": 1,
  "enrolledNobody": true,
  "durationMonths": 3,
  "resultsWithheld": false,
  "countries": ["United States", "Canada"],
  "url": "https://clinicaltrials.gov/study/NCT06185088"
}
```

**`analysis`** — one cohort summary with the full reason breakdown, results-
withheld rate, median months before stopping and the sponsor stopping the most
trials.

### Who uses this

- **Biotech and pharma competitive intelligence** — what kills trials in an
  indication before you commit to running one there
- **Site selection and CRO planning** — accrual failure rate by condition is
  the number that decides feasibility
- **Investors** — a Phase 3 halted for futility is a very different signal to
  one halted because the PI left
- **Research integrity** — the results-withheld rate, by sponsor

### Reliability

- Official ClinicalTrials.gov API v2. Public, unauthenticated, no key.
- Paging is by opaque `nextPageToken`. There is no offset parameter — passing
  one is silently ignored and every page comes back identical.
- Registry dates arrive as `YYYY`, `YYYY-MM` or `YYYY-MM-DD` and are normalised
  without inventing precision.
- One malformed record never fails the run; it lands in `RUN_SUMMARY`.

### Limits

- The classifier reads English free text written by thousands of different
  sponsors. On the live melanoma run **15% fell through to `other`** — those
  keep their raw `whyStopped` so you can read them yourself. The classifier is
  a filter over the sponsor's words, never a replacement for them.
- Where reasons overlap the strongest signal wins: a trial stopped "for safety
  after slow enrolment" is classified as a safety stop.
- `stopReason` is null for trials still running. It is not a prediction.
- Registry records are self-reported and frequently stale — which is precisely
  what `isOverdue` measures.

# Actor input Schema

## `condition` (type: `string`):

Disease or condition to search, for example melanoma, type 2 diabetes, alzheimer disease.

## `intervention` (type: `string`):

Drug, device or procedure name, for example pembrolizumab.

## `sponsor` (type: `string`):

Sponsor organisation, for example Pfizer or National Cancer Institute. Matches the lead sponsor only, not collaborators.

## `searchTerm` (type: `string`):

Any additional registry search terms, combined with the filters above.

## `stoppedOnly` (type: `boolean`):

Restrict to terminated, withdrawn and suspended studies. Applied as a registry filter, so the run does not download everything and throw most of it away.

## `statuses` (type: `array`):

Specific recruitment statuses to include. Overrides Stopped trials only.

## `phases` (type: `array`):

Trial phases to include. A halted Phase 2 or 3 is a material event; a halted NA study usually is not.

## `includeStudies` (type: `boolean`):

One row per study, with the classified stop reason and every derived flag.

## `includeAnalysis` (type: `boolean`):

One summary row: how many stopped, the reason breakdown, results-withheld rate and median time before stopping.

## `maxStudies` (type: `integer`):

How many studies to read. The registry holds 34,000+ terminated studies alone, so this is a real ceiling.

## `sortByNewest` (type: `boolean`):

Order by last update date descending, so recent activity comes back first.

## `pageSize` (type: `integer`):

Studies fetched per API call. Higher is faster; 1000 is the API maximum.

## `proxyConfiguration` (type: `object`):

Optional. ClinicalTrials.gov is a free public API and does not normally need one.

## Actor input object example

```json
{
  "condition": "melanoma",
  "stoppedOnly": true,
  "includeStudies": true,
  "includeAnalysis": true,
  "maxStudies": 500,
  "sortByNewest": true,
  "pageSize": 100,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `stopped` (type: `string`):

Terminated, withdrawn and suspended studies with the reason classified.

## `studies` (type: `string`):

Every study matched, with sponsor, phase, dates and site count.

## `analysis` (type: `string`):

How many stopped, why, and how many never posted results.

## `all` (type: `string`):

Studies and the cohort analysis together.

## `csv` (type: `string`):

Everything as a CSV download.

## `runSummary` (type: `string`):

The query run, how many matched, and anything that failed to parse.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "condition": "melanoma"
};

// Run the Actor and wait for it to finish
const run = await client.actor("datalayer/clinical-trials-failure-intel").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "condition": "melanoma" }

# Run the Actor and wait for it to finish
run = client.actor("datalayer/clinical-trials-failure-intel").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "condition": "melanoma"
}' |
apify call datalayer/clinical-trials-failure-intel --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,datalayer/clinical-trials-failure-intel"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/N85pRsxGRZf7xcccV/builds/mcAleoYyBMrSb9F9r/openapi.json