NCI GDC Project Summary Scraper

NCI GDC Project Summary Scraper

Scrape NCI GDC project summaries with case counts, file counts, and primary sites from the public API. Returns structured project metadata for every released cancer study. No login or API key required. Export to CSV, JSON, Excel, or XML.

The NCI Genomic Data Commons portal renders project lists in JavaScript, making direct scraping unreliable. This Actor reads the public GDC API endpoint directly, returning every released project's summary in a flat, consistent schema. Filter by primary cancer site to narrow results to a specific disease area.

Who uses it	What they scrape NCI Genomic Data Commons for
Cancer researchers	Catalog available genomic studies for a specific cancer type.
Bioinformaticians	Build an index of GDC projects and their data volumes for pipeline planning.
Data journalists	Gather case-count statistics across cancer projects for reporting.
Clinical trial coordinators	Identify GDC projects with sufficient case counts for cohort analysis.

What it does

This Actor collects NCI GDC project summaries from the public API and returns each project as a flat row with its ID, name, primary site, case count, and file count.

• Primary site filter: Narrow results to a single cancer site like Lung, Breast, or Brain.
• Summary counts: Each row includes case_count and file_count from the project summary.
• Bulk export: Pull up to 10 project summaries in one run and export to CSV, JSON, Excel, or XML.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with NCI Genomic Data Commons data

Catalog studies by cancer type.

A cancer researcher filters GDC projects by primary site 'Lung' to list every lung cancer study with its case and file counts.

Monitor data growth across projects.

A bioinformatics platform runs the Actor weekly to track file_count changes in all released GDC projects.

Gather statistics for a cancer report.

A data journalist pulls all GDC project summaries to compare case counts across the most-studied cancer sites.

Why choose this scraper

	What you get
No API key	The GDC public API requires no authentication or registration.
Structured output	Every project returns the same flat schema: ID, name, primary site, case count, file count.
Site filter	Restrict results to one cancer type instead of downloading all projects.
Scalable	Paid users can pull up to one million project summaries per run.

What a NCI Genomic Data Commons record looks like

Every record returns as one flat JSON row. Here is a real one from a run:

{
 "id": "TCGA-LUAD",
 "projectId": "TCGA-LUAD",
 "name": "Lung Adenocarcinoma",
 "caseCount": 585,
 "fileCount": 36740,
 "scrapedAt": "2026-09-26T03:08:45.298Z"
}

Every value above comes from a real run. A field a record does not have comes back as null.

Configure the run

Drive the Actor with an optional primary site filter and a max items limit. Filtering runs as each project is read so only matching cancer sites reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "maxItems": 10
}

A larger pull:

{
 "maxItems": 200
}

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account.
2. Open the NCI GDC Project Summary Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to NCI Genomic Data Commons through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=acquistion-automation/nci-gdc-project-summary-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check your primary site filter spelling. The API matches exact site names like 'Lung', 'Breast', or 'Brain'. Try leaving the filter empty to see all projects first.

Why does the run stop at 10 items?

Free Apify users are limited to 10 items. Upgrade to a paid plan and increase the maxItems input to pull more projects.

Why are some projects missing from the results?

The Actor only returns projects where released=true. Unreleased or embargoed projects are excluded by the API.

Why did I get a timeout or empty response?

The GDC API may occasionally be slow. Retry the run. If the problem persists, reduce maxItems or contact support.

FAQ

Question	Answer
Do I need an API key or GDC account?	No. The GDC public API is open and requires no authentication. This Actor calls it directly.
What fields does each row contain?	Each row returns the id, projectId, name, primarySite, caseCount, fileCount, scrapedAt, and error for a released GDC project.
Can I filter by cancer type?	Yes. Use the Primary Site Filter input to restrict results to a single site such as Lung, Breast, or Brain.
How many projects can I scrape in one run?	Free users are limited to 10 items for preview. Paid users can set maxItems up to 1,000,000.
Is the data real-time?	The Actor calls the live GDC API on each run, so you always receive the current project summaries.
What export formats are supported?	You can export the dataset to CSV, JSON, Excel, or XML from the Apify platform.
Does this scrape the GDC portal HTML?	No. The portal is a JavaScript application with no embedded data. This Actor reads the underlying public API directly.
Can I get project-level metadata beyond case and file counts?	This Actor focuses on the summary fields. For full project details, you would need a different endpoint or custom Actor.

Related actors

Browse the full Acquisition Automation collection for more scrapers.

🆘 Need help? Open an issue in the Issues tab of this Actor with your run ID, your input, and what you expected.

Pricing

This Actor uses pay-per-result pricing: $0.004 per result collected. You are billed only for the results you receive, so a run that returns nothing costs nothing.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by National Cancer Institute. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.