KEGG Pathways Scraper

KEGG Pathways Scraper

Scrape KEGG pathways, modules, and orthology entries from any organism, up to a million per run. Each entry returns its full detail, including genes, compounds, and reactions. No API key or registration required. Export to CSV, JSON, Excel, or XML.

KEGG is a foundational knowledge base for systems biology, but its REST API returns flat text that is hard to parse and requires manual pagination. This actor queries the KEGG API directly, retrieves structured pathway, module, and orthology data, and returns each entry in one consistent schema. You can list all entries in a database, search by keyword, or fetch full detail for specific IDs.

Who uses it	What they scrape KEGG for
Bioinformaticians	Fetching full pathway maps with gene and compound lists for enrichment analysis.
Systems biologists	Comparing metabolic modules across different organisms for evolutionary studies.
Pharma researchers	Extracting drug-target pathway associations for candidate screening.
Academic labs	Building local databases of KEGG Orthology assignments for newly sequenced genomes.

What it does

This Actor collects KEGG pathway, module, and orthology entries by list, keyword search, or specific ID, and returns each one as a flat row.

• 📋 List mode: enumerate all entries in a chosen KEGG database, optionally filtered by organism code.
• 🔍 Find mode: search any KEGG database by free-text keyword and retrieve matching entries.
• 🎯 Get mode: fetch full detail records for up to 10 specific KEGG IDs in a single run.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with KEGG data

🧬 Build a metabolic model.

A systems biologist lists all human pathways, then fetches full detail for glycolysis and TCA cycle to extract gene and reaction lists for constraint-based modeling.

🔬 Compare pathway conservation.

A researcher lists KEGG modules for E. coli and yeast, then compares the presence and absence of specific metabolic modules across the two organisms.

💊 Screen drug targets.

A pharma scientist searches the Drug database for a candidate compound, then retrieves its associated pathways to identify potential off-target effects.

📊 Annotate a new genome.

A bioinformatician lists all KO entries for a newly sequenced bacterial organism to build a first-pass functional annotation of its predicted genes.

Why choose this scraper

	What you get
No API key	Queries the public KEGG REST API with no registration or authentication.
Structured output	Parses flat text into a clean, flat schema ready for analysis.
Organism filtering	Restrict list queries to a single organism using standard KEGG codes.
Multi-ID fetch	Retrieve up to 10 full detail records in one run using the Get mode.

How it compares

This actor scrapes full KEGG pathway detail including genes, compounds, and reactions, while the KEGG Genes Scraper focuses on gene identifiers and descriptions.

Feature	ParseForge	KEGG Genes Scraper
Full pathway detail with genes and compounds	Yes	Not listed
Keyword search across databases	Yes	Yes
Organism-specific filtering	Yes	Not listed
Multi-ID detail fetch	Yes	Not listed
List all entries in a database	Yes	Not listed

Configure the run

Drive the Actor by choosing a query mode, a KEGG database, and an optional organism code or search term, and set a maximum number of pathways to collect. The Input tab lists every parameter.

A first run with the defaults:

{
  "maxItems": 10,
  "mode": "list",
  "database": "pathway",
  "organism": "hsa"
}

A larger pull:

{
  "maxItems": 200,
  "mode": "list",
  "database": "pathway",
  "organism": "hsa"
}

Pricing

Pay-per-result: $0.024 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$2.40
1,000 results	$24.00
10,000 results	$240.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the KEGG Pathways Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to KEGG through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/kegg-pathways-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check that your search term matches an entry in the selected database. For List mode, verify the organism code is valid for that database. Try a broader search term or switch to List mode to see available entries.

Why does the organism filter not work?

The organism code only filters results in List mode for the pathway, module, and ko databases. For other databases or modes, this field is ignored.

Why did my Get request fail?

Ensure you are using valid KEGG identifiers separated by a plus sign, with no spaces. The maximum is 10 IDs per request. Also confirm the IDs belong to the database you selected.

Why is the output missing some fields?

Not all KEGG entries contain the same level of detail. A pathway entry will include genes and compounds, while a brite entry has a different structure. The schema captures all available fields for each entry type.

Why is the run slow?

The KEGG API has rate limits. When listing large databases or fetching many entries, the actor respects these limits, which can extend the run time. Reduce the maximum items or use a more targeted query.

FAQ

Question	Answer
Do I need a KEGG license or API key to use this?	No. This actor uses the public KEGG REST API, which does not require registration or an API key for standard access.
What is the maximum number of pathways I can scrape in one run?	You can set the maximum up to 1,000,000 entries. The actual number retrieved depends on the database and mode you select.
Can I search for pathways by keyword?	Yes. Use the Find mode and enter a search term like 'glycolysis' or 'apoptosis' to retrieve matching entries from the chosen database.
How do I get the full detail for a specific pathway?	Switch to Get mode and enter one or more KEGG identifiers separated by a plus sign, for example 'hsa00010+hsa00020'.
What organism codes can I use?	You can use any standard KEGG organism code, such as 'hsa' for human, 'mmu' for mouse, 'eco' for E. coli, or 'sce' for yeast.
Which KEGG databases are supported?	All major databases are supported, including Pathway, Module, KO (Orthology), Compound, Reaction, Enzyme, Disease, Drug, and several genome databases.
Does the organism filter work with every database?	The organism code filter applies only to List mode with the pathway, module, and ko databases. It is ignored for other databases and modes.
What format is the output data in?	You can export your results to CSV, JSON, Excel, or XML directly from the Apify dataset.
Can I fetch more than 10 IDs at once in Get mode?	The KEGG API limits Get requests to 10 identifiers at a time. To fetch more, run the actor multiple times with different ID batches.
Is this actor suitable for commercial use?	This actor queries the public KEGG API. You are responsible for complying with the KEGG terms of use for any commercial application.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by Kanehisa Laboratories. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.