PubChem Scraper — Compounds, Properties & SMILES

Extract chemical compound data from PubChem in bulk — molecular formula, weight, SMILES, InChIKey, IUPAC name, XLogP, and synonyms. No API key or login required.

What does PubChem Scraper do?

PubChem Scraper connects to the official NCBI PubChem PUG REST API (pubchem.ncbi.nlm.nih.gov/rest/pug) — a free, keyless government endpoint — and retrieves structured chemical compound records in bulk. You can look up compounds by common name (e.g., "aspirin"), by PubChem CID (numeric identifier), or by keyword search that returns all matching compounds. For each compound, the actor fetches properties from the /compound/.../property/ endpoint and synonyms from the /compound/.../synonyms/ endpoint. Requests are throttled to ≤5 concurrent calls to respect NCBI's politeness policy. Results are streamed to an Apify dataset as they arrive, so you can start using data while the run continues. Runs of 1,000+ compounds are fully supported.

Who is it for?

• Cheminformatics researchers who need to build local compound databases from CID lists or drug name lists.
• Drug discovery teams looking for bulk property data (logP, formula, SMILES) without manually browsing PubChem's UI.
• Chemistry educators and students who want to programmatically retrieve compound data for reports, visualisations, or lab prep.
• Regulatory and compliance analysts who cross-reference ingredient lists against PubChem identifiers and InChIKeys.
• Data engineers and scientists building chemical knowledge graphs, ML training sets, or similarity pipelines that need SMILES strings at scale.

Use cases

• Drug pipeline screening: upload 500 candidate CIDs and extract SMILES + XLogP in one run for Lipinski rule-of-five filtering.
• Ingredient label auditing: input a list of ingredient names from a cosmetics or food product and retrieve canonical identifiers (InChIKey) for regulatory cross-referencing.
• Chemistry homework / teaching: fetch molecular formulas and IUPAC names for an entire curriculum compound list with a single API input.
• Chemical similarity search prep: extract CanonicalSMILES for a set of scaffolds to feed into RDKit or other cheminformatics pipelines.
• Synonym enrichment: look up trade names, CAS numbers, and other synonyms for compounds in your product catalogue.

Why use PubChem Scraper?

• 100% keyless — uses the public NCBI PubChem PUG REST API; no account or API key needed.
• 10 fields per compound — CID, name, IUPAC name, molecular formula, molecular weight, SMILES, InChIKey, XLogP, synonyms, PubChem URL.
• Bulk-ready — process thousands of names or CIDs in a single run; search mode auto-expands to up to 10,000 results.
• Three lookup modes — byName, byCid, and keyword search for maximum flexibility.
• Polite & reliable — respects NCBI's rate limits, retries transient errors, and handles 404 (not found) gracefully per compound.
• Export to CSV / JSON / Excel / JSONL — Apify dataset supports all formats out of the box; results are available via API or the Apify console.

What data can you extract?

Output fields

Field	Type	Description
cid	string	PubChem Compound ID (e.g., "2244")
name	string	Input name or CID used to look up this compound
iupacName	string	Official IUPAC name (e.g., "2-acetyloxybenzoic acid")
molecularFormula	string	Hill notation formula (e.g., "C9H8O4")
molecularWeight	string	Monoisotopic molecular weight in g/mol
canonicalSMILES	string	Canonical SMILES string for structure representation
inchiKey	string	IUPAC InChIKey hash for exact compound identification
xlogp	string	Computed XLogP (octanol-water partition coefficient)
synonyms	string	Semicolon-separated list of up to 10 synonyms / trade names
pubchemUrl	string	Direct PubChem compound page URL

JSON output example

{
  "cid": "2244",
  "name": "aspirin",
  "iupacName": "2-acetyloxybenzoic acid",
  "molecularFormula": "C9H8O4",
  "molecularWeight": "180.2",
  "canonicalSMILES": "CC(=O)Oc1ccccc1C(=O)O",
  "inchiKey": "BSYNRYMUTXBXSQ-UHFFFAOYSA-N",
  "xlogp": "1.2",
  "synonyms": "Aspirin; acetylsalicylic acid; Ecotrin; Bayer Aspirin; Disprin; 2-acetoxybenzoic acid; ASA; Aspro; Anacin; Entrophen",
  "pubchemUrl": "https://pubchem.ncbi.nlm.nih.gov/compound/2244"
}

How to use

Option A — Look up by name (byName)

Provide a list of chemical names. The actor resolves each name to a PubChem CID and fetches full properties.

Steps:

1. Set mode to byName.
2. Add one or more chemical names to the names array.
3. Click Start and wait for results in the Dataset tab.

Input example:

{
  "mode": "byName",
  "names": ["aspirin", "caffeine", "ibuprofen", "glucose", "paracetamol"],
  "proxyConfiguration": { "useApifyProxy": true }
}

Option B — Look up by CID (byCid)

Use this when you already have PubChem Compound IDs from another pipeline.

Steps:

1. Set mode to byCid.
2. Provide one or more numeric CIDs in the cids array.

Input example:

{
  "mode": "byCid",
  "cids": ["2244", "2519", "3672", "5793", "1983"]
}

Option C — Keyword search (search)

Search PubChem for all compounds matching a keyword. Great for exploring a chemical family (e.g., "penicillin", "vitamin").

Steps:

1. Set mode to search.
2. Set searchQuery to your keyword.
3. Optionally set maxResults to limit how many matching CIDs to process.

Input example:

{
  "mode": "search",
  "searchQuery": "penicillin",
  "maxResults": 50,
  "proxyConfiguration": { "useApifyProxy": true }
}

Input parameters

Parameter	Type	Default	Description
mode	string	byName	Lookup mode: byName, byCid, or search
names	array	[]	List of chemical names (byName mode)
cids	array	[]	List of PubChem CIDs as strings (byCid mode)
searchQuery	string	—	Keyword for compound search (search mode)
maxResults	integer	100	Max compounds to return (search mode)
proxyConfiguration	object	—	Apify proxy config (optional; datacenter is sufficient)

Full input JSON:

{
  "mode": "byName",
  "names": ["aspirin", "caffeine"],
  "cids": [],
  "searchQuery": "",
  "maxResults": 100,
  "proxyConfiguration": { "useApifyProxy": true }
}

Output example

{
  "cid": "2519",
  "name": "caffeine",
  "iupacName": "1,3,7-trimethylpurine-2,6-dione",
  "molecularFormula": "C8H10N4O2",
  "molecularWeight": "194.2",
  "canonicalSMILES": "Cn1cnc2c1c(=O)n(c(=O)n2C)C",
  "inchiKey": "RYYVLZVUVIJVGH-UHFFFAOYSA-N",
  "xlogp": "-0.1",
  "synonyms": "Caffeine; 1,3,7-trimethylxanthine; Guaranine; Mateina; Methyltheobromine; Theine; Coffeine; Koffein; Cafeina; No-Doz",
  "pubchemUrl": "https://pubchem.ncbi.nlm.nih.gov/compound/2519"
}

Tips for best results

• Use official names or common drug names in byName mode — PubChem resolves synonyms, trade names, and CAS numbers automatically.
• Prefer byCid when you have CID lists — it skips the name-resolution step and is faster.
• For broad families (e.g., "steroid", "antibiotic"), use search mode with maxResults ≤ 500 to keep run time manageable.
• Datacenter proxy is sufficient — PubChem is an open government API and does not block datacenter IPs; residential proxy is unnecessary.
• XLogP may be null for inorganic compounds or salts — this is expected and not an error.
• Synonyms list is capped at 10 entries — full synonym lists can have hundreds of entries; the top 10 are the most useful.
• IUPAC name may be long — for polymers and complex molecules, the IUPAC name can exceed 200 characters; store in a text column.
• Respect NCBI's fair-use policy — this actor throttles to ≤5 concurrent requests and 200 ms delay between calls.
• Export to Google Sheets using the integration below for collaborative review of compound libraries.
• Combine with Apify schedules to refresh compound databases automatically (e.g., weekly NDA ingredient cross-check).

Integrations

• Google Sheets — connect via Apify's Google Sheets integration; each run appends new compound rows automatically.
• Slack — configure a Webhook to notify your team when a bulk compound run completes.
• Zapier / Make — trigger downstream workflows (e.g., send a CSV to Dropbox, create a Jira ticket) when new compounds are scraped.
• Webhooks — Apify supports run-completion webhooks; POST results to your own endpoint in real time.
• Schedule — run the actor on a cron schedule (daily, weekly) to keep your chemical database fresh.

API usage

cURL

curl -X POST \
  "https://api.apify.com/v2/acts/logiover~pubchem-scraper/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"mode":"byName","names":["aspirin","caffeine"]}'

Node.js

import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('logiover/pubchem-scraper').call({
  mode: 'byName',
  names: ['aspirin', 'caffeine', 'ibuprofen'],
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient
client = ApifyClient("YOUR_TOKEN")
run = client.actor("logiover/pubchem-scraper").call(run_input={
    "mode": "byName",
    "names": ["aspirin", "caffeine", "ibuprofen"],
})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["molecularFormula"], item["canonicalSMILES"])

Use with AI agents (MCP)

PubChem Scraper is compatible with Apify's MCP (Model Context Protocol) server, allowing AI agents to retrieve chemical data on demand. Connect Claude, GPT, or any MCP-compatible agent to Apify and use a prompt like: "Use the PubChem scraper to look up the molecular formula, SMILES, and XLogP for aspirin, ibuprofen, and paracetamol, then export the results to a CSV." The agent handles input construction, run polling, and dataset retrieval automatically.

FAQ

Is an API key required?

No. PubChem Scraper uses the official NCBI PubChem PUG REST API (pubchem.ncbi.nlm.nih.gov/rest/pug), which is fully public and keyless. No account registration is needed.

How many compounds can I scrape in one run?

Practically unlimited for byName and byCid modes — just provide your full list. For search mode, set maxResults up to 10,000. Large runs of 1,000+ compounds typically complete within 10–20 minutes.

What are the rate limits?

NCBI requests a maximum of 5 requests per second from any single source. This actor throttles to ≤5 concurrent requests with a 200 ms inter-request delay to stay within limits. Transient errors are retried up to 3 times with exponential back-off.

What if a compound returns zero results?

If a name is misspelled or not indexed in PubChem, the actor logs a warning and continues with the next compound. The missing compound will not appear in the dataset. Check the actor logs for "No CIDs found for name:" messages.

Why are some fields null?

Some properties (especially xlogp and iupacName) may be absent for inorganic compounds, salts, or newly added compounds without full computational data. Null fields are expected and not errors.

Can I export results to Excel?

Yes. Open your dataset in the Apify console and click Export → Excel (XLSX). You can also export to CSV, JSON, JSONL, or XML.

How fast is the scraper?

With concurrency ≤5 and NCBI's politeness delay, expect roughly 10–15 compounds per minute. A 1,000-compound run takes approximately 70–100 minutes. byCid mode is faster than byName because it skips the name-resolution step.

Is the data up to date?

PubChem is updated continuously. This actor fetches live data from the API on every run, so results reflect the current state of the PubChem database.

Is scraping PubChem legal?

PubChem is a public database operated by the US National Institutes of Health (NIH). Its data is freely available under a public-domain licence. The PUG REST API is designed for programmatic access. This actor respects NCBI's fair-use policy and does not scrape HTML pages or circumvent any access controls. Always review NCBI's usage policies for your specific use case.

How often should I run it?

For drug pipeline monitoring or ingredient database maintenance, a weekly schedule is typically sufficient. PubChem's chemical properties rarely change; new compounds and updated records are added continuously. Use Apify Schedules to automate this.

What is XLogP?

XLogP is a computed estimate of the octanol-water partition coefficient (log P), a key parameter in drug-likeness assessment (Lipinski's rule of five). Higher values indicate more lipophilic (fat-soluble) compounds; lower values indicate more hydrophilic compounds. Oral drugs typically have XLogP between −2 and 5.

Can I use this with a Jupyter notebook?

Yes. Use the Apify Python client (pip install apify-client) to start a run and fetch results directly into a pandas DataFrame for cheminformatics analysis.

Is it legal?

PubChem is a free, public database provided by the US National Center for Biotechnology Information (NCBI), part of the National Institutes of Health. All data is in the public domain and explicitly designed for bulk programmatic access via the PUG REST API. This actor does not bypass any authentication, does not scrape private data, and respects NCBI's rate-limit guidelines. No personal data is collected or processed. Use of this actor is your responsibility — verify compliance with your institution's data policies and any downstream regulatory requirements for the chemical data you collect.

Related scrapers

• LinkedIn Company Scraper — extract company profiles for drug company intelligence.
• B2B Lead Scraper — find pharma and biotech companies by country and sector.
• GitHub Activity Scraper — monitor cheminformatics and bioinformatics open-source repositories.
• CVE Advisory Scraper — track security advisories for chemical data management software.