# Genomics & Variants Pack Scraper (`parseforge/genomics-variants-pack-scraper`) Actor Scrape variants, clinical significance, coordinates and ontology terms from dbSNP, ClinVar, UCSC and EBI OLS in one run. Export to CSV, JSON, Excel or XML. - **URL**: https://apify.com/parseforge/genomics-variants-pack-scraper.md - **Developed by:** [ParseForge](https://apify.com/parseforge) (community) - **Categories:** Other - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $19.00 / 1,000 results This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ![ParseForge Banner](https://github.com/ParseForge/apify-assets/blob/ad35ccc13ddd068b9d6cba33f323962e39aed5b2/banner.jpg?raw=true) ## 🧬 Genomics & Variants Pack Scraper > 🚀 **Export variant records, clinical significance, genomic coordinates and ontology terms from dbSNP, ClinVar, UCSC and EBI OLS in seconds.** Type a gene symbol such as BRCA1 or an rsID such as rs334 and the Actor queries 4 public genomics databases in one run: NCBI dbSNP, NCBI ClinVar, the UCSC Genome Browser and the EBI Ontology Lookup Service. Every match lands in one flat, 22-field row, ready for CSV, JSON, Excel or XML. A BRCA1 search alone reaches 44,306 dbSNP records, 16,062 ClinVar records and 594 OLS terms. ### 📋 What the Genomics & Variants Pack Scraper does Pulling variant data by hand means four portals, four query syntaxes and a spreadsheet to stitch them together. This Actor sends the same search term to each selected source, throttles NCBI to its 3 requests per second limit so no source is dropped, and collects records round-robin across sources until your item limit is reached. - 🧬 **dbSNP variants:** rsID, GRCh38 chromosome and position, gene symbols, variant class, functional consequence, HGVS notation and the first reported allele frequency. - 🧪 **ClinVar assertions:** germline classification (pathogenic, benign, conflicting and so on), review status, coordinates with start and stop, variant type, associated traits and VCV accession. - 🗺️ **UCSC coordinates:** gene and transcript matches with chromosome, start and end on hg38, hg19, mm39, mm10, rn7 or danRer11, plus the track they come from. - 📖 **OLS ontology terms:** matching terms from HGNC, NCIT, EFO, MONDO and other ontologies with description, synonyms and IRI. - 🔁 **One schema for all four:** identical column order in every row; values a source cannot provide are marked `N/A` (not applicable) or `Not Disclosed` (missing for that record), never left blank. | Who uses it | What they collect | |---|---| | Clinical geneticists | Every ClinVar classification and review status for the variants of a gene. | | Bioinformatics engineers | A local evidence cache of rsIDs, coordinates and consequences for annotation pipelines. | | Pharma R\&D teams | Allele frequencies and functional annotations linked to a target gene. | | Academic researchers | Ontology mappings and gene coordinates for systematic reviews. | ### 🎬 Full Demo (🚧 Coming soon) A short walkthrough video is in production. ### 📊 Output Each row is one variant, genomic feature or ontology term. Fields, in dataset order: | Field | Description | |---|---| | 🌐 `source` | `dbsnp`, `clinvar`, `ucsc` or `ols` | | 🆔 `id` | rsID, VCV accession, UCSC track/match id or ontology term id | | 📌 `title` | Human-readable name of the record | | 🔗 `url` | Link to the record on the source website | | 🏷 `recordType` | `variant`, `genomic feature` or `ontology term` | | 🧬 `chromosome` | Chromosome without the `chr` prefix | | 📍 `start` / `end` | Coordinates on the record's assembly | | 🗺 `assembly` | `GRCh38`, `hg38`, `hg19` and so on | | 🧪 `geneSymbol` | Gene symbol(s), comma separated | | 🔬 `variantType` | snv, del, ins, copy number loss and so on | | ⚕ `clinicalSignificance` | ClinVar germline classification or dbSNP clinical significance | | ⭐ `reviewStatus` | ClinVar review status | | 🧩 `consequence` | Functional or molecular consequence | | 📊 `alleleFrequency` | First reported allele frequency with its study | | 📝 `description` | HGVS notation, cDNA change, feature or term description | | 📚 `collection` | dbSNP build, ClinVar, UCSC track or ontology prefix | | 🔖 `accession` | RefSeq accession, VCV version, transcript id or term IRI | | 🩺 `traits` | ClinVar trait names (array) | | 🔁 `aliases` | SPDI, ClinVar aliases or ontology synonyms (array) | | 🕒 `scrapedAt` | ISO timestamp of the collection | | ❌ `error` | `null` on success; error rows carry only this field and are never charged | Three real records from a `BRCA1` run: ```json [ { "source": "dbsnp", "id": "rs2552282559", "title": "rs2552282559", "url": "https://www.ncbi.nlm.nih.gov/snp/rs2552282559", "recordType": "variant", "chromosome": "17", "start": "43127349", "end": "43127349", "assembly": "GRCh38", "geneSymbol": "BRCA1, NBR2", "variantType": "snv", "clinicalSignificance": "Not Disclosed", "reviewStatus": "N/A", "consequence": "upstream_transcript_variant, 2KB_upstream_variant, intron_variant", "alleleFrequency": "C=0.00001/1 (TOMMO)", "description": "NC_000017.11:g.43127349G>C, NC_000017.10:g.41279366G>C", "collection": "dbSNP build 157", "accession": "NC_000017.11", "traits": [], "aliases": ["NC_000017.11:43127348:G:C"], "scrapedAt": "2026-09-08T01:10:39.899Z", "error": null }, { "source": "clinvar", "id": "VCV004886868", "title": "GRCh38/hg38 17q21.31(chr17:43057598-43068066)x1", "url": "https://www.ncbi.nlm.nih.gov/clinvar/variation/4886868/", "recordType": "variant", "chromosome": "17", "start": "43057598", "end": "43068066", "assembly": "GRCh38", "geneSymbol": "BRCA1", "variantType": "copy number loss", "clinicalSignificance": "Likely pathogenic", "reviewStatus": "no assertion criteria provided", "consequence": "Not Disclosed", "alleleFrequency": "Not Disclosed", "description": "GRCh38/hg38 17q21.31(chr17:43057598-43068066)x1", "collection": "ClinVar", "accession": "VCV004886868.1", "traits": ["Breast-ovarian cancer, familial, susceptibility to, 1"], "aliases": [], "scrapedAt": "2026-09-08T01:10:40.664Z", "error": null }, { "source": "ucsc", "id": "mane/ENST00000357654.9", "title": "BRCA1 (NM_007294.4 / ENST00000357654.9)", "url": "https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg38&position=chr17%3A43044295-43125364", "recordType": "genomic feature", "chromosome": "17", "start": "43044295", "end": "43125364", "assembly": "hg38", "geneSymbol": "BRCA1", "variantType": "N/A", "clinicalSignificance": "N/A", "reviewStatus": "N/A", "consequence": "N/A", "alleleFrequency": "N/A", "description": "breast cancer type 1 susceptibility protein isoform 1", "collection": "MANE Select Plus Clinical: Representative transcript from RefSeq & GENCODE", "accession": "ENST00000357654.9", "traits": [], "aliases": [], "scrapedAt": "2026-09-08T01:10:43.239Z", "error": null } ] ``` ### ✨ Why choose this Actor | | What you get | |---|---| | **One search, four databases** | dbSNP, ClinVar, UCSC and OLS answered from a single term list. | | **Rate-limit safe** | NCBI calls are serialised and retried with backoff, so ClinVar is never dropped for hitting the 3 requests per second ceiling. | | **Exact item counts** | Records are collected round-robin across sources, so Max Items 10 means 10 rows, not 9 plus an error. | | **No API keys** | All four sources are open; nothing to register. | | **Six assemblies** | UCSC lookups on human, mouse, rat and zebrafish reference genomes. | | **Honest blanks** | `N/A` and `Not Disclosed` instead of empty cells, so filters in Excel behave. | ### 📈 How it compares to alternatives Single-source gene scrapers on the Store return gene identifiers from one database. This Actor works at the variant level and joins four sources. | Feature | This Actor | Single-source gene scrapers | |---|---|---| | dbSNP variants with coordinates and frequency | Yes | No | | ClinVar classification and review status | Yes | No | | UCSC coordinates on 6 assemblies | Yes | Sometimes | | EBI OLS ontology terms and synonyms | Yes | No | | Multiple search terms per run | Yes | Varies | | Ceiling | NCBI serves 3 requests per second; UCSC search returns the matches it indexes for a term | Depends on source | ### 🚀 How to use 1. [Create a free Apify account with $5 in credit](https://console.apify.com/sign-up?fpr=vmoqkp). 2. Open the [Genomics & Variants Pack Scraper](https://apify.com/parseforge/genomics-variants-pack-scraper?fpr=vmoqkp). 3. Enter one or more gene symbols or rsIDs, pick the sources, set Max Items and click **Start**. 4. Download the dataset as CSV, Excel, JSON or XML from the **Storage** tab, or read it through the API. A first run: ```json { "searchTerms": ["BRCA1"], "maxItems": 10, "sources": ["dbsnp", "clinvar", "ucsc", "ols"], "genome": "hg38" } ``` Free-plan runs return up to 10 records as a preview. [Upgrade your Apify plan](https://console.apify.com/sign-up?fpr=vmoqkp) to collect up to 1,000,000 records per run. ### 💼 Business use cases **🧬 Variant classification support.** A clinical laboratory pulls every ClinVar assertion and dbSNP frequency for a panel of genes each week and flags variants whose classification changed. **📊 Annotation pipeline cache.** A bioinformatics team schedules nightly runs for 50 genes and loads the rows into an internal database that powers their interpretation tool. **💊 Target assessment.** A pharma research group compares allele frequencies and functional consequences across candidate genes before selecting a lead. **📖 Knowledge graph enrichment.** A data team maps gene symbols to HGNC, NCIT and MONDO terms from OLS to link variant records with disease ontologies. ### 🔌 Automating Genomics & Variants Pack Scraper - **Make and Zapier:** trigger a run when a new gene is added to a sheet and push the rows to Airtable or Google Sheets. - **Slack:** post a summary when a scheduled run finds variants whose `clinicalSignificance` changed. - **Airbyte:** sync the dataset into BigQuery, Snowflake or Postgres. - **GitHub Actions:** rebuild a variant evidence file on every release of your pipeline. - **Google Drive:** save each run as CSV or Excel for a lab share. Every integration starts from the [Apify API](https://docs.apify.com/api/v2) or the [ApifyClient](https://docs.apify.com/api/client/js) for JavaScript and Python. ### 🌟 Beyond business use cases - **Research:** collect coordinates and ontology mappings for a systematic review in one export. - **Personal:** look up the public record of a variant from a consumer genotyping report. - **Non-profit:** patient advocacy groups can track new ClinVar submissions for their gene of interest. - **Experimentation:** feed dbSNP consequences and OLS synonyms into an LLM to prototype variant summaries. ### 🤖 Ask an AI assistant about this scraper Give an AI agent live access to the four databases through the Model Context Protocol: ```bash claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/genomics-variants-pack-scraper" ``` Then ask in plain language, for example "List the likely pathogenic ClinVar variants in TP53 and their review status". ### ❓ Frequently Asked Questions #### 🧬 Which databases does this Actor query? NCBI dbSNP, NCBI ClinVar, the UCSC Genome Browser and the EBI Ontology Lookup Service (OLS 4). Select any subset in the Sources field. #### 🔎 What search terms can I use? A gene symbol (BRCA1, TP53), an rsID (rs334) or a free-text term. Gene symbols are sent to NCBI as field-qualified queries so they answer in under a second. #### 🔑 Do I need an API key for NCBI, UCSC or OLS? No. All four APIs are public. The Actor identifies itself to NCBI with a tool name and contact email as their guidelines request. #### 📦 How many records can I collect? Free users get 10 records as a preview. Paid Apify users can set Max Items up to 1,000,000 per run. #### 🎯 Why do I get exactly Max Items rows? Sources are read round-robin, one record at a time, until the limit is reached. With four sources and Max Items 10 you get 3, 3, 2 and 2 rows. #### 🗺 Which genome assemblies does the UCSC source support? hg38, hg19, mm39, mm10, rn7 and danRer11. dbSNP and ClinVar coordinates are reported on GRCh38 regardless of this setting. #### 📖 What does the OLS source return? Ontology terms whose label or synonyms match the search term, with the ontology prefix, description, synonyms and IRI. #### ⚠️ What do `N/A` and `Not Disclosed` mean? `N/A` means the field does not exist for that source (an ontology term has no chromosome). `Not Disclosed` means the source carries the field but this record has no value (most dbSNP entries have no clinical significance). #### 🚦 Does it handle NCBI rate limiting? Yes. NCBI allows 3 requests per second per IP. The Actor serialises dbSNP and ClinVar calls with a 400 ms gap and retries HTTP 429 with exponential backoff. #### ❌ What happens if a source is down? The other sources keep going. The failed source adds one row containing only an `error` field, which is never charged, and the run finishes with a status message that says so. #### ⏰ Can I run this on a schedule? Yes. Create an Apify schedule to run the Actor daily or weekly and connect the dataset to your pipeline. #### 📄 What output formats are supported? CSV, JSON, Excel and XML from the Apify Console or API. ### 🔌 Integrate with any app The dataset is available through the Apify API the moment the run finishes. Use the `run-sync-get-dataset-items` endpoint for one-shot calls, webhooks to trigger downstream jobs, or the Make, Zapier, Airbyte and LangChain integrations listed on the Actor page. ### 🔗 Recommended Actors - [Healthgrades Doctor & Provider Scraper](https://apify.com/parseforge/healthgrades-scraper?fpr=vmoqkp) - [Website Content Crawler](https://apify.com/parseforge/website-content-crawler?fpr=vmoqkp) - [USAspending Awards Scraper](https://apify.com/parseforge/usaspending-scraper?fpr=vmoqkp) - [Audio Transcriber - Speech to Text](https://apify.com/parseforge/audio-transcriber?fpr=vmoqkp) *** > 💡 **Pro Tip:** browse the complete [ParseForge collection](https://apify.com/parseforge?fpr=vmoqkp). **🆘 Need Help?** [Open our contact form](https://tally.so/r/BzdKgA) with your run ID, your input and what you expected. > **⚠️ Disclaimer:** This Actor is an independent tool and is not affiliated with, endorsed by, or sponsored by the National Center for Biotechnology Information, the University of California Santa Cruz or the European Bioinformatics Institute. It collects only publicly available data. You are responsible for using the data in compliance with each source's terms of use and applicable data-protection laws. # Actor input Schema ## `searchTerms` (type: `array`): One term per line: a gene symbol (BRCA1, TP53), an rsID (rs334) or a free-text term. Every term is sent to each selected source. ## `maxItems` (type: `integer`): Free users: Limited to 10 items (preview). Paid users: Optional, max 1,000,000 ## `sources` (type: `array`): Which databases to query. Records are collected round-robin across the selected sources until Max Items is reached. ## `genome` (type: `string`): Reference assembly for the UCSC Genome Browser search (used only by the UCSC source). ## Actor input object example ```json { "searchTerms": [ "BRCA1" ], "maxItems": 10, "sources": [ "dbsnp", "clinvar", "ucsc", "ols" ], "genome": "hg38" } ``` # Actor output Schema ## `overview` (type: `string`): Key fields: source, id, title, coordinates, gene, clinical significance ## `fullData` (type: `string`): Complete dataset with all 22 fields # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "searchTerms": [ "BRCA1" ], "maxItems": 10, "sources": [ "dbsnp", "clinvar", "ucsc", "ols" ] }; // Run the Actor and wait for it to finish const run = await client.actor("parseforge/genomics-variants-pack-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "searchTerms": ["BRCA1"], "maxItems": 10, "sources": [ "dbsnp", "clinvar", "ucsc", "ols", ], } # Run the Actor and wait for it to finish run = client.actor("parseforge/genomics-variants-pack-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "searchTerms": [ "BRCA1" ], "maxItems": 10, "sources": [ "dbsnp", "clinvar", "ucsc", "ols" ] }' | apify call parseforge/genomics-variants-pack-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,parseforge/genomics-variants-pack-scraper" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/7sL57qOYZfM4ljmVo/builds/KvHKw9bYQZLupJ23W/openapi.json