# NSF Research Awards Scraper (`automation-lab/nsf-research-awards-scraper`) Actor Extract NSF award and grant records from Research.gov with PI, institution, funding, dates, program, and raw award data. - **URL**: https://apify.com/automation-lab/nsf-research-awards-scraper.md - **Developed by:** [Stas Persiianenko](https://apify.com/automation-lab) (community) - **Categories:** Education - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per event This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## NSF Research Awards Scraper Extract public National Science Foundation award and grant records from Research.gov into clean Apify datasets for research intelligence, grant monitoring, prospecting, and academic market analysis. ### What does NSF Research Awards Scraper do? This actor searches the public NSF / Research.gov Award API and exports normalized award records. It captures award titles, abstracts, principal investigators, recipient institutions, obligated funding, dates, program data, and source URLs. Use it when you need repeatable NSF grant data without manually clicking through Award Search pages. ### Who is it for? ๐Ÿซ University research offices tracking recent awards by topic, state, institution, or program. ๐Ÿ’ผ Grant consultants building prospect lists of funded PIs and active research teams. ๐Ÿงช R&D and academic sales teams monitoring funded labs and emerging research areas. ๐Ÿ“Š Market intelligence teams analyzing NSF funding trends by geography or program. ### Why use it? The actor turns public award search results into structured rows that can be exported as JSON, CSV, Excel, XML, or RSS from Apify. It uses the public API directly, so runs are lightweight and do not need browser automation. Raw API records can be preserved for audit trails and downstream custom mapping. ### What data can you extract? The dataset includes award IDs, titles, abstracts, agency, awardee institution, city, state, country, UEI, principal investigators, PI emails when available, funding amounts, dates, program names, directorate/division codes, performance location, and source links. Optional rawAward keeps the original Research.gov response object. ### How much does it cost to scrape NSF research awards? Pricing is pay-per-event: a $0.005 run-start charge plus a per-award record charge. The formula-derived BRONZE price is $0.000023219 per saved award, with lower unit prices on higher Apify tiers (SILVER $0.000018111, GOLD $0.000013932, PLATINUM/DIAMOND $0.00001). The input prefill is intentionally small enough for a cheap first test, and larger exports scale linearly by saved award record. A 100-award validation run used about $0.00056 of platform resources before Apify billing margin, so the actor remains profitable at the published PPE rates. ### How to use 1. Enter a keyword such as artificial intelligence, climate, quantum, CAREER, or SBIR. 2. Optionally add award date range, awardee state, performance state, funding program, agency, or program officer filters. 3. Set maxItems to the number of award records you want. 4. Run the actor and export the dataset in your preferred format. ### Input options keyword: free-text NSF award search. dateStart/dateEnd: Research.gov award date bounds in MM/DD/YYYY format. awardeeStateCode: two-letter state for recipient institution. performanceStateCode: two-letter state for where the work is performed. fundProgramName: program text such as Computer, Biology, CAREER, or SBIR. agency: normally NSF. programOfficerName: optional NSF program officer name. maxItems: maximum awards to save. includeRaw: include the original API record. ### Output example Each result is one NSF award record with normalized fields and a source URL. Example fields: awardId, title, abstractText, awardeeName, primaryPiName, principalInvestigators, fundsObligatedAmount, awardDate, startDate, expirationDate, fundProgramName, directorate, division, performanceStateCode, sourceUrl, scrapedAt, rawAward. ### Tips for better results Use specific topic phrases when broad terms return too many awards. Combine keyword with dateStart and dateEnd for recent-award monitoring. Use awardeeStateCode for institution prospecting by state. Use fundProgramName for program-focused workflows such as CAREER or SBIR-style research. ### Integrations Send the dataset to Google Sheets to maintain a living prospect list. Use Apify webhooks to trigger CRM enrichment after each scheduled monitoring run. Export JSON into a data warehouse for grant trend dashboards. Connect with automation-lab actors for website/contact enrichment after identifying institutions and PIs. ### API usage with Node.js ```js import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('automation-lab/nsf-research-awards-scraper').call({ keyword: 'quantum computing', maxItems: 100 }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items[0]); ```` ### API usage with Python ```python from apify_client import ApifyClient import os client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("automation-lab/nsf-research-awards-scraper").call(run_input={"keyword": "climate", "maxItems": 100}) items = client.dataset(run["defaultDatasetId"]).list_items().items print(items[0]) ``` ### API usage with cURL ```bash curl -X POST "https://api.apify.com/v2/acts/automation-lab/nsf-research-awards-scraper/runs?token=$APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{"keyword":"artificial intelligence","maxItems":100}' ``` ### Use with MCP Claude Desktop, Claude Code, Cursor, and VS Code can call this actor through Apify MCP with the actor tool URL `https://mcp.apify.com?tools=automation-lab/nsf-research-awards-scraper`. For Claude Code, add the remote MCP server with this command: ```bash claude mcp add --transport http apify-nsf-research-awards "https://mcp.apify.com?tools=automation-lab/nsf-research-awards-scraper" ``` For Claude Desktop, Cursor, or VS Code, add this JSON server configuration and keep your Apify token in the environment used by the app: ```json { "mcpServers": { "apify-nsf-research-awards": { "url": "https://mcp.apify.com?tools=automation-lab/nsf-research-awards-scraper", "headers": { "Authorization": "Bearer ${APIFY_TOKEN}" } } } } ``` Example prompt: "Find 100 recent NSF artificial intelligence awards in California and summarize the top funded institutions." Another prompt: "Export NSF CAREER awards from 2024 and identify principal investigators with university affiliations." ### Scheduling and monitoring Schedule weekly or monthly runs to monitor newly funded NSF awards by keyword or program. Keep maxItems aligned with the expected result volume for your search. Use date filters to avoid reprocessing the same historical awards. ### Data quality notes Research.gov field availability can vary by record. The actor normalizes missing fields defensively and keeps rawAward when requested. PI emails and phone fields are public API fields when present, but not guaranteed for every award. ### Legality and responsible use This actor uses publicly available Research.gov award data. Always follow applicable laws, Apify terms, and your organization's data governance policies. Avoid using contact information for spam or prohibited outreach. ### FAQ Q: Do I need an NSF API key? A: No, the public Research.gov Award API is accessible without a paid key for this workflow. Q: Why are some PI emails missing? A: The public API does not include every contact field for every award. Q: Why did a broad keyword stop at maxItems? A: maxItems is a user-set run limit; increase it for larger exports. Q: Can I filter by state? A: Yes, use awardeeStateCode or performanceStateCode. ### Related scrapers Explore other automation-lab actors for government data, grant intelligence, contact enrichment, and organization research workflows. Related actor links should use the automation-lab namespace after publication. ### Field reference - `awardId` โ€” normalized NSF award field for downstream analysis. - `title` โ€” normalized NSF award field for downstream analysis. - `abstractText` โ€” normalized NSF award field for downstream analysis. - `agency` โ€” normalized NSF award field for downstream analysis. - `awardeeName` โ€” normalized NSF award field for downstream analysis. - `awardeeCity` โ€” normalized NSF award field for downstream analysis. - `awardeeStateCode` โ€” normalized NSF award field for downstream analysis. - `awardeeCountryCode` โ€” normalized NSF award field for downstream analysis. - `awardeeUeiNumber` โ€” normalized NSF award field for downstream analysis. - `primaryPiName` โ€” normalized NSF award field for downstream analysis. - `principalInvestigators` โ€” normalized NSF award field for downstream analysis. - `piEmails` โ€” normalized NSF award field for downstream analysis. - `programOfficerName` โ€” normalized NSF award field for downstream analysis. - `programOfficerEmail` โ€” normalized NSF award field for downstream analysis. - `fundsObligatedAmount` โ€” normalized NSF award field for downstream analysis. - `estimatedTotalAmount` โ€” normalized NSF award field for downstream analysis. - `awardDate` โ€” normalized NSF award field for downstream analysis. - `startDate` โ€” normalized NSF award field for downstream analysis. - `expirationDate` โ€” normalized NSF award field for downstream analysis. - `activeAward` โ€” normalized NSF award field for downstream analysis. - `historicalAward` โ€” normalized NSF award field for downstream analysis. - `fundProgramName` โ€” normalized NSF award field for downstream analysis. - `primaryProgram` โ€” normalized NSF award field for downstream analysis. - `programElementCode` โ€” normalized NSF award field for downstream analysis. - `programReferenceCode` โ€” normalized NSF award field for downstream analysis. - `directorate` โ€” normalized NSF award field for downstream analysis. - `division` โ€” normalized NSF award field for downstream analysis. - `organizationName` โ€” normalized NSF award field for downstream analysis. - `performanceCity` โ€” normalized NSF award field for downstream analysis. - `performanceStateCode` โ€” normalized NSF award field for downstream analysis. - `performanceCountryCode` โ€” normalized NSF award field for downstream analysis. - `performanceLocation` โ€” normalized NSF award field for downstream analysis. - `sourceUrl` โ€” normalized NSF award field for downstream analysis. - `queryKeyword` โ€” normalized NSF award field for downstream analysis. - `scrapedAt` โ€” normalized NSF award field for downstream analysis. - `rawAward` โ€” normalized NSF award field for downstream analysis. ### Troubleshooting - If the API rejects a filter, remove the newest structured filter and rerun with keyword/date filters first. - If your dataset is empty, broaden the keyword or remove state/program constraints. - If you need exact API auditability, keep includeRaw enabled. ### Changelog - 0.1 โ€” Initial private build for NSF award search exports. # Actor input Schema ## `keyword` (type: `string`): Search term for NSF award titles, abstracts, investigators, and related award text. Leave empty to use only the structured filters. ## `dateStart` (type: `string`): Optional award-date lower bound in MM/DD/YYYY format supported by the Research.gov Award API. ## `dateEnd` (type: `string`): Optional award-date upper bound in MM/DD/YYYY format supported by the Research.gov Award API. ## `awardeeStateCode` (type: `string`): Two-letter U.S. state code for the recipient institution, for example CA, MA, TX, or NY. ## `performanceStateCode` (type: `string`): Two-letter U.S. state code for where the award work is performed. Leave empty to search all locations. ## `fundProgramName` (type: `string`): Optional NSF funding program text filter, for example Computer, Biology, CAREER, or SBIR. ## `agency` (type: `string`): Optional agency filter. NSF is the usual value for National Science Foundation awards. ## `programOfficerName` (type: `string`): Optional program officer name filter when you want awards associated with a specific NSF officer. ## `maxItems` (type: `integer`): Maximum number of NSF award records to save to the dataset. ## `includeRaw` (type: `boolean`): Keep the original Research.gov award object in rawAward for auditing and downstream mapping. ## Actor input object example ```json { "keyword": "artificial intelligence", "dateStart": "01/01/2024", "dateEnd": "12/31/2024", "awardeeStateCode": "CA", "agency": "NSF", "maxItems": 20, "includeRaw": true } ``` # Actor output Schema ## `overview` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "keyword": "artificial intelligence", "dateStart": "01/01/2024", "dateEnd": "12/31/2024", "awardeeStateCode": "CA", "agency": "NSF", "maxItems": 20, "includeRaw": true }; // Run the Actor and wait for it to finish const run = await client.actor("automation-lab/nsf-research-awards-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "keyword": "artificial intelligence", "dateStart": "01/01/2024", "dateEnd": "12/31/2024", "awardeeStateCode": "CA", "agency": "NSF", "maxItems": 20, "includeRaw": True, } # Run the Actor and wait for it to finish run = client.actor("automation-lab/nsf-research-awards-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("๐Ÿ’พ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # ๐Ÿ“š Want to learn more ๐Ÿ“–? Go to โ†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "keyword": "artificial intelligence", "dateStart": "01/01/2024", "dateEnd": "12/31/2024", "awardeeStateCode": "CA", "agency": "NSF", "maxItems": 20, "includeRaw": true }' | apify call automation-lab/nsf-research-awards-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=automation-lab/nsf-research-awards-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "NSF Research Awards Scraper", "description": "Extract NSF award and grant records from Research.gov with PI, institution, funding, dates, program, and raw award data.", "version": "0.1", "x-build-id": "s63VafkTmDnCUPsT4" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/automation-lab~nsf-research-awards-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-automation-lab-nsf-research-awards-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/automation-lab~nsf-research-awards-scraper/runs": { "post": { "operationId": "runs-sync-automation-lab-nsf-research-awards-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/automation-lab~nsf-research-awards-scraper/run-sync": { "post": { "operationId": "run-sync-automation-lab-nsf-research-awards-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "properties": { "keyword": { "title": "Keyword", "type": "string", "description": "Search term for NSF award titles, abstracts, investigators, and related award text. Leave empty to use only the structured filters." }, "dateStart": { "title": "Award date from", "type": "string", "description": "Optional award-date lower bound in MM/DD/YYYY format supported by the Research.gov Award API." }, "dateEnd": { "title": "Award date to", "type": "string", "description": "Optional award-date upper bound in MM/DD/YYYY format supported by the Research.gov Award API." }, "awardeeStateCode": { "title": "Awardee state code", "type": "string", "description": "Two-letter U.S. state code for the recipient institution, for example CA, MA, TX, or NY." }, "performanceStateCode": { "title": "Performance state code", "type": "string", "description": "Two-letter U.S. state code for where the award work is performed. Leave empty to search all locations." }, "fundProgramName": { "title": "Funding program name", "type": "string", "description": "Optional NSF funding program text filter, for example Computer, Biology, CAREER, or SBIR." }, "agency": { "title": "Agency", "type": "string", "description": "Optional agency filter. NSF is the usual value for National Science Foundation awards." }, "programOfficerName": { "title": "Program officer name", "type": "string", "description": "Optional program officer name filter when you want awards associated with a specific NSF officer." }, "maxItems": { "title": "Maximum awards", "minimum": 1, "maximum": 10000, "type": "integer", "description": "Maximum number of NSF award records to save to the dataset.", "default": 20 }, "includeRaw": { "title": "Include raw API record", "type": "boolean", "description": "Keep the original Research.gov award object in rawAward for auditing and downstream mapping.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```