# Government Bids + RFP Documents as RAG Chunks (`opendatafrontierlab/bids-with-documents-rag`) Actor

US public bids, RFPs and grant notices (E-Rate Form 470, Delaware, Grants.gov) with their attached PDF, DOCX and XLSX documents converted to Markdown and RAG chunks. Each chunk links to its notice, agency, deadline and page. OCR for scans.

- **URL**: https://apify.com/opendatafrontierlab/bids-with-documents-rag.md
- **Developed by:** [OpenDataFrontierLabs](https://apify.com/opendatafrontierlab) (community)
- **Categories:** AI, Lead generation, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Government Bids + RFP Documents as RAG Chunks

Get open US public bids, RFPs and grant notices **together with their attached solicitation documents**, converted to clean Markdown and heading-aware RAG chunks. Every chunk carries the notice ID, title, agency, dates, official notice link, document link and a citation, so a vector database or AI agent can answer "what does this RFP require?" and point to the source page.

**Tutorials & examples:** [Turn government PDFs into RAG-ready chunks with page citations](https://github.com/opendatafrontierlab/awesome-public-data-actors/blob/main/tutorials/pdf-rag-chunks.md) shows how to load chunks like these into a vector database and answer with citations (GitHub).

### Sources (official, public, no login)

| Source | What you get |
|---|---|
| **E-Rate Form 470** (USAC/FCC) | School & library technology RFPs nationwide, with the RFP documents applicants upload (PDF, DOCX, XLSX). One notice per Form 470 application. |
| **State of Delaware** (MyMarketplace) | Open state bids with their bid documents, specifications and pricing sheets. |
| **Grants.gov** | Federal funding opportunities with their NOFO attachments. |

Sources are processed in this order until **Max notices** is reached, so with the defaults (20 notices, 10 per source) Grants.gov is only reached if the first two sources have fewer than 10 notices with documents. Pick sources explicitly to control the mix.

Notices without downloadable documents are skipped and **not charged**. **States** filters E-Rate applicants; Delaware is included only when DE is in the list; Grants.gov opportunities are national and are not filtered by state.

Data: USAC Open Data (E-Rate), Delaware Open Data / Delaware MyMarketplace, Grants.gov. Documents remain the publishers' files; see `documentRights`.

### Output (one item per chunk)

Real item from a platform test run on 2026-10-29 (signed link shortened):

```json
{
  "recordType": "chunk",
  "id": "d44ba83daab5-0-6",
  "noticeId": "erate-form-470:270002130",
  "sourceKey": "erate-form-470",
  "sourceName": "E-Rate (USAC/FCC) — Form 470 school & library technology RFPs",
  "noticeTitle": "tusd_2027_470_c2_new_cabling",
  "noticeReference": "270002130",
  "agency": "Tucson Unified School District",
  "department": "School District",
  "noticeType": "Solicitation",
  "postedDate": "2026-10-28",
  "dueDate": null,
  "dueDateTime": null,
  "allowableContractDate": "2026-11-25",
  "locationState": "AZ",
  "noticeUrl": "https://publicdata.usac.org/EPC/Prd/Form470/270002130/143120/22836057-USAC_FCC_FORM_470_APPLICATION_270002130_CERTIFIED.pdf",
  "noticeDataLicense": "USAC Open Data (datahub.usac.org) - notice metadata",
  "documentRights": "Public procurement record published by Tucson Unified School District; copyright not determined - check the source before reuse.",
  "docId": "d44ba83daab5",
  "documentUrl": "https://publicdata.usac.org/EPC/Prd/Form470/270002130/143120/22835651-cabling_services_for_e_rate_funding_year_27_28_20261028_215021.pdf",
  "fileName": "22835651-cabling_services_for_e_rate_funding_year_27_28_20261028_215021.pdf",
  "documentLabel": "RFP document (Form 470 attachment)",
  "format": "pdf",
  "pageCount": 49,
  "sectionCount": null,
  "processedFrom": 1,
  "processedThrough": 49,
  "truncated": false,
  "continuation": false,
  "billedPages": 49,
  "ocrPages": 0,
  "sha256": "35945ade14e42cacea54b47e357067a29524ba2d431120e27048b890e4b307d9",
  "markdownUrl": "https://api.apify.com/v2/key-value-stores/ewbruSoBjA3KJkejW/records/doc-d44ba83daab5.md?signature=...",
  "text": "# **2. Documents Referenced**\n\nDOCUMENTS REFERENCED:\n\nYou may access a copy of the documents referenced within this proposal at the following web addresses:\n\nArizona Revised Statutes (A.R.S.) is available at: www.azleg.state.az.us/ArizonaRevisedStatutes.asp.\n\nThe Arizona School District Procurement Rules in the Arizona Administrative Code is available at:\n\n<u>www.azsos.gov/rules/arizona-administrative-code</u>\n\nI.R.S W-9 Form (Request for Taxpayer I.D. Number) is available at: www.irs.gov/pub/irs-pdf/fw9.pdf.\n\nCode of Federal Regulations is available at: http://www.ecfr.gov",
  "headingPath": [
    "2. Documents Referenced"
  ],
  "pageStart": 6,
  "pageEnd": 6,
  "chunkIndex": 6,
  "charCount": 580,
  "approxTokens": 145,
  "chunkCount": 92,
  "citation": "Tucson Unified School District Form 470 #270002130 — 22835651-cabling_services_for_e_rate_funding_year_27_28_20261028_215021.pdf, p. 6"
}
```

Field notes:

- `dueDate`: the bid/application deadline when the source publishes one (Delaware, Grants.gov; `YYYY-MM-DD`, `dueDateTime` when a time is given). E-Rate Form 470 has no bid due date; `allowableContractDate` is the earliest date the applicant may sign a contract (certification + 28 days). Check the RFP text for the actual response deadline.
- `truncated: true` means you did not get the whole document: it was longer than **Max pages per document**, or converting it hit **Per-document time limit** (heavy OCR scans). Pages finished before the limit are delivered and charged; `processedFrom`/`processedThrough` say which pages you got. With **Only new documents** + **Continue truncated documents**, the next run delivers the following pages (page fees only).
- `pageStart`/`pageEnd` are real PDF page numbers. They are `null` for DOCX/XLSX/HTML, which have no pages.
- `documentRights`: the attached documents are public procurement records published by the issuing agency (school districts, libraries, state and federal agencies); their copyright is **not determined** by this actor. Check the source before republishing. `noticeDataLicense` covers only the notice metadata feed.
- `markdownUrl`: the full document Markdown in the run's key-value store.

Choose **Documents** output for one item per document with the full Markdown and the full notice record. Each run's `RUN_SUMMARY` record lists counts, skipped documents with reasons and the charged events. It is updated after every document, so a run that times out or is aborted still has one (`status`: `in progress` or `finished`).

### Pricing (pay per event)

| Event | Price |
|---|---|
| Actor start (per GB of memory; default 2 GB) | $0.001 |
| Notice with at least one delivered document | $0.004 |
| Document delivered | $0.003 |
| Page delivered (PDF page with text; DOCX/XLSX/HTML: per 3,000 characters of content, table pipes not counted; large tables are split at row boundaries with the header row repeated) | $0.001 |
| OCR page (scanned PDF page where OCR returned text) | $0.004 **on top of** the page fee, so $0.005 per scanned page |

Not charged: notices without documents, blank pages, files that fail to download or convert, HTML error/login pages served instead of a file, oversized or unsupported files (ZIP, DOC, images, CAD), large-format scanned drawing sheets (not OCRed), and pages not reached before the per-document time limit. They are listed in `RUN_SUMMARY` → `skippedDocuments`.

**The notice fee is per run**: if a later run (e.g. with Only new documents) delivers an addendum for a notice you already received, that run charges the notice fee again ($0.004) plus the addendum's document and page fees.

Measured on the platform (2026-10-29, default 2 GB memory):
| Run | Delivered | Charged | Run time |
|---|---|---|---|
| Default input (20 notices, ≤10 documents and ≤100 pages each) | 20 notices, 56 documents, 1,197 pages (25 OCR), 2,714 chunks | **$1.55** | 43 min |
| One Delaware road bid (proposal, 79 plan sheets, cross sections, appendix) | 1 notice, 4 documents, 200 pages, 545 chunks | **$0.22** | 2.2 min |
| One Delaware roof bid (131-page ITB, many scanned pages) | 1 notice, 2 documents, 71 pages (23 OCR); ITB truncated at the 600 s time limit | **$0.18** | 10 min |

Large specifications dominate the cost: in the default run a notice cost $0.01-0.21 (median $0.06). Lower **Max pages per document** or **Max documents per notice** to cap it, and set a maximum charge per run. A run never charges more than your limit; it stops cleanly when the limit is reached.

### Tips

- Schedule daily with **Only new documents since last run**: documents are recognized by content (SHA-256), so a changed file at the same URL is delivered again and an identical re-upload is not. Each saved task keeps its own memory (see **State key**).
- If a run is restarted or migrated by the platform, it resumes and does not charge the same document twice (each document is delivered, marked done, then charged). In the rare case of a crash in the instant between delivering a document and marking it done, that document can appear twice in the dataset but is charged once.
- Contact data: phone numbers are redacted (all common US formats). Only official agency/public-entity emails are kept so vendors can reach the procurement office: official domains (.gov, .mil, .edu, .us, k12) and, for E-Rate, the applicant's own domain (e.g. a school district's .org). Consultant, private-firm and freemail addresses are redacted, even when listed as the Form 470 contact. Choose **Redact all** to remove every email.
- OCR text from scanned tables can misalign columns; check numbers against the original (`documentUrl`).
- Downloads are polite (delay per site) and respect robots.txt.
- The issuing agency's official notice always controls; check deadlines and addenda at `noticeUrl`.

### FAQ

**Where does the data come from?** Official public government portals and open-data APIs (USAC Open Data, data.delaware.gov and the Delaware bid document portal, the Grants.gov API). The actor fetches only documents that are publicly downloadable without a login, captcha or paywall. Reuse of the documents is your responsibility; see `documentRights`.

**Why are some notices missing?** Only notices that publish downloadable documents are returned. Need every notice without documents? Use the US Public Bids actor.

**Why was a document skipped?** See `RUN_SUMMARY.skippedDocuments`: e.g. a file that the portal no longer serves (it returns an HTML page instead), a password-protected PDF, a ZIP/DOC/CAD file, or a very large drawing set that ran out of time or memory. Skipped documents are not charged.

# Actor input Schema

## `sources` (type: `array`):

Official sources that publish downloadable solicitation documents. Notices without documents are skipped and never charged.

## `keyword` (type: `string`):

Optional full-text filter on the notice (e.g. 'firewall', 'roofing', 'broadband').

## `states` (type: `array`):

Optional two-letter state codes, e.g. TX, CA. Filters E-Rate applicants by state; Delaware is included only if DE is listed. Grants.gov opportunities are national and are not filtered by state.

## `daysBack` (type: `integer`):

Only notices posted within this many days.

## `postedSince` (type: `string`):

YYYY-MM-DD.

## `maxNotices` (type: `integer`):

Maximum notices with converted documents per run, across all sources.

## `maxNoticesPerSource` (type: `integer`):

Maximum notices with converted documents per source.

## `maxCandidatesPerSource` (type: `integer`):

How many notices to look at per source while searching for ones with documents (not charged).

## `maxDocumentsPerNotice` (type: `integer`):

Documents converted per notice (addenda, specs, pricing sheets).

## `maxPagesPerDocument` (type: `integer`):

Longer documents are truncated to this many pages and marked truncated: true (only delivered pages are charged). For DOCX/XLSX/HTML a page is 3,000 characters of content.

## `maxFileSizeMb` (type: `integer`):

Larger files are skipped and not charged.

## `outputMode` (type: `string`):

Full Markdown of every document is also saved to the run's key-value store (markdownUrl).

## `chunkSizeChars` (type: `integer`):

Target maximum characters per chunk (~4 characters = 1 token).

## `chunkOverlapChars` (type: `integer`):

Characters repeated between consecutive chunks when a long section is split.

## `ocrMode` (type: `string`):

OCR is used only for PDF pages without a text layer and charged only when it returns text.

## `onlyNewSinceLastRun` (type: `boolean`):

For scheduled runs: skip documents whose exact content (SHA-256) was already delivered by an earlier run with the same state key. A changed file at the same URL and an addendum on a known notice are delivered (and the addendum's notice fee is charged again).

## `continueTruncatedDocuments` (type: `boolean`):

With 'Only new documents', a document that was truncated by Max pages in an earlier run is continued from the next page (pages only; no notice or document fee).

## `stateKey` (type: `string`):

Name of the 'already delivered' memory used by Only new documents. Default: derived from sources, keyword and states, so different saved tasks don't hide each other's documents.

## `contactRedaction` (type: `string`):

Personal contact data is redacted from notice text and document Markdown.

## `documentTimeoutSecs` (type: `integer`):

Conversion time budget per document. Pages finished within it are delivered and charged; the rest is marked truncated (continue it in a later run with Continue truncated documents). Measured at 2 GB: a 131-page bid with many scanned pages delivered 70 pages (23 OCR) in 600 s.

## `downloadDelaySecs` (type: `number`):

Politeness delay per host. robots.txt rules are always respected.

## Actor input object example

```json
{
  "sources": [
    "erate-form-470",
    "delaware-state",
    "grants-gov"
  ],
  "daysBack": 14,
  "maxNotices": 20,
  "maxNoticesPerSource": 10,
  "maxCandidatesPerSource": 150,
  "maxDocumentsPerNotice": 10,
  "maxPagesPerDocument": 100,
  "maxFileSizeMb": 50,
  "outputMode": "chunks",
  "chunkSizeChars": 2000,
  "chunkOverlapChars": 200,
  "ocrMode": "auto",
  "onlyNewSinceLastRun": false,
  "continueTruncatedDocuments": false,
  "contactRedaction": "keep-official",
  "documentTimeoutSecs": 600,
  "downloadDelaySecs": 1
}
```

# Actor output Schema

## `chunks` (type: `string`):

No description

## `markdownFiles` (type: `string`):

No description

## `runSummary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("opendatafrontierlab/bids-with-documents-rag").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("opendatafrontierlab/bids-with-documents-rag").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call opendatafrontierlab/bids-with-documents-rag --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,opendatafrontierlab/bids-with-documents-rag"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/3hZJCkTeERZgb6WqW/builds/2gQEEwBNR6XUaMgEx/openapi.json