# Sourcewell Contract Scraper (`gallant_illustration/sourcewell-scrapper`) Actor

Extracts Sourcewell's full cooperative purchasing contract registry from the official XLSX feed and enriches every entry with categories from the HTML listing. Outputs contract numbers, suppliers, descriptions, solicitation titles, maturity dates and categories as a ready-to-use dataset.

- **URL**: https://apify.com/gallant\_illustration/sourcewell-scrapper.md
- **Developed by:** [Alexei Pannicov](https://apify.com/gallant_illustration) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 new items

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Sourcewell Contract Scraper

**Sourcewell contract search API alternative** — collects the public registry
of Sourcewell cooperative purchasing contracts from
[sourcewell-mn.gov](https://www.sourcewell-mn.gov/contract-search) and
delivers them as a structured Apify Dataset: ~1,060 contracts with suppliers,
descriptions, expiration dates and categories.

It covers the contract **registry**, not each contract's detail page — so no
contract documents, supplier contacts, or full products & services list (see
[Roadmap](#-roadmap)).

### What this Actor does

- **Official spreadsheet feed as the primary source** — faster and more
  reliable than paging through HTML: one request returns all ~1,060 contracts
  with accurate `supplier`/`description`/`solicitationTitle`/`maturityDate`
- **Category enrichment** — categories only exist on the HTML listing, so the
  Actor crawls it and merges categories into the feed rows by `contractNumber`
- **Category filter and limits** — export a single category or cap the
  dataset size for a quick test run
- **Apify platform advantages** — run on a schedule, trigger via API,
  connect to integrations, and monitor runs from the Apify Console

### Who is it for

- **Procurement and purchasing teams** checking what's already covered by
  Sourcewell cooperative contracts before running their own RFP
- **Suppliers and bid consultants** tracking competitor contracts and
  categories on Sourcewell
- **Sales and lead-gen teams** building a list of active Sourcewell suppliers
  by category
- **Market-intelligence and compliance tools** that need an up-to-date,
  machine-readable copy of the contract registry

### 🚀 How to use

1. Open the Actor's **Input** tab and optionally set `category` or
   `maxItems`.
2. Click **Start**. The Actor downloads the official contract-list
   spreadsheet feed and parses every contract row.
3. It then crawls the `/contract-search` listing page by page to collect
   categories for each `contractNumber`.
4. It merges both sources, optionally filters by `category`, and trims to
   `maxItems`.
5. The result is written to the run's **Dataset** — open the **Output** tab
   or fetch it via the API.

A full run takes a few minutes and returns the complete registry; no proxy,
login, or configuration is required.

### Input

All input fields are optional — see the **Input** tab for the full schema:

| Field | Type | Description |
|-------|------|-------------|
| `category` | string | One of the site's 14 top-level categories (dropdown); if omitted, contracts from all categories are returned |
| `maxItems` | integer | Maximum number of contracts in the resulting dataset — handy for a cheap test run |

### Output

You can download the dataset in various formats such as JSON, CSV, Excel, or
HTML via the **Output** tab or the API.

```json
[
  {
    "contractNumber": "030425-CEN",
    "supplier": "22nd Century Technologies",
    "description": "Public safety and law enforcement software solutions",
    "solicitationTitle": "Public Safety Software",
    "maturityDate": "2029-07-17",
    "categories": ["Office & Technology", "Public Safety"],
    "detailUrl": "https://www.sourcewell-mn.gov/cooperative-purchasing/030425-CEN"
  }
]
```

| Field | Type | Description |
|-------|------|-------------|
| `contractNumber` | string | Contract number (stable key, unique per contract) |
| `supplier` | string | Supplier name |
| `description` | string | Short contract description |
| `solicitationTitle` | string | Solicitation/RFP title |
| `maturityDate` | string | Contract expiration date (`YYYY-MM-DD`) |
| `categories` | string\[] | Contract categories — zero or more of the 14 top-level categories |
| `detailUrl` | string | Link to the contract's page on the Sourcewell site |

### How it works

The Actor combines two public sources on the official Sourcewell site:

1. **The official contract-list spreadsheet feed** — one request returns every
   contract with `contractNumber`, `supplier`, `description`,
   `solicitationTitle` and `maturityDate`. This feed has no categories.
2. **The `/contract-search` HTML listing** — crawled page by page and used
   only to collect `categories` per `contractNumber`.

Both sources are merged by `contractNumber`, optionally filtered by
`category`, trimmed to `maxItems`, and written to the dataset.

### Pricing

This Actor uses **Pay Per Event** pricing, with no charge for compute time
itself. Each contract written to the dataset is charged one `new-item`
event, so a run costs exactly as many events as the number of contracts it
delivers. See the **Pricing** tab on this Actor's page for the exact price
per event.

If you cap a run with Apify's **maximum cost per run** setting, the Actor
stops delivering once that budget is reached instead of writing contracts it
cannot charge for. The dataset then holds exactly what you paid for.

### Limitations & notes

- A few contracts have an empty `categories` array: they are present in the
  contract feed but not in the HTML listing the categories come from
  (1,056 of 1,063 contracts had categories in testing).
- `maxItems` trims the final list **after** both sources are merged, so it
  caps what you are charged for but doesn't make the run itself much
  faster.
- Contract documents, supplier contacts, supplier websites and the full
  products & services list live behind each contract's detail page and are
  not collected by this Actor yet — see the Roadmap below.

### 🗺 Roadmap

Planned next, in order:

1. **Contract detail crawling** — an optional, higher-priced mode that opens
   each contract's page to extract contract documents, supplier contacts and
   website, and the full products & services list.
2. **Wider source coverage** — evaluating whether to also include the
   A–Z supplier listing and the construction contractors (IDIQ) registry,
   which sit outside the main contract search.

Need one of these sooner, or a field that isn't in the output yet? Open an
issue on the Actor's **Issues** tab — requests with a concrete use case get
prioritized.

### Legality

Our Actors are ethical and do not extract any private user data, such as
email addresses, gender, or location. They only extract what the user has
chosen to share publicly. We therefore believe that our Actors, when used
for ethical purposes by Apify users, are safe. However, you should be aware
that your results could contain personal data. Personal data is protected
by the GDPR in the European Union and by other regulations around the
world. You should not scrape personal data unless you have a legitimate
reason to do so. If you're unsure whether your reason is legitimate,
consult your lawyers.

### ❓ FAQ

**Does the dataset include every Sourcewell contract, or only the ones
matching my `category` filter?**
The Actor always downloads the full contract feed first; `category` only
filters the final output, so switching categories between runs never requires
a different crawl of the contract list itself.

**Where do `categories` come from?**
From the `/contract-search` HTML listing, matched to feed rows by
`contractNumber` — the spreadsheet feed itself has no categories. A contract
that is in the feed but missing from the HTML listing comes back with an
empty `categories` array.

**How often is the underlying data updated?**
Sourcewell refreshes the contract feed on its own schedule, typically daily.
The Actor always fetches the current version on each run.

**Can I get supplier contacts, contract documents, or the full products &
services list?**
Not yet — that data lives on each contract's detail page and is on the
roadmap above.

**Can I run this on a schedule?**
Yes. Use Apify's **Schedules** to run it daily or weekly. Pair it with
`maxItems` or the maximum cost per run setting to keep recurring runs
predictable.

### 🛟 Support & feedback

Bugs and feature requests — via the **Issues** tab on this Actor's page.
For programmatic access, see the **API** tab.

# Actor input Schema

## `category` (type: `string`):

Return only contracts in this category. Leave empty to include all categories.

## `maxItems` (type: `integer`):

Maximum number of contracts written to the dataset. Applied after merging, so it does not make the run faster. Leave empty to return all contracts.

## Actor input object example

```json
{}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("gallant_illustration/sourcewell-scrapper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("gallant_illustration/sourcewell-scrapper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call gallant_illustration/sourcewell-scrapper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,gallant_illustration/sourcewell-scrapper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/z1CN3BQnyBAfGccGK/builds/iyPulFIfP9sMrhEK6/openapi.json