USDA BioPreferred Program Catalog Scraper

Scrapes the official USDA BioPreferred Program product catalog — the federal government's authoritative list of biobased products, including which products carry the official "USDA Certified Biobased" label and which fall into a mandatory federal purchasing product category. Covers all 276 sub-categories (26 top-level groups) in the live catalog.

Use cases

1. Federal contractors / procurement teams checking which products in a category qualify for mandatory federal biobased purchasing requirements before sourcing decisions.
2. Sustainability / market researchers tracking the biobased-products market: which companies and product categories are represented, and at what certification rate.
3. Competitive intelligence for biobased brands — see which competitors are listed in a given category and whether they hold the official USDA Certified Biobased label.

Input

Field	Type	Default	Description
categories	array of strings	[] (all)	Sub-category names to scrape, e.g. ["Baby Wipes", "Diapers"]. Leave empty to scrape all 276 sub-categories.
maxItems	integer	500	Cap on total products scraped across the run.
delaySeconds	number	1.0	Polite delay before each category request.
concurrency	integer	5	How many categories are fetched at once.

Output fields

Field	Type	Description
productName	string	Product name as listed in the catalog
productDescription	string	Manufacturer-provided description
companyName	string	Manufacturer/company name
usdaCertifiedBiobased	boolean	Carries the official "USDA Certified Biobased" label
mandatoryFederalPurchasing	boolean	In a mandatory federal purchasing product category
topCategory	string	Top-level category group (e.g. "Baby And Kids")
subCategory	string	Specific sub-category scraped (e.g. "Diapers")

Not available from this data source (checked directly against the live catalog, not assumed): there is no company website, no per-product details URL, and no structured biobased-content percentage field in the catalog's product table — only a plain product name/description and company name per row, plus the two certification badges above.

Example output

{
  "productName": "SKILCRAFT Enviro Care Glass Cleaner",
  "productDescription": "GREEN SEAL APPROVED! A concentrated solution that leaves windows, mirrors, and all other polished surfaces squeaky clean and free of streaks and hard water deposits. Highly effective, even on heavily soiled surfaces.",
  "companyName": "Lighthouse for the Blind - St. Louis",
  "usdaCertifiedBiobased": true,
  "mandatoryFederalPurchasing": true,
  "topCategory": "Custodial Services",
  "subCategory": "Cleaning Supplies"
}

How it works

The live catalog is a JSF/PrimeFaces single-page app with no per-category or per-product URLs — browsing happens through AJAX postbacks. This Actor discovers the full category hierarchy from the catalog's initial page load, then replicates the same AJAX postback per sub-category to pull that category's complete product table in one request (no pagination exists on the target, even for a 500+ item category). Rate-limited with a configurable delay and bounded concurrency.

Pricing

Recommended: $5–$8 per 1,000 results.

Results	Estimated cost
100	$0.50–$0.80
500	$2.50–$4.00
1,000	$5.00–$8.00
5,000	$25.00–$40.00