Chef Supermarket Scraper

Scrape Chef Supermarket — the public community registry for Chef cookbooks at supermarket.chef.io. Search the catalog by keyword, browse it by category, or fetch named cookbooks directly, and get the maintainer, category, latest version, version count, total downloads, average rating, deprecation status, and links to source and issue trackers. HTTP-only via the public Supermarket v1 API. No auth, no proxy.

What this actor does

• Two modes: searchCookbooks (keyword search or full-catalog browse) and getByName (exact cookbook lookups)
• Detail enrichment: every search hit is expanded into full cookbook detail, not just the summary row
• Category filter: narrow results to a single Supermarket category
• Adoption signals: download totals, version counts, average rating, and deprecation replacements in one record
• Empty fields are omitted

Output per cookbook

• cookbookName — Supermarket cookbook slug (e.g. nginx)
• description — cookbook summary
• maintainer — Supermarket account that owns the cookbook
• category — Supermarket category (e.g. Web Servers, Databases)
• latestVersion — most recent published version string (e.g. 12.2.5)
• versionCount — number of published versions
• downloadCount — lifetime download total
• averageRating — community rating, when the cookbook has one
• deprecated — true/false
• deprecatedReplacedBy — successor cookbook, when a deprecated cookbook names one
• sourceUrl — cookbook page on Chef Supermarket
• externalUrl — upstream project / repository URL
• issuesUrl — issue tracker URL
• recordType: "cookbook", scrapedAt

Input

Field	Type	Default	Description
mode	string	searchCookbooks	searchCookbooks / getByName
searchQuery	string	nginx	Keyword to search for (mode=searchCookbooks). Leave empty to browse the whole catalog
cookbookNames	array	["nginx", "mysql", "apache2"]	Exact cookbook names to fetch (mode=getByName)
category	string	–	Restrict results to one category (mode=searchCookbooks)
maxItems	int	50	Hard cap (1–500)

Example: keyword search

{
  "mode": "searchCookbooks",
  "searchQuery": "nginx",
  "maxItems": 25
}

Example: browse one category

{
  "mode": "searchCookbooks",
  "searchQuery": "",
  "category": "Databases",
  "maxItems": 100
}

Example: fetch a known dependency set

{
  "mode": "getByName",
  "cookbookNames": ["nginx", "mysql", "apache2", "postgresql"]
}

Example: wide catalog sweep

{
  "mode": "searchCookbooks",
  "searchQuery": "",
  "maxItems": 500
}

Use cases

• Infrastructure teams — audit which community cookbooks a Chef estate depends on and whether any are deprecated
• Dependency hygiene — flag pinned cookbooks that have newer versions or a named replacement
• Platform engineering — compare competing cookbooks for the same service by downloads and rating
• Ecosystem research — measure category-level activity across the Supermarket registry
• Migration planning — inventory cookbooks by maintainer before moving off Chef
• Internal catalogs — mirror Supermarket metadata into an internal service catalog

FAQ

What is Chef Supermarket? The community registry for Chef Infra cookbooks — the equivalent of npm or PyPI for Chef configuration code. See supermarket.chef.io.

Is this affiliated with Chef or Progress Software? No. This is a third-party actor built on Supermarket's public v1 API.

Do I need a Chef account or API key? No. The endpoints used are open and unauthenticated.

What's the difference between searchCookbooks and getByName? searchCookbooks runs a relevance search (or browses the full catalog when the query is blank) and returns many cookbooks. getByName takes exact slugs and returns exactly those — faster and deterministic when you already know what you want.

Why does category only apply when browsing? Supermarket's category filter is applied to the listing, so results are narrowed to your chosen category after each cookbook's detail is resolved. Combining a keyword search with a category can therefore return fewer rows than maxItems.

How do I find valid category names? Use the category labels shown on Supermarket itself — for example Web Servers, Databases, Monitoring & Trending, Other. Matching is case-insensitive.

What does deprecated mean here? The maintainer has marked the cookbook as no longer supported. When they nominate a successor, it appears as deprecatedReplacedBy.

Why do some cookbooks have no averageRating? Ratings are optional and many cookbooks have never been rated. Rather than emitting a misleading zero, the field is dropped.

How fresh is the data? Live — every run queries Supermarket directly, so download counts and version numbers reflect the moment of the run.