Conda Repodata Extractor avatar

Conda Repodata Extractor

Pricing

from $0.35 / 1,000 packages

Go to Apify Store
Conda Repodata Extractor

Conda Repodata Extractor

Point at any Conda channel's repodata.json and get one structured row per package: name, version, build, subdir, license, md5/sha256, size, timestamp, depends and constrains.

Pricing

from $0.35 / 1,000 packages

Rating

0.0

(0)

Developer

Datamule

Datamule

Maintained by Community

Actor stats

0

Bookmarked

1

Total users

0

Monthly active users

a month ago

Last modified

Categories

Share

Point at any Conda channel's repodata.json and get one clean, structured row per package.

Every Conda channel — conda-forge, bioconda, PyTorch, NVIDIA, the Anaconda pkgs/main defaults, and the tens of thousands of self‑hosted, org and anaconda.org channels — publishes the identical repodata.json index for each platform subdir. This actor speaks that one standard, so a single actor spans every Conda channel in existence — no per‑channel scraper, no host enumeration.

Give it a channel and a subdir; get back a flat, fully‑typed dataset row for every package build, ready for a spreadsheet, a database, a dependency‑graph, a license audit or an SBOM.

Quick start (3 rows, ≤ $0.0015)

  1. Click Try for free.
  2. The input form opens pre-filled with this example (package-build rows for numpy on Apple Silicon pkgs/main, capped at 3). On a blank form, paste it in the JSON tab:
    { "channelUrl": "https://repo.anaconda.com/pkgs/main/osx-arm64/repodata.json", "packageFilter": "numpy", "maxRecords": 3 }
  3. Click Start. The run downloads the channel index, then stops after 3 rows.
  4. Open the Output tab and export JSON, CSV or Excel.

Always set maxRecords on a first run. Big channels are large: this pkgs/main/osx-arm64 index held 105,150 package records on 2026-09-25 — an uncapped run would emit (and bill) every one of them.

Output preview (real rows, local run 2026-09-25, quick-start input above)

filenamenameversionbuildlicensesizetimestamp
msgpack-numpy-0.4.8-py310hd096484_0.condamsgpack-numpy0.4.8py310hd096484_0BSD-3-Clause199232025-08-07T10:35:10.973000Z
msgpack-numpy-0.4.8-py310hd096484_0.tar.bz2msgpack-numpy0.4.8py310hd096484_0BSD-3-Clause196932025-08-07T10:35:10.973000Z
msgpack-numpy-0.4.8-py311hd096484_0.condamsgpack-numpy0.4.8py311hd096484_0BSD-3-Clause217722025-08-07T10:34:35.525000Z

Each row also carries depends, constrains, md5/sha256, _raw and _packageCount (105150 here). packageFilter is a substring match, so numpy also matches msgpack-numpy; rows come in sorted filename order. Channel contents change over time, so your rows may differ.

What it does

  • Any channel, one input. Pass a channel base (https://conda.anaconda.org/pytorch) + a subdir, or paste a full .../<subdir>/repodata.json URL directly. A compressed .json.bz2 / .json.zst URL is normalized to the plain .json.
  • Both package maps, merged. conda ships .tar.bz2 builds under packages and the newer .conda builds under packages.conda. This actor merges both into one globally‑sorted stream — nothing is missed, nothing is double‑counted.
  • Memory‑flat streaming. repodata.json for a big subdir (e.g. conda-forge/linux-64) is hundreds of megabytes. The feed is streamed to disk and parsed incrementally, so RAM stays flat regardless of channel size, and maxRecords stops early.
  • Deterministic under a cap. Records are emitted in sorted filename order, so a capped run returns the same first N records for a given index snapshot. Live channels publish new builds, so a later run against the same URL can differ.

Input

FieldRequiredDescription
channelUrl✅A conda channel base (https://conda.anaconda.org/pytorch) or a full repodata.json URL.
subdirPlatform subdir used to complete a channel base. Default noarch. One of noarch, linux-64, linux-aarch64, linux-ppc64le, osx-64, osx-arm64, win-64, win-arm64. Ignored when channelUrl is already a full repodata URL.
packageFilterCase‑insensitive substring matched against each package name; non‑matching records are skipped.
maxRecordsGlobal cap on emitted rows (each row = one package record = one billable event).
bearerBearer token for a private/gated channel. Never logged.
extraHeadersExtra HTTP headers as a JSON object for gated channels. Never logged.

Example inputs

{ "channelUrl": "https://conda.anaconda.org/pytorch", "subdir": "noarch" }
{ "channelUrl": "https://conda.anaconda.org/nvidia", "subdir": "linux-64", "packageFilter": "cuda", "maxRecords": 500 }
{ "channelUrl": "https://repo.anaconda.com/pkgs/main/osx-arm64/repodata.json", "maxRecords": 100 }

Output

One row per package record (from both the .tar.bz2 and .conda maps):

ColumnDescription
filenameThe package filename (the repodata map key), e.g. botorch-0.1.0-0.tar.bz2.
namePackage name.
versionPackage version.
build / buildNumberBuild string and build number.
subdirPlatform subdir.
licenseDeclared license (SPDX‑ish string, when present).
md5 / sha256Artifact checksums.
sizeArtifact size in bytes.
timestampBuild timestamp, normalized to ISO‑8601 UTC.
dependsArray of runtime dependency specs ("pytorch >=1.1", …).
constrainsArray of optional‑constraint specs.
track_featuresFeature flags (space‑delimited string), when present.
_rawThe complete original record — nothing is ever dropped (license_family, noarch, platform, …).
_source / _subdir / _packageCountThe exact repodata URL, the subdir, and the total record count.

Every field is nullable and read by key presence — a channel that omits license / depends / constrains / track_features simply yields null for that column, never an error.

Notes & scope

  • Public metadata only. This reads a channel's published package index — it does not download the packages themselves.
  • GTFS‑style honesty. An empty subdir, or a packageFilter that excludes everything, returns 0 records and a clean success. A URL that isn't conda repodata (an HTML error page, a 404 body, JSON without a packages key) fails fast with a clear message.
  • Find channels on anaconda.org. Common bases: https://conda.anaconda.org/<channel> and the Anaconda defaults at https://repo.anaconda.com/pkgs/main.

Pricing

Pay‑per‑event: $0.0005 per package record returned on the free tier, dropping to $0.00025 on the highest volume tier. A capped run charges only for what it emits; 0 rows = no record charge.

Your budget is a hard limit. If you set a maximum cost per run, the Actor never returns more package rows than that budget pays for, and every row it returns is billed together with the row itself. When the budget doesn't cover even one package, the Actor stops before downloading the repodata file. If billing fails, or the source fails partway through, the run ends as failed instead of looking successful, and any rows already delivered stay in your dataset. If the platform restarts the run, it continues where it left off and doesn't re-bill rows you already have.

RunRowsRecord cost (free tier)
Quick start above3$0.0015
maxRecords: 100100$0.05
maxRecords: 500 (NVIDIA cuda example)≤ 500≤ $0.25
Uncapped pkgs/main/osx-arm64 (105,150 records on 2026-09-25)105,150≈ $52.58

Use packageFilter + maxRecords to pay only for the packages you need.