DUB D Language Packages Scraper

DUB D Language Packages Scraper

Scrape D language package metadata from the DUB registry by keyword, up to a million per run. Every package comes with its name, description, version, license, dependencies, and download stats. No login or API key. Export to CSV, JSON, Excel, or XML.

The DUB registry's web interface is built for browsing one package at a time, not for bulk analysis. This reads the public search index directly, so you can pull every package matching a keyword into a single dataset. Filter by name, description, or version, and collect up to a million records in one run.

Who uses it	What they scrape DUB Registry for
D language maintainers	Audit which libraries depend on a package they own.
Developer tool vendors	Build a catalog of available D libraries for an IDE plugin.
Security researchers	Map the dependency graph of the D ecosystem for vulnerability scanning.
Data analysts	Track growth trends in the D package ecosystem over time.

What it does

This Actor searches the DUB registry for D language packages by keyword and returns each matching package as a flat row.

• 🔍 Keyword search: feed it a term like 'vibe', 'http', or 'json' and get every package whose name or description matches.
• 📦 Full metadata: each row includes the package name, latest version, description, license, homepage, and download count.
• 🔗 Dependency tree: every package row lists its direct dependencies with version ranges, so you can rebuild the graph.
• 📊 Bulk export: collect up to a million packages per run and export the flat dataset to CSV, JSON, Excel, or XML.

Results export to CSV, JSON, Excel, or XML, or straight from the API.

What you can do with DUB Registry data

📈 Track D ecosystem growth.

A data analyst runs the Actor weekly with a broad keyword to count new packages and measure adoption trends.

🔗 Map dependency chains.

A maintainer searches for their own package name to see every reverse dependency and plan breaking changes.

🛡️ Audit license compliance.

A legal reviewer scrapes all packages in a category to check license fields before a product release.

🧪 Find libraries for a task.

A D developer searches for 'http' or 'serialization' to compare available libraries by version and download count.

Why choose this scraper

	What you get
No API key	Reads the public DUB search endpoint directly, no registration or token.
Fixed schema	Every package returns the same fields: name, version, description, license, dependencies, and stats.
High throughput	Collect up to a million packages in a single run without pagination logic.
Flat output	One row per package, ready for spreadsheets, databases, or downstream scripts.

How it compares

No other Store actor targets DUB Registry the same way, so the honest comparison is with the alternatives teams actually weigh.

	DUB D Language Packages Scraper	Build it in-house	By hand
Setup	Run it now, zero config	Days of engineering	None, but hours per pull
When DUB Registry changes	Maintained for you	You fix it	You re-learn the page
Proxies, retries, anti-bot	Built in	Your problem	Browser only
Output	Fixed JSON schema, CSV/Excel export	Whatever you build	Copy-paste
Cost	Pay per result	Engineering time	Analyst hours

Configure the run

Drive the Actor from a single keyword, and the search runs against package names and descriptions so only matches reach your dataset. The Input tab lists every parameter.

A first run with the defaults:

{
 "searchQuery": "vibe",
 "maxItems": 10
}

A larger pull:

{
 "searchQuery": "vibe",
 "maxItems": 200
}

Pricing

Pay-per-result: $0.021 per result collected. You pay only for the results written to your dataset.

Results collected	Approximate cost
100 results	$2.10
1,000 results	$21.00
10,000 results	$210.00

New Apify accounts start with $5 in free credit.

Free users

Free-plan runs return up to 10 results as a preview. Upgrade your Apify plan to collect up to 1,000,000 results per run.

Run it

1. Create a free Apify account with $5 in credit.
2. Open the DUB D Language Packages Scraper.
3. Set your inputs and any filters, then click Start.
4. Export the results as CSV, Excel, JSON, or XML from the Dataset tab.

Run it programmatically through the Apify API (run-sync-get-dataset-items) or the ApifyClient for JavaScript and Python.

Use with AI agents (MCP)

Give an AI agent live access to DUB Registry through the Model Context Protocol. Add the Actor to Claude, Cursor, or any MCP client:

$claude mcp add --transport http apify "https://mcp.apify.com?tools=parseforge/dub-dlang-packages-scraper"

Then prompt it in plain language to run the scraper and read back the results.

Troubleshooting

Why am I getting no results?

Check your 'searchQuery' spelling. The DUB search is literal; try a shorter or more common keyword. Also verify that the registry is reachable from your network.

Why are some packages missing fields?

Some packages on DUB do not declare a license, homepage, or description. The Actor returns empty strings for missing fields rather than omitting them.

The run stopped before reaching my maxItems.

The search returned fewer total results than your limit. Try a broader keyword or reduce 'maxItems' to match the available count.

Why are download counts different from the website?

Download counts are cached by the registry and may lag behind real-time. The Actor returns whatever value the public search endpoint provides at that moment.

I get a timeout or the run is slow.

The DUB registry may rate-limit aggressive requests. Try reducing 'maxItems' or running during off-peak hours. The Actor includes automatic retries for transient errors.

FAQ

Question	Answer
What is the DUB registry?	DUB is the official package manager and registry for the D programming language, similar to npm for JavaScript or PyPI for Python. It hosts thousands of libraries and applications.
Do I need a DUB account or API key?	No. This Actor reads the public search interface, so no login, token, or registration is required.
What data does each package row contain?	Each row includes the package name, latest version, description, license, homepage URL, download count, and a list of direct dependencies with version ranges.
How do I search for a specific package?	Enter the package name or a keyword in the 'searchQuery' field. The search matches against both package names and descriptions.
Can I scrape all packages in the registry?	Use a broad keyword like 'd' or a common term, and set 'maxItems' high enough to cover the registry. The Actor will collect every match up to your limit.
What output formats are supported?	You can export the dataset as CSV, JSON, Excel, or XML from the Apify platform.
How many packages can I collect in one run?	You can set 'maxItems' up to 1,000,000. The actual number collected depends on how many packages match your keyword.
Does this Actor handle pagination?	Yes. It automatically follows search result pages until it reaches your 'maxItems' limit or exhausts the results.
Can I filter by license or dependency?	The Actor collects all matching packages. You can filter by license or dependency after export using your own scripts or spreadsheet tools.
Is this Actor affiliated with the D Language Foundation?	No. This is an independent tool that reads publicly available data from the DUB registry website.

Related actors

Browse the full ParseForge collection for more scrapers.

🆘 Need help? Email parseforge@protonmail.com with your run ID, your input, and what you expected.

⚠️ Disclaimer. This Actor is unofficial and is not affiliated with, endorsed by, or sponsored by D Language Foundation. It collects only publicly available data. You are responsible for using the collected data in compliance with the source's terms of service and applicable data-protection laws, including GDPR, CCPA, and PIPL. Do not use it to collect personal data unlawfully.