# Hex Pm Elixir Packages Scraper (`bigibyte/hex-pm-elixir-packages-scraper`) Actor

Search Elixir packages on Hex.pm or load specific libraries to extract versions, licenses, download stats, and project links. Easily download your data as a JSON, CSV, or Excel dataset.

- **URL**: https://apify.com/bigibyte/hex-pm-elixir-packages-scraper.md
- **Developed by:** [BigiByte](https://apify.com/bigibyte) (community)
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Hex.pm Elixir Packages Scraper: Elixir Packages Search and CSV Export

Search Elixir packages with ease. This actor allows you to quickly gather package names, version details, download stats, licensing info, and project links. Once extracted, your dataset is ready to download in CSV, JSON, or Excel format.

You can search by keyword to walk through package results or specify exact package names to load detailed metadata, including maintainers and owners when listed. If a package isn't found, the run gracefully flags it and keeps going without interrupting your workflow.

### Use cases

- **Dependency research**: Find Elixir libraries matching your query, rank them by popularity, and keep a clean list for your team.
- **Version tracking**: Stay on top of `latest_stable_version`, publish dates, and recent updates for packages in your tech stack.
- **License review**: Inspect `licenses` and official project links before adding new dependencies to your software.
- **Download ranking**: Compare total vs. recent downloads to pick active, well-maintained libraries.
- **Scheduled monitoring**: Run periodic searches to track package adoption and changes over time.

### Input

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `mode` | string | `search` | `search` looks up package lists. `package` loads specific named packages. |
| `query` | string | `phoenix` | Search term used when mode is set to `search`. |
| `sort` | string | `downloads` | Choose how results are sorted: `downloads`, `recent_downloads`, `inserted_at`, `updated_at`, or `name`. |
| `packages` | array | `phoenix` | List of specific package names to fetch when mode is set to `package`. |
| `maxItems` | integer | `100` | Maximum number of packages to fetch per run (up to 1,000). |
| `requestDelaySecs` | number | `1` | Pause between requests to maintain steady request rates. |
| `timeoutSecs` | integer | `300` | Maximum runtime limit in seconds. |
| `requestTimeoutSecs` | integer | `30` | Time limit per network request in seconds. |
| `userAgent` | string | `BigiByteActorResearch/1.0 (contact: research@bigibyte.com)` | Custom User-Agent header sent with requests. |
| `proxyConfiguration` | object | `{"useApifyProxy": false}` | Optional proxy configuration if needed. |

#### Example input

```json
{
    "mode": "search",
    "query": "phoenix",
    "sort": "downloads",
    "maxItems": 100,
    "requestDelaySecs": 1,
    "timeoutSecs": 300,
    "requestTimeoutSecs": 30,
    "userAgent": "BigiByteActorResearch/1.0 (contact: research@bigibyte.com)",
    "proxyConfiguration": { "useApifyProxy": false }
}

```

For direct package lookup:

```json
{
    "mode": "package",
    "packages": ["phoenix"],
    "maxItems": 100,
    "proxyConfiguration": { "useApifyProxy": false }
}

```

### Output Data Structure

Below is an example of a successfully extracted package record:

```json
{
    "name": "phoenix",
    "packageDescription": "Peace of mind from prototype to production",
    "licenses": ["MIT"],
    "links": {
        "Changelog": "[https://phoenix.hexdocs.pm/changelog.html](https://phoenix.hexdocs.pm/changelog.html)",
        "GitHub": "[https://github.com/phoenixframework/phoenix](https://github.com/phoenixframework/phoenix)"
    },
    "latest_stable_version": "1.8.15",
    "downloads_all": 157172271,
    "downloads_recent": 5314428,
    "inserted_at": "2014-04-21T22:38:32.000000Z",
    "updated_at": "2026-09-25T13:21:10.641246Z",
    "html_url": "[https://hex.pm/packages/phoenix](https://hex.pm/packages/phoenix)",
    "owners": [
        {"username": "chrismccord", "email": "chris@chrismccord.com"}
    ],
    "scraped_at": "2026-10-12T00:00:00+00:00"
}

```

| Field | Type | Description |
| --- | --- | --- |
| `name` | string | Unique package name. |
| `packageDescription` | string | Short summary describing the package. |
| `licenses` | array | Included software licenses. |
| `links` | object | Important project links (e.g., GitHub, documentation, changelog). |
| `latest_stable_version` | string | Current stable version release. |
| `downloads_all` | integer | Total lifetime download count. |
| `downloads_recent` | integer | Download count over recent periods. |
| `inserted_at` | string | Initial publication timestamp. |
| `updated_at` | string | Most recent update timestamp. |
| `html_url` | string | Web link to the package on Hex.pm. |
| `owners` | array | List of maintainer usernames and contact emails (when available). |
| `scraped_at` | string | UTC timestamp when the data was extracted. |

If a specific package is missing or encounters an error, a single error item is recorded:

```json
{
    "error": true,
    "status": 404,
    "message": "Page not found",
    "scraped_at": "2026-10-12T00:00:00+00:00"
}

```

### How It Works

1. **Search Mode**: Iterates through search results page by page based on your query and sorting criteria until `maxItems` or timeout limits are reached.
2. **Package Mode**: Directly retrieves information for each package name supplied in your input array.
3. **Resilience**: Temporary server errors automatically trigger retries. Unresolvable errors (like 404s) generate clear error records without interrupting the rest of the scrape.
4. **Pacing**: Built-in delays ensure polite pacing so runs finish smoothly without hitting request caps.
5. **Batching**: Results save periodically in batches during execution.

### Pricing & Usage Limits

This actor does not charge additional platform markup fees you only cover standard compute usage on Apify.

- **Rate Limits**: Pacing is capped at ~100 requests per minute with built-in pauses to ensure steady execution.
- **Item Limits**: Supports up to 1,000 package records per run.
- **Attribution**: If you redistribute or publish exported data, please credit Hex.pm as the original source.

### FAQ

**How do I export Elixir package data to CSV or Excel?**

Once the run finishes, navigate to the dataset tab in Apify and click **Export**. You can download your data as CSV, JSON, Excel, or XML.

**What happens if a package name doesn't exist?**

The actor records a single line with `error: true` and the status code, then immediately moves on to the next item in your list.

**Why is there a built-in delay between requests?**

Proper pacing prevents your run from being throttled and ensures long search requests complete reliably.

**How should I handle owner email addresses?**

Package owner contact details are public maintainer records. If you plan to store or process this information, ensure your usage complies with applicable data privacy regulations like GDPR or CCPA.

### Related Tools & Custom Development

If you plan to map, analyze, or enrich Elixir package data, combining this export with CSV cleanup workflows or custom data pipelines is a natural next step. BigiByte builds custom scrapers, automated workflows, and data pipelines for package registries, open-data portals, and web platforms whenever you need them.

Need a specialized dataset, scheduled monitoring, or a custom scraper built to your specifications? Explore custom solutions with BigiByte: <https://bigibyte.com>

***

#### Trademark Notice

Hex.pm, Elixir, and associated names, logos, or trademarks belong to their respective owners. Their mention here is purely for identification purposes and does not imply affiliation, sponsorship, or endorsement.

#### Disclaimer

This actor is an independent scraper built by BigiByte and is not officially affiliated with or endorsed by Hex.pm or its maintainers. Users are responsible for adhering to source terms of use, licensing, and attribution policies.

#### Is web scraping legal?

This actor collects publicly accessible data without bypassing access controls. Extracting public data is widely recognized as legal, but how you handle and use the data remains important:

- **Personal Data**: Maintainer contact information should be handled responsibly in compliance with relevant privacy regulations such as GDPR or CCPA.
- **Copyright & Content**: Descriptions and documentation may be subject to copyright. Avoid re-publishing full text content without permission.
- **Terms & Attribution**: Respect rate limits and provide proper attribution when re-using data publicly.

For more details, check out Apify's helpful guide: [Is web scraping legal?](https://blog.apify.com/is-web-scraping-legal/)

# Actor input Schema

## `mode` (type: `string`):

Search lists packages that match a query. Package loads one or more packages by name.

## `query` (type: `string`):

Search text for search mode. Sent as the search parameter to the public package list. Example: phoenix.

## `sort` (type: `string`):

Sort order for search mode. downloads, recent_downloads, inserted_at, updated_at, or name.

## `packages` (type: `array`):

Package names to fetch in package mode, one name per line. Example: phoenix.

## `maxItems` (type: `integer`):

Maximum number of packages to save in one run. Hard cap is 1000.

## `requestDelaySecs` (type: `number`):

Extra pause between Hex.pm requests. The run also waits at least 0.6 seconds so it stays under the published 100 requests per minute limit.

## `timeoutSecs` (type: `integer`):

Stop the run after this many seconds and keep the rows already saved.

## `requestTimeoutSecs` (type: `integer`):

How long to wait for a single Hex.pm response before retrying or recording an error.

## `userAgent` (type: `string`):

User-Agent sent on every request. Keep a contact address so the source can identify the client.

## `proxyConfiguration` (type: `object`):

Optional proxy for Hex.pm requests. Leave disabled unless your IP is blocked.

## Actor input object example

```json
{
  "mode": "search",
  "query": "phoenix",
  "sort": "downloads",
  "packages": [
    "phoenix"
  ],
  "maxItems": 100,
  "requestDelaySecs": 1,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "userAgent": "BigiByteActorResearch/1.0 (contact: research@bigibyte.com)",
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `datasetUrl` (type: `string`):

Full dataset for this run.

## `overviewUrl` (type: `string`):

Table view of the saved packages.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "query": "phoenix",
    "packages": [
        "phoenix"
    ],
    "proxyConfiguration": {
        "useApifyProxy": false
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("bigibyte/hex-pm-elixir-packages-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "query": "phoenix",
    "packages": ["phoenix"],
    "proxyConfiguration": { "useApifyProxy": False },
}

# Run the Actor and wait for it to finish
run = client.actor("bigibyte/hex-pm-elixir-packages-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "query": "phoenix",
  "packages": [
    "phoenix"
  ],
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}' |
apify call bigibyte/hex-pm-elixir-packages-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,bigibyte/hex-pm-elixir-packages-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/zWhcV3gPyMCnQhRbC/builds/nlBSwSramXy6kbta7/openapi.json