Arch Linux Packages Scraper

Arch Linux Packages Scraper

This Arch Linux packages scraper collects detailed package information from official repositories and the AUR. You can easily target specific repositories, search for a individual package, or explore AUR submissions, then export structured dataset results in JSON, CSV, or Excel formats.

Use Cases

• Package Lookup: Search official repositories for specific package names (like python-requests) to retrieve version information, maintainers, and architecture details.
• Single Package Detail: Load detailed package metadata—including dependencies, licenses, maintainers, and installed size—for targeted packages.
• AUR Search: Query AUR submissions for specific keywords (like yay) and extract matching results.
• Repository Filtering: Limit searches to specific repositories like extra, core, or multilib.
• Dataset Export: Set item limits and download clean structured datasets directly in JSON, CSV, or Excel formats.

Data Fields Extracted

Depending on your run configuration, the scraper outputs clean dataset rows containing:

• Package name, release, version, and repository assignment
• System architecture details
• Short description and upstream package URLs
• License details, dependency lists, and maintainer information
• Installed package size in bytes
• Last update timestamp
• Dataset origin metadata and timestamp tracking

Input Options

Parameter	Type	Default	Description
mode	string	search	Choose between official repo search, detailed single package lookup, or AUR search.
query	string	python-requests	Search keyword, exact package name, or structured package path.
repos	array	empty	Filter by specific official repository names. Leave empty to cover all official repositories.
includeAur	boolean	false	Include matching AUR results alongside official search results.
maxItems	integer	100	Maximum number of package rows to extract and save.
requestDelaySecs	number	1	Delay interval between outbound requests.
timeoutSecs	integer	300	Total runtime timeout for the actor run.
requestTimeoutSecs	integer	30	Maximum wait time per individual request.
proxyConfiguration	object	Default settings	Optional proxy setup if needed for custom routing.

Example Input

{
  "mode": "search",
  "query": "python-requests",
  "repos": ["extra"],
  "includeAur": false,
  "maxItems": 100,
  "requestDelaySecs": 1,
  "timeoutSecs": 300,
  "requestTimeoutSecs": 30,
  "proxyConfiguration": { "useApifyProxy": false }
}

Dataset Output

Field	Type	Description
pkgname	string	Package name.
repo	string	Associated repository or aur.
arch	string	Architecture assignment.
pkgver	string	Package version string.
pkgrel	string	Release number.
pkgdesc	string	Summary description.
url	string	Upstream website or project URL.
licenses	array	Associated license identifiers.
maintainers	array	Names or IDs of maintaining contributors.
depends	array	Direct dependency requirements.
installed_size	integer	Total installed footprint in bytes.
last_update	string	Timestamp of the latest update in UTC ISO format.
source	string	Origin indicator (official or aur).
scraped_at	string	Timestamp indicating when the row was collected.
error	string	Displayed on error records when applicable.
status	integer	HTTP response status on error records.
message	string	Descriptive status message on error records.

Example Output

{
  "pkgname": "python-requests",
  "repo": "extra",
  "arch": "any",
  "pkgver": "2.34.2",
  "pkgrel": "1",
  "pkgdesc": "Python HTTP for Humans",
  "url": "[https://requests.readthedocs.io/](https://requests.readthedocs.io/)",
  "licenses": ["Apache-2.0"],
  "maintainers": ["grawlinson", "polyzen"],
  "depends": ["python"],
  "installed_size": 796237,
  "last_update": "2026-05-15T09:18:41.190Z",
  "source": "official",
  "scraped_at": "2026-10-12T00:00:00+00:00"
}

Frequently Asked Questions

Can I export the extracted dataset to CSV or Excel? Yes. You can export datasets from Apify directly in CSV, Excel, or JSON formats.

How does deduplication work across runs? Records are automatically deduplicated by repository, architecture, and package name. Each run stamps a fresh scraped_at timestamp.

What happens when a package search returns no results? If a query produces no results, the scraper completes cleanly with zero saved records and logs a clear run summary without raising errors.

Trademark Notice

Arch Linux and related names, logos, and trademarks belong to their respective owners. Their use in this documentation is solely to identify the target dataset and does not imply affiliation, sponsorship, endorsement, or partnership.

Disclaimer

This Actor is an independent tool and is not affiliated with, endorsed by, sponsored by, or otherwise officially associated with Arch Linux or its owners. All data is collected from publicly accessible sources. Users are responsible for complying with applicable terms of use, rate limits, and licensing requirements.

Is web scraping legal?

This Actor only collects publicly available data without logging in or bypassing access controls. Scraping public data is generally legal, but how you use it matters:

• Personal data: If the results include personal data (such as names or contact details), privacy laws like GDPR and CCPA may apply. Only collect and store personal data when you have a legitimate reason.
• Copyright: Content like package descriptions or documentation may be protected by copyright. Avoid republishing full copyrighted materials without permission.
• Terms of use: Respect source guidelines, terms, and rate limits.
• Legal advice: This general information is not legal advice. Consult a legal expert if you have specific regulatory questions.

To learn more, check out Apify's comprehensive guide: Is web scraping legal?