# Changelog of arXiv Scraper — Search & Export Paper Metadata (`devilscrapes/arxiv-papers-scraper`) Actor

- **URL**: https://apify.com/devilscrapes/arxiv-papers-scraper/changelog.md
- **Full Actor documentation**: https://apify.com/devilscrapes/arxiv-papers-scraper.md

## arXiv Papers Scraper — Changelog

### 0.5 — 2026-06-05

- **Robustness**: added exponential-backoff retry to `_fetch_page` in
  `src/scraper.py`. The arXiv Atom API (`export.arxiv.org`) can return
  HTTP 503 during weekly maintenance windows or 429 under rate-limiting;
  it also occasionally returns an HTML maintenance page with HTTP 200,
  causing `ET.ParseError`. All three cases are now retried up to 5
  attempts (backoff: 2 s → 4 s → 8 s → 16 s → 30 s, capped).
  `Retry-After` headers are honoured. Non-retryable 4xx responses
  (e.g. 400 bad query) still raise immediately. After all attempts
  exhaust, the run fails loud with a `RuntimeError` — never silently
  returning an empty dataset.
- New constants: `RETRY_STATUS_CODES`, `BACKOFF_BASE_S`, `BACKOFF_CAP_S`,
  `MAX_ATTEMPTS`, `RETRY_AFTER_HEADER`.
- New helpers: `_backoff_delay`, `_parse_atom` (pure, unit-testable).
- Added `tests/test_scraper_retry.py` covering: 503×2 then success,
  persistent 503, HTML-maintenance-page 200 retry, non-retryable 400,
  Retry-After header, backoff growth/cap, and `_parse_atom` directly.

### 0.1.0 — 2026-05-15

- Initial scaffolded release.