# Changelog of Crossref API New Works Feed: New DOIs & Journal Articles (`fecklesspleb/crossref-new-works-feed`) Actor

- **URL**: https://apify.com/fecklesspleb/crossref-new-works-feed/changelog.md
- **Full Actor documentation**: https://apify.com/fecklesspleb/crossref-new-works-feed.md

## Changelog

### Unreleased

- **Invalid input no longer FAILS the run** (2026-11-04, local, not built or pushed; `actor.json` version stays `0.2`). Input that `parse_input` rejects (e.g. an unknown `types` value, `mode: "weekly"`, a bad ISSN, `maxItems: "abc"`) used to call `Actor.fail` (run FAILED, exit 1, status message = the reason). The run now ends SUCCEEDED (exit 0) with one free status row `{"recordType": "status", "type": "status", "status": "error", "statusRow": true, "message": "Invalid input, nothing was fetched or charged: <reason>", "error": "<reason>", "charged": false, "scrapedAt": ...}` and the same message as the terminal status message. No request is made (not even robots.txt), no work is charged and no `RUN_SUMMARY` is saved. The prefix is "nothing was fetched or charged" (not "...and no results were charged") because this Actor's pricing has no start event; the reason texts are unchanged.
  - Wrong-type crashes now give the same row with the reason `a field has the wrong type (<python error>).` (e.g. `lookbackDays` or `maxItems` = Infinity used to crash with an uncaught `OverflowError`, exit 91).
  - A whole input that is not a JSON object (a string, a number, a list of objects) used to crash with `AttributeError` (exit 91); it now gives the same row with the reason `the input must be a JSON object, got <type>.` An empty list or null input still means `{}`.
  - Real upstream and runtime failures (API down, 403, robots.txt) are unchanged: they keep the existing `{"type": "status", "status": "warning"}` fail-soft row.
  - `requirements.txt`: `apify >= 4.0.0, < 5.0.0` pinned to `apify >= 4.1, < 4.2`.
  - Memory unchanged (default 256, min 256, max 2048, defaultRunOptions 256). Pricing, default input, prefill and `exampleRunInput` unchanged.
  - Files: `my_actor/main.py` (new `end_invalid_input`, wrong-type wrap around `parse_input`, non-object check), `README.md` ("Invalid input" bullet under How it works), `requirements.txt`, `CHANGELOG.md`, `tests/test_invalid_input.py` (new).

- Default `maxItems` lowered from 10 to 9 (2026-11-02) in the input schema default, the prefill, the code fallback (`DEFAULT_MAX_ITEMS`, used for omitted, null or blank `maxItems`) and `exampleRunInput`. `{}`, the prefill and the example now cost at most $0.009, under the $0.01 Store test limit (10 works were exactly $0.01, no margin). The first page asks for 29 rows instead of 30. `QUICK_MAX_ITEMS` still equals the default, so the small-run time budget now applies to `maxItems` 9 or less: a run with `maxItems` 10 uses the normal request timeouts and retries. `actor.json` version stays `0.2`.
  - Files: `my_actor/main.py` (constant and comment), `.actor/input_schema.json` (`maxItems` default, prefill, description "Default 9 (about $0.009)"), `.actor/actor.json` (`exampleRunInput.maxItems` 9), `pricing.json` (`_note` and `emptyInputAndPrefill`, which still said 100 / $0.10), `README.md` lines 70, 108, 125 and 126 only. `README.proposed.md` is not touched (it has a pending edit).
  - Tests: `tests/test_input.py` (`test_empty_defaults`; `test_blank_max_items_is_default_10` renamed `..._9`), `tests/test_main.py::test_empty_input_one_request` (9 works, 29 rows), `tests/test_polite_pool.py` (3 tests, 9 pushed), `tests/test_pricing.py` (`test_readme_price`, `test_small_default_run_about_one_cent`), `tests/test_gap12_fail_soft.py` (`test_success_has_no_status_item`, `test_store_that_cannot_be_opened_is_a_warning`, `test_default_is_ten_and_quick` renamed `test_default_is_nine_and_quick`).

- Optional Crossref polite pool. If the environment variable `CROSSREF_MAILTO` holds a valid email address, requests to api.crossref.org add it as the `mailto` query parameter, and the User-Agent ends with `; mailto:<address>)`, the form in Crossref's etiquette example. If it is unset or invalid, requests are unchanged (public pool).

- The address is never logged, pushed to the dataset or saved to RUN_SUMMARY or the key-value store. The log says only "Crossref polite pool: on/off", once per run. RUN_SUMMARY has a `politePool` true/false flag. Error text and httpx request logs are redacted.

- The rate stays at 1 request per second with 1 connection, inside Crossref's limits for list requests (since 21 July 2026: 1/s in the public pool, 3/s in the polite pool).

- No address is hardcoded anywhere. Tests use a fake `example.org` address in mocked requests only.

- Docs: the `query` description no longer says matches are "ranked by Crossref". Results are always sorted newest first by date, and Crossref keyword matching is loose, so the description suggests filters or narrower phrases for precision. No relevance-sort option was added.

- Docs: the input schema and README now state the dedupe limit: at most 200,000 DOIs per state key, with the earliest-saved DOIs dropped first (`SEEN_CAP` in `my_actor/main.py`). No code change; `actor.json` version stays `0.2`.

- Docs: README rewritten for the Store (use cases, "Getting a complete daily feed", Limits). The daily volume now uses verified counts (about 27,000 to 35,000 new journal articles and 42,000 to 87,000 works of all types per weekday, 26 to 30 October 2026) instead of the wrong "46,000 per day"; the `maxItems` description says the same. Rate-limit text now matches Crossref's limits since 21 July 2026 (list requests 1/s public, 3/s polite). Store title, SEO title, SEO description and description updated. No code change.

### 0.2, later build (2026-10-30)

- Built as version 0.2 (`actor.json` version stays `0.2`); there is no 0.3.
- Default `maxItems` is now 10 (about $0.01 at $0.001 per work) in the input schema default, the prefill, the code fallback (`DEFAULT_MAX_ITEMS`) and `exampleRunInput`, so `{}` and the Store test cost about one cent.
- Default memory is now 256 MB (`defaultMemoryMbytes` and `defaultRunOptions.memoryMbytes`). Measured peak for the Actor process is 47 MB.
- Time budget: a small run (`maxItems` 10 or less) gives each request 10 s and 2 tries, caps retry waits at 2 s and starts no new request after about 40 s. Every run stops starting requests 20 s before the platform timeout, and an abort stops new requests at once.
- Fail soft: when no work is saved (API down, slow or blocking, robots.txt unreadable, nothing matched) the run pushes one free status item with the reason, logs a warning and ends SUCCEEDED.
- The dedupe store is optional: if it cannot be opened or saved, the run logs a warning and carries on.
- The status message now names the real reason a host was skipped (it used to say "robots.txt disallows the API" for any robots.txt problem) and says "unknown number matched" instead of "None matched".

### 0.2 (2026-10-29)

- Default `maxItems` stays at 100 (about $0.10) in the input schema default, the prefill and the code fallback (`DEFAULT_MAX_ITEMS`), so empty input `{}` stays cheap. Users can raise it up to 100,000, and the docs now say so.
- `maxItems` left blank gives 100. 0 or less is treated as 1 (the schema minimum is 1); there is no unlimited setting. Both are now documented and tested.
- README cost warning: an unfiltered full day is about 46,000 works and could cost tens of dollars, so use filters or a cap.

### 0.1 (2026-10-29)

- First version: Crossref REST API `/works` feed with cursor paging, 1 req/s, robots.txt check, filters (type, ISSN, DOI prefix, member, funder, keywords, abstract/funder/CC flags), dedupe across runs, pay-per-event `work` event. No contributor names, ORCID iDs, affiliations or abstracts in output.