Turn any RSS or Atom feed into clean JSON. Paste a website and it finds the feed. Keyword and date filters, only-new-items mode for schedules. $0.80 per 1K.
Unreleased: default mode for an empty input (GAP8)
An empty input ({}) no longer fails with HTTP 400 or a missing-field error. feedUrls is no longer required and now has a schema default (and prefill) of ["https://www.nasa.gov/feed/"]. When feedUrls is empty or is exactly that sample, the run is in default mode: it reads the NASA feed, returns at most 10 items (a smaller maxItemsPerFeed wins), logs Default mode: ..., adds defaultMode and defaultFeed to RUN_SUMMARY and prefixes the status message with Default mode (sample feed): .
Input schema: the prefill changed from Hacker News and The Verge to the NASA sample feed, so the Store test runs default mode. Runs that list any other feeds behave exactly as before.
Lint only: removed unused imports in common.py (pyflakes clean). No behaviour change.
Unreleased: robots.txt on every redirect, unreadable robots.txt skips the host
Redirects are now followed one hop at a time, and robots.txt is checked for every hop. A feed or page that redirects to a disallowed path, or to another host whose robots.txt disallows it, is reported as skipped in RUN_SUMMARY and not fetched. Up to 5 redirects are followed.
If a host's robots.txt can't be read (a server error after one retry, a network failure, or a 200 that is an HTML page or not text/plain), the whole host is skipped for the run with a warning, and the feed is skipped with the reason. Before, such a host was read as if it had no rules. A missing robots.txt (any 4xx) still means no rules.
Unreleased: undated items
With postedWithinDays set, items without a publish date are counted in RUN_SUMMARY (undatedKept) and logged. Set keepUndatedItems to false to drop them.
Unreleased: JSON Feed
JSON Feed 1.0 and 1.1 feeds are read, and /feed.json is one of the paths tried when a page has no feed link.
