Medium User RSS Posts Scraper

Collects public posts from one or more Medium user RSS feeds. The active entry point uses direct HTTP requests and Cheerio to parse feed-provided content; it does not launch a browser or use a proxy.

What this Actor does

• Accepts one username or a list of usernames.
• Divides maxItems fairly across requested feeds.
• Reads the public feed at medium.com/feed/@username.
• Deduplicates posts by article URL across all requested users.
• Extracts article identity, author and publication data, feed content HTML and text, excerpts, content counts, headings, images, links, tags, timestamps, and feed metadata.

The content fields reflect what Medium publishes in the RSS feed. They can be excerpts or full feed content depending on the user feed; the Actor does not bypass paywalls or fetch protected article pages.

Input

{
  "username": "karpathy",
  "maxItems": 5
}

For several users:

{
  "usernames": [
    "karpathy",
    "example"
  ],
  "maxItems": 10
}

At least one non-empty username or usernames list is required. Usernames may include a leading @ and must contain only letters, numbers, dots, underscores, or hyphens. maxItems defaults to 10 and accepts values from 1 to 200.

Output

Each successful dataset row is one public user-feed post. Every row also includes status, position, source, sourcePageUrl, found, dataAvailable, and success. A failed or empty feed is stored as an explicit diagnostic row and is never shaped like a successful post.

{
  "recordType": "mediumUserPost",
  "userPostRank": 1,
  "articleId": "example-post",
  "articleTitle": "An example post",
  "articleUrl": "https://medium.com/@karpathy/an-example-post",
  "articleDomain": "medium.com",
  "authorName": "Example author",
  "authorUsername": "karpathy",
  "authorProfileUrl": "https://medium.com/@karpathy",
  "contentText": "Public feed content...",
  "contentWordCount": 1200,
  "estimatedReadMinutes": 6,
  "feedUrl": "https://medium.com/feed/@karpathy",
  "sourceHttpStatus": 200,
  "extractionMethod": "medium-full-content-rss",
  "proxyConfigured": false,
  "scrapedAt": "2026-08-16T10:00:00.000Z"
}

Optional values are omitted when the feed does not publish them. The dataset view highlights rank, title, author, publication, excerpt, tags, publication time, and URL.

Storage

Posts and explicit diagnostics are written to the default dataset. The fixed OUTPUT key reconciles dataset, success, and diagnostic counts plus the normalized usernames and run status.

Local QA

npm test
apify run --purge --input-file qa-inputs/local-single.json
node validate-datasets.js
apify run --purge --input-file qa-inputs/local-multiple.json
node validate-datasets.js

Cost and limitations

There is no external API fee. Apify compute charges may still apply. Medium can disable or change RSS feeds, return fewer items, omit metadata, or expose only excerpts.

FAQ

Does this Actor scrape private or paywalled content?

No. It reads public RSS feeds only and does not authenticate or open protected article pages.

Why are there fewer posts than maxItems?

The feed may contain fewer entries, a feed request may fail, or duplicate URLs may be removed.

Can I pass a proxy?

No. The active implementation uses direct HTTP requests and reports proxyConfigured as false.

Disclaimer

Use this Actor only where you have a lawful basis to collect and use public feed data. Respect Medium terms, RSS policies, rate limits, copyright, privacy requirements, and applicable law.