Medium Blog Scraper

Extract articles from Medium by topic via RSS feed. Get titles, authors, author URLs, article URLs, tags, publication date, publication name, summary, cover images, language, claps, and reading time — clean JSON, no Cloudflare issues.

Sample output

{
  "title": "AWS Compute Options: Lambda and ECS",
  "author": "Rhuturaj Takle",
  "authorUrl": "https://medium.com/@rhutu.takle",
  "authorImageUrl": "https://miro.medium.com/v2/...",
  "url": "https://medium.com/@rhutu.takle/aws-compute-options-lambda-and-ecs-be23da0a096d",
  "tags": ["devops", "programming", "learning", "aws"],
  "topics": ["DevOps", "Cloud Computing", "Programming"],
  "publishedDate": "Tue, 14 Jul 2026 16:31:58 GMT",
  "updatedDate": "2026-07-14T16:31:58.837Z",
  "publication": "",
  "summary": "A practical guide to two of AWS's core compute services — Lambda for serverless, event-driven code execution, and ECS for orchestrating…",
  "topicUrl": "https://medium.com/feed/tag/programming",
  "imageUrl": "https://miro.medium.com/v2/...",
  "language": "en",
  "claps": 347,
  "readingTime": "5 min read",
  "readingTimeMinutes": 5,
  "subtitle": "A comparison of AWS Lambda and ECS for different workloads",
  "responsesCount": 12,
  "responses": [
    {"id": "r_123", "responseToPostId": "p_456", "author": "Jane Doe", "text": "Great writeup, the ECS section helped me a lot.", "createdAt": "Wed, 15 Jul 2026 08:12:00 GMT"}
  ],
  "firstPublishedAt": "Tue, 14 Jul 2026 16:31:58 GMT",
  "wordCount": 1200,
  "isMemberOnly": false,
  "readingTimeSeconds": 300,
  "license": "all-rights-reserved",
  "allowResponses": true,
  "guid": "aws-compute-options-lambda-and-ecs-be23da0a096d",
  "id": "abc123def456",
  "slug": "aws-compute-options-lambda-and-ecs-be23da0a096d",
  "isPublished": true,
  "voterCount": 89,
  "totalClapCount": 347,
  "sectionCount": 12,
  "audience": "public",
  "series": "",
  "isSeries": false,
  "type": "article",
  "source": "medium"
}

Input

Field	Type	Default	Description
topic	string	programming	Medium topic tag (e.g. programming, ai, technology) or full Medium/custom domain URL
maxResults	integer	10	Max articles (1-50)
fetchDetails	boolean	true	Fetch article pages for claps, images, reading time (slower but richer)
detailConcurrency	integer	5	Concurrent detail fetches (1-20)

How it works

Three-phase extraction:

1. RSS feed (medium.com/feed/tag/{topic}) — returns structured XML with titles, authors, tags, dates, and summaries. Fast and not blocked by anti-bot. Supports full URLs and custom domains. Note: Medium RSS caps at 10 items with no pagination links.
2. Stream fallback (only when maxResults > what RSS returned) — loads the topic's tag/user/publication page in a headless browser, reads the page's Apollo state, and scrolls the infinite feed to keep hydrating more Post entities, yielding up to maxResults additional posts with claps, reading time, tags, and response counts. This is what makes maxResults above 10 actually work.
3. Article pages (one browser tab per article, concurrent) — extracts actual clap counts, images, author avatars, language, reading time, word count, member-only status, additional tags from the page's Apollo state (merged with RSS tags, deduplicated), and a best-effort responses list. If the browser path is unavailable it falls back to plain HTTP.

Why a browser?

Medium serves article and tag pages behind a Cloudflare JS challenge. Plain HTTP requests from datacenter IPs get the challenge page, which previously made every detail field (claps, responsesCount, wordCount, readingTimeMinutes, …) come back null. The actor's Docker base image (apify/actor-python-playwright) ships Chromium, so detail extraction actually works.

Output fields

All output fields plus a few extra ones beyond the classic set:

• responses — array of {author, text, createdAt} for responses Medium hydrated on the article page (empty when unavailable).
• firstPublishedAt — first-publication timestamp (RFC 2822) from the article page.
• topics — array of Medium topic names the post belongs to (from Apollo Post.topics), kept distinct from tags and merged into tags when tags are missing.

Use cases

• Content curation for newsletters
• Trend analysis in tech topics
• Competitive research
• Building reading lists by topic
• Monitoring new articles in your niche

Pricing

$0.0005 per article ($0.50 per 1k articles).