Medium Scraper

Extract articles from Medium by tag, author, publication, or search query. Get full content, claps, read time, author data, and engagement metrics. Output as text, HTML, or markdown — perfect for content research and AI training data.

What Can You Extract?

Field	Description
title / subtitle	Article title and subtitle
content	Full article text, HTML, or markdown
author	Name, username, bio, follower count
publication	Publication name and URL
claps	Number of claps (engagement metric)
responses	Number of responses/comments
readTime	Estimated reading time (minutes)
tags	Topic tags
isMemberOnly	Whether article is behind paywall
publishedDate	Publication date (ISO 8601)
featuredImage	Cover image URL
wordCount	Approximate word count
language	Detected language

How to Use

Scrape by tag (zero-config):

{}

Extracts 50 articles tagged "web-scraping" by default.

Scrape specific articles:

{
  "articleUrls": [
    "https://medium.com/@user/article-title-abc123",
    "https://betterprogramming.pub/some-article-def456"
  ]
}

Scrape by author:

{
  "authorUrls": ["https://medium.com/@towardsdatascience"],
  "maxArticlesPerSource": 100,
  "contentFormat": "markdown"
}

Scrape by publication with filters:

{
  "publicationUrls": ["https://betterprogramming.pub"],
  "minClaps": 500,
  "dateFrom": "2025-01-01",
  "memberOnly": "free"
}

Input Configuration

Parameter	Type	Default	Description
articleUrls	array	[]	Direct Medium article URLs
tags	array	["web-scraping"]	Medium topic tags
authorUrls	array	[]	Author profile URLs
publicationUrls	array	[]	Publication homepage URLs
searchQueries	array	[]	Search queries
maxArticlesPerSource	integer	50	Max articles per tag/author/publication
includeContent	boolean	true	Extract full article content
contentFormat	string	"text"	text, html, or markdown
dateFrom	string	—	Filter: only articles after this date
dateTo	string	—	Filter: only articles before this date
memberOnly	string	"all"	all, free, or member_only
minClaps	integer	0	Minimum clap count filter
sortBy	string	"relevance"	relevance, latest, or most_clapped

Output Example

{
  "url": "https://medium.com/@user/the-complete-guide-abc123",
  "articleId": "abc123",
  "title": "The Complete Guide to Web Scraping in 2026",
  "subtitle": "Everything you need to know",
  "author": {
    "name": "John Smith",
    "username": "johnsmith",
    "url": "https://medium.com/@johnsmith"
  },
  "content": "Full article text content...",
  "readTime": 12,
  "claps": 4500,
  "responses": 23,
  "tags": ["web-scraping", "python", "automation"],
  "isMemberOnly": false,
  "publishedDate": "2026-02-15T10:30:00.000Z",
  "wordCount": 3200,
  "featuredImage": "https://miro.medium.com/...",
  "scrapedAt": "2026-03-11T12:00:00.000Z"
}

Edge Cases

• Member-only articles: Metadata (title, claps, tags) extracted; content availability depends on server-side rendering
• Custom domain publications: Detected as Medium via meta tags, extracted normally
• Rate limiting (429): Automatic retry with 2s → 5s → 10s backoff, proxy rotation
• Deleted articles: error: "ARTICLE_NOT_FOUND" returned, no PPE charge
• Non-English articles: Content extracted with Unicode support, language detected

Pricing

$3.90 per 1,000 articles scraped (PPE — pay per article successfully extracted)

Pricing includes all platform compute costs — no hidden fees.

Run Type	Articles	Cost
Tag research	50	$0.20
Author archive	200	$0.78
Publication scrape	1,000	$3.90
AI training dataset	50,000	$195.00

FAQ

Can I scrape member-only (paywalled) articles?

Metadata (title, author, claps, tags) is extractable for all articles. Full content extraction for member-only articles depends on what Medium renders without authentication.

What content formats are supported?

Plain text (cleaned, readable), HTML (raw article HTML), and markdown (converted from HTML structure).

Can I filter articles by date or popularity?

Yes: dateFrom/dateTo for date range, minClaps for engagement threshold.

Does it work with custom domain publications?

Yes. Publications on custom domains (e.g., betterprogramming.pub) are detected as Medium via meta tags and extracted normally.

Can I use this for AI training data?

Yes — text or markdown output is directly usable for fine-tuning and RAG pipelines. Respect author copyright: data for analysis only, not republication.

---

Legal notice: Medium's ToS prohibits automated access, but article content is publicly available and indexed by search engines. Use responsibly. Do not circumvent authentication or republish scraped content.

Related actors by Junipr: Spotify Playlist Scraper | AI Content Detector