Medium Scraper

Medium Scraper extracts articles, metadata, and content from Medium.com — the world's largest publishing platform with 170M+ monthly readers. Scrape by tag/topic, author profile, or publication without any API key or login.

Use it to monitor trending content, research thought leaders, analyze publishing patterns, or build content datasets — all from a single, no-code tool.

What does Medium Scraper do?

Medium Scraper reads Medium's public RSS feeds to collect article data in bulk. Point it at any Medium tag page (e.g., medium.com/tag/artificial-intelligence), author profile (e.g., medium.com/@username), or publication (e.g., medium.com/towards-data-science) and it returns structured JSON records for every article — title, author, publication date, tags, excerpt, thumbnail, and article content (for author/publication feeds).

No browser automation, no API keys, no rate-limit headaches. The actor uses pure HTTP requests, making it extremely fast and cheap — 100 articles in under 5 seconds.

Who is Medium Scraper for?

Content researchers & analysts

• Track what topics are trending in your industry week over week
• Build content datasets for NLP models or sentiment analysis
• Find the most-published authors in a niche for outreach

SEO & content marketing teams

• Monitor competitor publications and author output
• Analyze which tags and topics drive the most engagement
• Track new article publication frequency by author or tag

Investors & market researchers

• Follow thought leaders on specific investment themes (AI, biotech, fintech)
• Monitor startup founders' public writing for product/strategy signals
• Aggregate newsletter-style content from multiple sources automatically

Developers & data engineers

• Build content aggregation pipelines without scraping browser pages
• Feed article data into vector databases for RAG (retrieval-augmented generation)
• Power internal knowledge bases with curated Medium content

Why use Medium Scraper?

• 🚀 Blazing fast — pure RSS parsing, no browser overhead. 100 articles in ~1 second
• 💰 Extremely cheap — HTTP-only, no proxy needed. The cheapest way to get Medium data
• 🔑 No API key required — Medium's RSS feeds are fully public
• 📦 Ready-to-use data — structured JSON/CSV/Excel export, no cleaning needed
• 🏷️ Full tag metadata — every article's topic tags included
• 📝 Article content included — full article text available for author/publication feeds
• 🔄 Scheduled runs — monitor tags or authors on a cron schedule via Apify platform
• 🔌 5,000+ integrations — connect to Google Sheets, Slack, Make, Zapier, Airtable and more

What data can you extract?

Field	Description	Available
title	Article headline	✅ Always
url	Full article URL (cleaned, no tracking params)	✅ Always
author	Author display name	✅ Always
publicationDate	ISO 8601 date (YYYY-MM-DD)	✅ Always
tags	Topic tags (e.g., ["python", "machine-learning"])	✅ Always
excerpt	Article preview snippet	✅ Always
thumbnail	Cover image URL	✅ Always
id	Unique Medium article ID	✅ Always
content	Full article text	✅ Author/publication feeds
publicationName	Publication name (if in a publication)	✅ Always
isLocked	Whether article is member-only	✅ Always
clapCount	Number of claps	❌ Not in public feeds
readingTimeMinutes	Estimated reading time	❌ Not in public feeds

Note: Clap counts and reading times are not exposed in Medium's public RSS feeds. This is a platform limitation, not a scraper limitation.

How much does it cost to scrape Medium?

This actor uses pay-per-event pricing — you only pay for articles you actually scrape. No monthly subscription. All platform costs are included.

	Free	Starter ($29/mo)	Scale ($199/mo)	Business ($999/mo)
Per article	$0.0023	$0.002	$0.00156	$0.0012
100 articles	$0.23	$0.20	$0.156	$0.12
1,000 articles	$2.30	$2.00	$1.56	$1.20

A run fee of $0.005 is charged once per run (covers initialization overhead).

Real-world cost examples:

Input	Articles	Duration	Cost (Free tier)
1 tag page	~10	~1s	~$0.03
1 author profile	~10	~1s	~$0.03
10 tag pages	~100	~3s	~$0.23

Free plan: Apify gives you $5 in free credits. At $0.0023/article, that's ~2,100 articles free to start.

How to scrape Medium articles

1. Go to Medium Scraper on Apify Store
2. Click Try for free
3. In the Start URLs field, enter Medium URLs you want to scrape:
   • Tag page: https://medium.com/tag/technology
   • Author profile: https://medium.com/@username
   • Publication: https://medium.com/towards-data-science
4. Set Max articles (default: 50)
5. Click Start and wait for results (usually under 10 seconds)
6. Download results as JSON, CSV, or Excel from the Dataset tab

Example input for scraping a tag:

{
  "startUrls": [
    { "url": "https://medium.com/tag/artificial-intelligence" }
  ],
  "maxArticles": 100
}

Example input for scraping an author profile:

{
  "startUrls": [
    { "url": "https://medium.com/@ev" }
  ],
  "maxArticles": 50
}

Example input for multiple sources:

{
  "startUrls": [
    { "url": "https://medium.com/tag/python" },
    { "url": "https://medium.com/tag/javascript" },
    { "url": "https://medium.com/@towardsdatascience" }
  ],
  "maxArticles": 200
}

Input parameters

Parameter	Type	Default	Description
startUrls	Array	Required	Medium tag pages, author profiles, or publications to scrape
maxArticles	Integer	50	Maximum number of articles to extract across all URLs
includeLockedArticles	Boolean	true	Whether to include member-only (paywalled) articles
maxConcurrency	Integer	5	Number of parallel feed requests
requestTimeoutSecs	Integer	30	HTTP request timeout in seconds
proxy	Object	None	Optional proxy configuration

Supported URL formats:

URL format	Example
Tag page	https://medium.com/tag/technology
Author profile	https://medium.com/@username
Custom domain author	https://username.medium.com
Publication	https://medium.com/towards-data-science

Direct article URLs (e.g., https://medium.com/p/abc123) are not supported because Medium's Cloudflare protection blocks individual article pages without a real browser. Use tag/author/publication feeds to discover articles instead.

Output examples

Article from tag feed:

{
  "id": "ea82231fdcdc",
  "title": "18 Years on a MacBook: 3 Principles I Use Every Single Day",
  "url": "https://medium.com/macoclock/18-years-on-a-macbook-3-principles-i-use-every-single-day-ea82231fdcdc",
  "author": "Georg Plankl",
  "authorUrl": "",
  "publicationDate": "2026-04-04",
  "publicationName": null,
  "excerpt": "After 30,000 hours, three habits survived everything else — and they'll change how you work.",
  "thumbnail": "https://cdn-images-1.medium.com/max/600/1*8wF4i2sk4g8ypZ-uzyvhkA.jpeg",
  "clapCount": null,
  "responseCount": null,
  "readingTimeMinutes": null,
  "isLocked": false,
  "tags": ["mac", "productivity", "technology", "advice", "apple"],
  "content": null
}

Article from author feed (content included):

{
  "id": "0126fa5c6ce8",
  "title": "Making \"Social\" Social Again",
  "url": "https://ev.medium.com/making-social-social-again-0126fa5c6ce8",
  "author": "Ev Williams",
  "authorUrl": "",
  "publicationDate": "2024-12-12",
  "publicationName": null,
  "excerpt": null,
  "thumbnail": "https://cdn-images-1.medium.com/max/1024/1*NMXyOoeQu3L1ZAXqf8QIYw.jpeg",
  "clapCount": null,
  "responseCount": null,
  "readingTimeMinutes": null,
  "isLocked": false,
  "tags": ["social", "relationships"],
  "content": "Announcing MoziEv Williams, Twitter and Medium Founder, Unveils New Social App I think it's tough to appreciate how much relationships determine the course of our lives..."
}

Tips for best results

• 🏷️ Tag pages return the 10 most recent articles — for a specific author's full history, use their profile URL
• 📚 Author profiles include full content — Medium's author RSS feeds include content:encoded with the full article HTML converted to text
• 🔄 Schedule runs for monitoring — use Apify's scheduler to run every hour/day and track new publications
• 🧩 Combine multiple tags — add several tag URLs to capture articles across related topics in one run
• 📊 Filter by date in post-processing — use publicationDate to filter to articles published in the last N days
• 💡 Use the maxArticles limit — start with 10-20 to verify your URLs work before running large batches

Integrations

Medium Scraper → Google Sheets Use the Google Sheets integration to automatically append new articles to a spreadsheet. Great for content calendars or competitive tracking dashboards.

Medium Scraper → Slack alerts Connect via Make or Zapier to post new articles matching a tag to a Slack channel. Monitor when key authors or topics publish new content.

Medium Scraper → Vector database (RAG) Export article content to Pinecone, Weaviate, or Chroma for semantic search. Power internal knowledge bases or AI assistants grounded in curated Medium content.

Scheduled monitoring Set up a daily Apify schedule to scrape medium.com/tag/your-industry and detect new articles automatically. Combine with webhooks to trigger downstream workflows.

Medium Scraper → Airtable content database Build a content research database — pipe article data into Airtable to track publications by tag, author, and date with built-in filtering and views.

Using the Apify API

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_APIFY_TOKEN' });

const run = await client.actor('automation-lab/medium-scraper').call({
    startUrls: [{ url: 'https://medium.com/tag/technology' }],
    maxArticles: 100,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient(token="YOUR_APIFY_TOKEN")

run = client.actor("automation-lab/medium-scraper").call(
    run_input={
        "startUrls": [{"url": "https://medium.com/tag/technology"}],
        "maxArticles": 100,
    }
)

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["title"], item["author"], item["publicationDate"])

cURL

# Start the actor run
curl -X POST \
  "https://api.apify.com/v2/acts/automation-lab~medium-scraper/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "startUrls": [{"url": "https://medium.com/tag/technology"}],
    "maxArticles": 100
  }'

# Get results (replace DATASET_ID from the run response)
curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_APIFY_TOKEN"

Use with AI agents via MCP

Medium Scraper is available as a tool for AI assistants that support the Model Context Protocol (MCP).

Add the Apify MCP server to your AI client — this gives you access to all Apify actors, including this one:

Setup for Claude Code

$claude mcp add --transport http apify "https://mcp.apify.com?tools=automation-lab/medium-scraper"

Setup for Claude Desktop, Cursor, or VS Code

Add this to your MCP config file:

{
    "mcpServers": {
        "apify": {
            "url": "https://mcp.apify.com?tools=automation-lab/medium-scraper"
        }
    }
}

Your AI assistant will use OAuth to authenticate with your Apify account on first use.

Example prompts

Once connected, try asking your AI assistant:

• "Use automation-lab/medium-scraper to scrape the 50 most recent articles from medium.com/tag/artificial-intelligence and summarize the top themes"
• "Scrape all articles by @ev on Medium and tell me what topics he writes about most"
• "Get the latest 20 articles from the Towards Data Science publication and create a reading list with titles, authors, and links"

Learn more in the Apify MCP documentation.

Is it legal to scrape Medium?

Medium's public RSS feeds (medium.com/feed/...) are designed for automated consumption — they're how RSS readers, aggregators, and tools like this one work. The data returned is the same data displayed on publicly accessible pages.

Best practices for responsible use:

• Only scrape publicly available content (this actor does not access member-only content)
• Respect Medium's robots.txt and rate limits (this actor is rate-limited by default)
• Do not use scraped content for spam or deceptive purposes
• Attribution: always credit Medium and the original authors when republishing
• Review Medium's Terms of Service for your specific use case

This actor does not log in, does not bypass paywalls, and does not access private or member-only content.

FAQ

How many articles can I scrape from one tag page? Medium's tag RSS feeds return the 10 most recent articles per request. To get more articles from a topic, combine multiple tag URLs (e.g., technology, software-engineering, programming) or monitor the same tag over time with scheduled runs.

Why does some article content show as null? Article content (content field) is only included in author profile and publication feeds — not in tag/topic feeds. Tag feeds only include a short snippet. Use an author profile URL (e.g., medium.com/@username) or publication URL (e.g., medium.com/towards-data-science) if you need full article text.

Why are clap counts not available? Clap counts and reading times are not exposed in Medium's public RSS feeds — they're only available on individual article pages, which require JavaScript execution (Cloudflare protection). The actor uses RSS for reliability and speed. If you need clap counts, note that they change frequently anyway, making historical comparisons unreliable.

How fast does this scraper run? Very fast. A single tag page (10 articles) takes about 1 second. Multiple sources run in parallel — 10 feeds with 10 articles each finishes in 2-3 seconds. This actor uses pure HTTP (no browser), making it among the fastest scrapers on Apify Store.

Why can't I scrape direct article URLs? Individual article pages on medium.com are protected by Cloudflare's bot management, which requires JavaScript execution to pass. This actor uses HTTP-only requests (no browser), so article pages return a Cloudflare challenge instead of content. Workaround: scrape the author's profile feed to get all their articles, or use a tag feed to discover articles by topic.

Does it work with custom Medium domains? Yes! If an author has a custom domain like username.medium.com, use that URL directly — the actor automatically converts it to the correct RSS feed URL.

Can I scrape member-only (paywalled) articles? This actor collects metadata (title, author, date, tags, excerpt) for member-only articles from the feed. Full content of paywalled articles is not available without a Medium membership, so the content field will be null for locked articles.

Other scrapers you might find useful

• Reddit Scraper — Collect posts and comments from subreddits
• Substack Scraper — Extract newsletters and posts from Substack
• RSS Feed Reader — Aggregate content from any RSS/Atom feed
• YouTube Transcript — Extract transcripts and metadata from YouTube videos
• Trustpilot Scraper — Scrape reviews and ratings from Trustpilot