Medium Scraper

The newest stories from any Medium tag, writer or publication — with authors, tags, dates and the full text of free stories — as clean JSON.

Medium hosts millions of writers and thousands of publications, from Better Programming to Python in Plain English, on medium.com, on writers' own <name>.medium.com subdomains, and on publications' own domains. This Actor reads any of them and returns their newest stories: title, author, publication, tags, publish and update dates, cover image, the story body as HTML and plain text, word count, and an estimated reading time. Free stories come in full; member-only stories come with the preview Medium shows the public and are flagged as such. Paste a tag, writer, publication or story URL; there is no login, no API key, and no proxy to configure for small runs. Built for content monitoring, writer and publication research, topic tracking, and AI and search datasets.

⚡ Quick start

{
  "startUrls": [{ "url": "https://medium.com/tag/programming" }],
  "maxItems": 5
}

🧩 Input

Field	Type	Required	Default	Description
startUrls	array	yes	—	Medium tag pages (medium.com/tag/<tag>), writer profiles (medium.com/@<name> or <name>.medium.com), publications (medium.com/<publication>, the publication's own domain, or …/tagged/<tag>), or single story URLs.
maxItems	integer	no	—	Maximum number of stories per start URL, newest first. The input form starts at 5; Medium lists the newest 10 stories of each tag, writer or publication, so a start URL returns at most 10.
includeContent	boolean	no	true	Include full text for tag pages: look each story on a tag page up on its writer's or publication's page, for its full text, word count and member-only flag. Writer and publication URLs always include the full text.
proxyConfiguration	object	no	{ "useApifyProxy": false }	Proxy settings. Small runs need no proxy; use Apify Proxy (datacenter) when scraping dozens of tags, writers or publications at once.

Custom domains work. Many publications run on their own domain (for example python.plainenglish.io). Paste it as it is: the Actor checks that the site is a Medium publication and says so in the log when it is not.

📤 Output

Each story is one dataset row.

Fields

Field	Type	Description
id	string	Medium story ID (the hex code at the end of the story's URL).
title	string	Story title.
url	string	The story's public page, without tracking parameters.
author	string	The writer's display name.
authorUrl	string	The writer's Medium profile. null for stories in a publication, unless you scraped the writer's own URL.
publication	string	The publication the story appeared in, or null for a writer's own story.
publicationUrl	string	The publication's home page.
tags	array	The story's tags, as Medium spells them in tag URLs.
publishedAt	string	When the story was published (ISO 8601).
updatedAt	string	When the story was last updated (ISO 8601).
snippet	string	The one-line preview Medium shows on tag pages and for member-only stories, when it shows one.
imageUrl	string	Cover image URL.
contentHtml	string	The full story as HTML. null for member-only stories, and for tag-page stories when includeContent is off.
contentText	string	The full story as plain text, one paragraph per line.
wordCount	number	Words in contentText.
readingTimeMinutes	number	Estimated reading time, at 265 words per minute. An estimate from wordCount, not Medium's own figure.
isMemberOnly	boolean	true when Medium withholds the full text from the public (a member-only story), false when the full text is included, null when not checked.

💡 Use cases

• Content monitoring — follow the tags, writers and publications in your field and catch every new story as it is published.
• Writer and publication research — see who publishes on a topic, how often, and how much of their work is member-only.
• Topic tracking — collect stories and their tags to spot rising subjects in tech, business, and culture.
• AI and search datasets — build clean, attributed corpora from the full text of free Medium stories.

❓ FAQ

• Do I need a Medium account or an API key? No. The Actor reads what Medium publishes to any visitor.
• Why do I get at most 10 stories per URL? Medium publicly lists only the newest 10 stories of each tag, writer or publication. To collect more, add more URLs — several tags, or a publication filtered by tag (…/tagged/<tag>) — or run the Actor on a schedule and keep the new stories each time.
• Can I get the full text of member-only stories? No. Member-only stories come with the preview Medium shows the public; isMemberOnly is true on those rows and the text fields are null.
• Can I get claps and response counts? No. Medium only shows them on story pages, which it reserves for browsers; this Actor returns the data Medium publishes for readers' feeds, which leaves those counts out.
• Can I search Medium by keyword? Not directly: Medium's search is not open to automated access. Use tag pages instead (medium.com/tag/<keyword>), which cover most topics.
• Can I scrape an old story? A single story URL works while the story is among the newest 10 of its writer or publication. Older stories are not listed publicly, and the log says so.
• Why is authorUrl empty for some stories? Stories in a publication do not name the writer's profile. Scrape the writer's own URL (medium.com/@<name>) to get it on every row.
• Do I need a proxy? Not for small runs. Medium throttles many requests from one IP; for dozens of URLs, turn on Apify Proxy (datacenter).

🔗 More scrapers by Anyx

• Substack Scraper
• Hacker News Scraper
• Google News Scraper
• Web Content Scraper

🤝 Anyx Solutions

We build custom scrapers and data-extraction pipelines.

• Email: tantosthor@gmail.com

🖼 Image credit

Image credit: medium.com