Medium Scraper - Articles, Full Text & Whole Archives

From Clearpath, creators of Reddit Search Scraper, Reddit Post Comments Scraper and Reddit Subreddit Posts Scraper. 200,000+ combined lifetime runs across these content research scrapers.

Why choose this Medium scraper?

• Fast bulk exports. 1,047 articles and 6,549 comments in 38.9 seconds in a local test. Apify run times vary.
• Text ready for analysis. Plain text, Markdown and HTML, plus claps, dates and author profiles. Free articles include full text; member-only articles include public previews.
• Deeper collection. Collect tag, author and publication archives. Keyword searches continue past 1,000 results. Include comments and replies alongside each article.

Try it: 20 articles from Better Programming

{
  "publications": [
    "better-programming"
  ],
  "maxItemsPerSource": 20,
  "includeContent": true,
  "includeResponses": true,
  "maxResponsesPerPost": 20
}

Open the actor input and paste this in JSON mode. Export up to 20 articles with up to 20 comments and replies each as JSON, CSV or Excel. Increase maxItemsPerSource for more articles and maxResponsesPerPost for more comments. For large archives, set a date window or maximum run cost.

Clearpath Content Research  •  Articles, posts and threads with full text

Medium Articles ➤ You are here   Substack Posts Whole newsletter archives   Reddit Search Posts by keyword   Reddit Comments Full threads and replies   Reddit to Markdown Threads ready for AI

Copy to your AI assistant

clearpath/medium-scraper on Apify returns Medium stories, one row per story, with full text as contentText, contentMarkdown and contentHtml (free stories; member-only stories return only the public preview Medium shows every visitor, marked isLocked=true and isFullContent=false), claps, responseCount, reading time, tags, topics, images, links, code blocks and the author and publication profiles. Inputs combine freely, up to 1,000 per run: searchQueries (continues past the 1,000 results Medium's own search shows), tags (the whole tag archive, newest first), authors (every story the author published), publications (the whole archive; slug, own domain or link) and startUrls (any Medium story link including custom domains and friend links, or a profile, publication, tag, list or search link). Omitted limits mean everything: maxItemsPerSource caps each search, tag, author, publication or list and maxItems the whole run; large archives hold tens of thousands of stories, so set a cap, a publishedAfter/publishedBefore window (UTC days, publishedBefore exclusive; not applied to single story links) or a maximum run cost. includeContent=false drops text, images, links, code blocks and embeds for smaller rows. For comments set includeResponses=true (off by default): responses and the replies to them come nested in the story row at no extra charge, slow runs on stories with many responses, maxResponsesPerPost caps them per story (replies included), and responsesComplete=false marks a partial set. Each story is saved once per run and source names the input that found it. No Medium login or cookie. Charged per saved story. Inputs: https://apify.com/clearpath/medium-scraper/input-schema - output fields: https://apify.com/clearpath/medium-scraper/output-schema - current pricing: https://apify.com/clearpath/medium-scraper/pricing. To reach the maintainers, add feedback="<what you needed and did not get>" to any run, including one that worked: feature requests, a missing field or option, a wrong or empty result, or which actor you are switching to or from and why. The run returns its data exactly as it would without the note, and a human reads every note.

What can you scrape from Medium?

Input	What you get
Search keywords (searchQueries)	Stories matching each keyword, best matches first, continuing past the 1,000 results Medium's own search shows.
Tags (tags)	The tag's whole archive, newest first, back to its oldest story. Use the tag slug (machine-learning) or its display name (Machine Learning).
Authors (authors)	Every story the author has published, newest first. Use a username (dhh, @dhh) or a profile link.
Publications (publications)	The publication's whole archive, newest first. Use its slug (better-programming), its own domain (python.plainenglish.io) or a link.
Medium links (startUrls)	Any Medium link: a single story (also on a publication's own domain, or an author's friend link), an author profile, a publication, a tag page, a reading list or a search page.

Mix as many as you like, up to 1,000 searches, tags, authors, publications and links per run. Each story appears once per run, even when two inputs find it, and every row says which input found it in source.

How to scrape Medium stories

1. Open the actor's input page.
2. Enter search keywords, tags, authors, publications or Medium links.
3. Optionally set a limit per source, a total limit or a publish-date window.
4. Start the run and download the stories as JSON, CSV, Excel or HTML, or connect the dataset to your own tools.

The input form starts with the keyword artificial intelligence and 100 stories per source, a small first run. When you start runs from your own code, a schedule or an AI agent, any limit you leave out means everything.

Every story an author has published

{
  "authors": [
    "dhh"
  ]
}

Returns all of DHH's stories, 134 at the time of our test run, newest first, with full text.

One month of a tag

{
  "tags": [
    "python"
  ],
  "publishedAfter": "2024-05-01",
  "publishedBefore": "2024-06-01"
}

Every Python story first published in May 2024, about 2,700 of them. Dates are in UTC; publishedAfter includes its day and publishedBefore excludes its day. The window works for searches, tags, authors, publications and lists.

A whole publication, capped

{
  "publications": [
    "better-programming"
  ],
  "maxItemsPerSource": 1000
}

The 1,000 newest stories of Better Programming, which took about 80 seconds in our test run. Remove the cap for the whole archive, more than 15,000 stories.

A keyword search past 1,000 results

{
  "searchQueries": [
    "python"
  ],
  "maxItemsPerSource": 2500
}

2,500 unique stories for one keyword. Medium's own search stops at 1,000; this actor keeps going.

Specific stories, profiles, lists and tag pages

{
  "startUrls": [
    {
      "url": "https://medium.com/p/3bd3267cc151"
    },
    {
      "url": "https://dhh.medium.com/"
    },
    {
      "url": "https://medium.com/tag/rust"
    },
    {
      "url": "https://medium.com/search?q=htmx"
    }
  ],
  "maxItemsPerSource": 20
}

A story link returns that story. A profile, publication, tag, list or search link returns its stories, here capped at 20 each. A reading list returns the stories still live on Medium; removed stories in a list are skipped.

Stories with their responses

{
  "startUrls": [
    {
      "url": "https://medium.com/p/3bd3267cc151"
    }
  ],
  "includeResponses": true,
  "maxResponsesPerPost": 5
}

Adds the first 5 responses (Medium's comments) inside the story's row, replies included, in the order Medium shows readers: each response followed by its replies. Responses cost nothing extra; they make runs slower on stories with many responses.

Metadata only

{
  "tags": [
    "python"
  ],
  "maxItemsPerSource": 1000,
  "includeContent": false
}

Leaves out the story text, images, links, code blocks and embeds. Rows are smaller; everything else stays.

Input options

Parameter	Type	Default	Meaning
searchQueries	list of text	none	Keywords to search stories for, one search each.
tags	list of text	none	Tag slugs or display names; each returns the whole tag archive.
authors	list of text	none	Usernames or profile links; each returns every story by that author.
publications	list of text	none	Publication slugs, own domains or links; each returns the whole archive.
startUrls	list of links	none	Any Medium story, profile, publication, tag, list or search link.
maxItemsPerSource	integer	none (everything)	Stop each search, tag, author, publication or list after this many stories.
maxItems	integer	none (no total limit)	Stop the whole run after this many stories.
publishedAfter	date	none	Only stories first published on or after this day (UTC).
publishedBefore	date	none	Only stories first published before this day (UTC).
includeContent	true/false	true	Story text as plain text, Markdown and HTML, plus images, links, code blocks and embeds.
includeResponses	true/false	false	Each story's responses inside its row.
maxResponsesPerPost	integer	none (all)	With responses on, keep at most this many per story, replies included.

The Input tab has the full description of every option.

What data do you get from Medium?

One row per story, with up to 59 fields:

Group	Fields
Story	postId, title, excerpt, url, shortUrl, canonicalUrl, uniqueSlug, language, license, type, visibility, isLocked, isFullContent, memberOnlySource
Text	contentText, contentMarkdown, contentHtml, images (url, size, alt text, caption, link), links (including mentioned members and jumps within the story), codeBlocks (language, code), embeds (with size or thumbnail), topHighlight (the passage readers highlighted most), highlights (every top highlight)
Engagement	clapCount, voterCount (readers who clapped), responseCount, readingTimeMinutes, wordCount, optional responses
Dates	publishedAt, latestPublishedAt, updatedAt, createdAt, pinnedAt, pinnedByAuthorAt (pinned to the author's profile), boostedAt (when Medium boosted the story), scrapedAt
Author	authorName, authorUsername, authorUrl, author (bio, avatar, followers, following, Medium membership and tier, member since, book author, X username, newsletter)
Publication	publicationName, publication (slug, link, own domain, description, tagline, short description, followers, avatar, logo, X and Facebook pages), featuredInPublications, isFeaturedInPublication
Topics	tags, tagDetails (each tag's story and follower count), topics, series
Images and SEO	imageUrl, image (size and alt text), seo (SEO title and description, meta description, social title and description)
Flags	isPublished, isSeries, isShortform, isNewsletter, allowsResponses, responsesLocked, inResponseToPostId, platform
Source	source: which search, tag, author, publication, list or link found the story

A real row from the Python tag, a free story in a publication with charts and code:

{
  "postId": "7e6e37e25355",
  "title": "The Vehicles Killing More American Pedestrians Are SUVs and Pickups",
  "url": "https://medium.com/python-gis-workflows/the-vehicles-killing-more-american-pedestrians-are-suvs-and-pickups-7e6e37e25355",
  "excerpt": "US pedestrian deaths were 1,292 higher in 2023 than in 2017. Federal crash records show which vehicles account for most of the rise.",
  "authorName": "nunacode",
  "authorUsername": "nunacode",
  "authorUrl": "https://medium.com/@nunacode",
  "publicationName": "Python GIS Workflows",
  "publishedAt": "2026-10-10T15:37:29.194Z",
  "latestPublishedAt": "2026-10-10T15:37:29.194Z",
  "updatedAt": "2026-10-10T16:37:03.288Z",
  "clapCount": 0,
  "voterCount": 0,
  "responseCount": 0,
  "readingTimeMinutes": 4.9,
  "wordCount": 1189,
  "language": "en",
  "isLocked": false,
  "isFullContent": true,
  "tags": [
    "road-safety",
    "transportation",
    "data-journalism",
    "cities",
    "python"
  ],
  "imageUrl": "https://miro.medium.com/v2/1*JOpjs7Jx437HDJK1ql1brA.png",
  "topHighlight": null,
  "highlights": [],
  "type": "story",
  "platform": "medium",
  "shortUrl": "https://medium.com/p/7e6e37e25355",
  "canonicalUrl": "https://medium.com/python-gis-workflows/the-vehicles-killing-more-american-pedestrians-are-suvs-and-pickups-7e6e37e25355",
  "uniqueSlug": "the-vehicles-killing-more-american-pedestrians-are-suvs-and-pickups-7e6e37e25355",
  "visibility": "public",
  "memberOnlySource": null,
  "isPublished": true,
  "isSeries": false,
  "isShortform": false,
  "isNewsletter": false,
  "allowsResponses": true,
  "responsesLocked": false,
  "isFeaturedInPublication": false,
  "inResponseToPostId": null,
  "createdAt": "2026-10-10T15:27:27.899Z",
  "pinnedAt": null,
  "pinnedByAuthorAt": null,
  "boostedAt": null,
  "license": "All rights reserved",
  "author": {
    "id": "f4c89ee3e0f0",
    "name": "nunacode",
    "username": "nunacode",
    "url": "https://medium.com/@nunacode",
    "customDomain": null,
    "bio": null,
    "imageUrl": "https://miro.medium.com/v2/resize:fill:176:176/1*XKaj1cvRuyzKrwkacrXOmA.jpeg",
    "followerCount": 16,
    "followingCount": 20,
    "isMediumMember": false,
    "memberSince": null,
    "membershipTier": null,
    "isBookAuthor": false,
    "twitterUsername": null,
    "newsletterId": "e7f9a3187f6b"
  },
  "publication": {
    "id": "d9726c783628",
    "name": "Python GIS Workflows",
    "slug": "python-gis-workflows",
    "url": "https://medium.com/python-gis-workflows",
    "domain": null,
    "description": "Practical guides for working with spatial data in Python.",
    "tagline": "Practical guides for working with spatial data in Python.",
    "shortDescription": "Practical guides for working with spatial data in Python.",
    "followerCount": 6,
    "avatarUrl": "https://miro.medium.com/v2/1*RXXKOwSPFEzcU-LxFklq2A.png",
    "logoUrl": null,
    "twitterUsername": null,
    "facebookPageName": null,
    "createdAt": null
  },
  "featuredInPublications": [],
  "tagDetails": [
    {
      "slug": "road-safety",
      "name": "Road Safety",
      "storyCount": 1319,
      "followerCount": 28
    },
    {
      "slug": "transportation",
      "name": "Transportation",
      "storyCount": 21841,
      "followerCount": 448128
    },
    {
      "slug": "data-journalism",
      "name": "Data Journalism",
      "storyCount": 2040,
      "followerCount": 855
    },
    {
      "slug": "cities",
      "name": "Cities",
      "storyCount": 16841,
      "followerCount": 205737
    },
    {
      "slug": "python",
      "name": "Python",
      "storyCount": 260251,
      "followerCount": 5041216
    }
  ],
  "topics": [
    {
      "name": "Data Science",
      "slug": "data-science"
    }
  ],
  "image": {
    "id": "1*JOpjs7Jx437HDJK1ql1brA.png",
    "url": "https://miro.medium.com/v2/1*JOpjs7Jx437HDJK1ql1brA.png",
    "width": 1735,
    "height": 900,
    "alt": "Line chart of US pedestrian deaths by striking vehicle, 2017–2023: cars around 2,500–2,950 a year, SUVs rising from 1,223 to 1,686, pickups from 1,017 to 1,270, buses and heavy trucks around 400–550, vans around 300."
  },
  "series": null,
  "seo": {
    "title": "SUVs, Pickups and the Rise in US Pedestrian Deaths",
    "description": "US pedestrian deaths were 1,292 higher in 2023 than in 2017, and 716 of the extra deaths involved SUVs or pickups. What federal FARS crash data shows.",
    "metaDescription": null,
    "socialTitle": null,
    "socialDescription": null
  },
  "source": {
    "type": "tag",
    "value": "python",
    "title": "Python",
    "url": "https://medium.com/tag/python"
  },
  "scrapedAt": "2026-10-10T21:15:24.854Z",
  "contentText": "The Vehicles Killing More American Pedestrians Are SUVs and Pickups\n\nUS pedestrian deaths were 1,292 higher in 2023 than in 2017. Federal crash records show which vehicles account for most of the rise.\n\nPedestrians killed on US roads each year, by the type of vehicle that struck them.",
  "contentMarkdown": "## The Vehicles Killing More American Pedestrians Are SUVs and Pickups\n\n### US pedestrian deaths were 1,292 higher in 2023 than in 2017. Federal crash records show which vehicles account for most of the rise.",
  "contentHtml": "<h2>The Vehicles Killing More American Pedestrians Are SUVs and Pickups</h2>\n<h3>US pedestrian deaths were 1,292 higher in 2023 than in 2017. Federal crash records show which vehicles account for most of the rise.</h3>",
  "images": [
    {
      "url": "https://miro.medium.com/v2/1*JOpjs7Jx437HDJK1ql1brA.png",
      "width": 1735,
      "height": 900,
      "alt": "Line chart of US pedestrian deaths by striking vehicle, 2017–2023: cars around 2,500–2,950 a year, SUVs rising from 1,223 to 1,686, pickups from 1,017 to 1,270, buses and heavy trucks around 400–550, vans around 300.",
      "caption": "Pedestrians killed on US roads each year, by the type of vehicle that struck them.",
      "link": null
    },
    {
      "url": "https://miro.medium.com/v2/1*AnjT_oJslZ9AyD2lknTjNQ.png",
      "width": 1757,
      "height": 830,
      "alt": "Stacked area chart of the share of pedestrian deaths by striking vehicle, where its type is known: SUVs rising from 22.1% to 25.9%, pickups around 18–19%, cars falling from 46.3% to 41.4%; SUVs and pickups together from 40.6% to 45.3%.",
      "caption": "Share of pedestrian deaths by striking vehicle type, among those where the type is known. Buses, trucks, vans and motorcycles are not shown.",
      "link": null
    }
  ],
  "links": [
    {
      "url": "https://www.spatialworkflow.io/fars-fatal-crashes-python/",
      "text": "usual quirks of a long-running federal dataset"
    },
    {
      "url": "https://www.spatialworkflow.io/fars-code-breaks-other-years/",
      "text": "change from year to year"
    }
  ],
  "codeBlocks": [
    {
      "language": "python",
      "code": "import zipfile\nfrom pathlib import Path\n\nimport pandas as pd\nimport requests"
    }
  ],
  "embeds": []
}

To keep this example short, contentText, contentMarkdown and contentHtml are cut to their opening lines, codeBlocks shows the opening lines of the first of its 10 blocks, and links shows 2 of its 4 links. Every key in the row is present, and the dataset holds the full values: this story's contentText is 8,875 characters.

responses and responsesComplete appear only when responses are turned on. series holds the series' id, slug, title, subtitle and description when the story is a Medium series.

Member-only stories

Medium shows every logged-out visitor a preview of a member-only story. This actor returns that preview and marks the row, so you always know whether you have the whole story:

{
  "postId": "a134ec2448ab",
  "visibility": "member-only",
  "memberOnlySource": "author",
  "isLocked": true,
  "isFullContent": false,
  "wordCount": 3368,
  "readingTimeMinutes": 13.1
}

This row's contentText holds the first 160 words of the story's 3,368. All other fields (title, author, claps, tags, publication and the rest) are complete for member-only stories too.

Responses

With includeResponses on, each story's responses come inside its row, each response followed by the replies to it:

{
  "postId": "3bd3267cc151",
  "title": "The AI apocalypse is already here",
  "responseCount": 12,
  "responses": [
    {
      "postId": "cdb8fba36f54",
      "url": "https://medium.com/@amalrikmaia/you-are-free-to-not-use-ai-based-services-your-entire-life-if-you-want-cdb8fba36f54",
      "authorName": "Amalrik Maia",
      "authorUsername": "amalrikmaia",
      "text": "You are free to not use AI based services your entire life if you want.\nYou are already doing a good job exposing those companies that are making bad use of AI.",
      "clapCount": 11,
      "responseCount": 1,
      "publishedAt": "2018-11-28T12:52:37.267Z",
      "isLocked": false,
      "inResponseToPostId": "3bd3267cc151"
    },
    {
      "postId": "e1d04b581a4a",
      "url": "https://medium.com/@tallulahdirac/the-person-being-protected-is-not-the-parent-but-the-babysitter-e1d04b581a4a",
      "authorName": "Tallulah Dirac",
      "authorUsername": "tallulahdirac",
      "text": "The person being protected is not the parent, but the babysitter. Being snooped on against your will (or under pressure from the job market) is not at all analogous to voluntarily taking a (non-addictive) drug.",
      "clapCount": 0,
      "responseCount": 1,
      "publishedAt": "2018-12-27T11:02:28.160Z",
      "isLocked": false,
      "inResponseToPostId": "cdb8fba36f54"
    }
  ],
  "responsesComplete": true
}

Two of this story's 15 responses are shown, and the first response's text is cut to its opening lines: a response, then a reply to it. inResponseToPostId names the story for a response and the response for a reply, so the whole thread can be rebuilt; responseCount on the story counts its direct responses (12), not the replies. maxResponsesPerPost counts replies too. responsesComplete is false when a story's responses could only be partly collected; the responses already collected are kept.

Other fields that appear when a story has them

[
  {
    "postId": "3596e36fe7f9",
    "isFeaturedInPublication": true,
    "featuredInPublications": [
      {
        "id": "2ddc5c035323",
        "name": "Codexa",
        "slug": "codexa"
      }
    ]
  },
  {
    "postId": "f1d04f4f194b",
    "embeds": [
      {
        "type": "link_card",
        "url": "https://basecamp.com/svn",
        "title": "Try Basecamp free for 30 days\nTrusted by millions, Basecamp puts everything you need to get work done in one streamlined, organized place.basecamp.com",
        "caption": null,
        "thumbnailUrl": "https://miro.medium.com/v2/0*hHlzu-IeaP4lBQHM."
      }
    ]
  }
]

featuredInPublications comes from a Python story featured by the Codexa publication; embeds from one of DHH's stories, which ends with a link card. embeds also holds embedded media such as code snippets and videos.

The dataset's Stories, Engagement and Text and media views show the most useful columns. JSON keeps the nested author, publication and text fields intact; CSV and Excel flatten them into columns.

What can you do with Medium data?

• Archive export and backup. Download every story of a tag, author or publication, or your own stories as Markdown for a move to another blog.
• AI and RAG datasets. Clean Markdown and plain text with language, word count, topics and license on every story.
• Content and trend research. Follow a topic month by month with a date window, or compare claps and responses across tags and publications.
• Competitor and publication monitoring. Schedule a run on the publications and authors you watch with publishedAfter set to your last run.
• Author discovery. Find writers by topic with their followers, bio, membership and engagement per story.
• SEO and content gaps. Titles, SEO descriptions, tags with their story and follower counts, and every outbound link.

How much does it cost to scrape Medium?

You pay per story saved to your dataset. The Free plan pays $4.99 per 1,000 stories; Bronze $4.59, Silver $4.19, Gold and above $3.49. Full text, profiles and responses are included in that price; responses never cost extra. Story charges for common runs on the Free plan:

• 100 newest stories of a tag: about $0.50
• Every story by one author with 134 stories: about $0.67
• 1,000 stories: $4.99

Set a maximum cost per run in the run options: the run stops at that amount, keeps every story it saved and says so in the log. A run that finds no stories has no story charges. The Pricing tab shows every charge.

Free Apify plan users can run this actor; there are no extra limits beyond your plan's usage.

FAQ

Why do Medium's RSS feeds show only 10 stories? Medium's feeds list the latest 10 stories of a tag, author or publication, and scrapers that read them inherit that limit. This actor collects the whole archive instead: every story of an author, all of a publication, and every story in a tag back to its oldest month.

Can I download all articles from a Medium author? Yes. Enter the username in authors and leave maxItemsPerSource empty. Every story the author has published comes back, newest first, with full text for free stories.

Can I export Medium articles to Markdown? Yes. Every free story comes as contentMarkdown with headings, lists, links, images and fenced code blocks, plus contentText and contentHtml.

Can I get the full text of member-only stories? No. Member-only stories come with the preview Medium shows every logged-out visitor, marked isFullContent: false and isLocked: true. All other fields are complete. The actor does not get around Medium's paywall.

Do I need a Medium account, login or cookie? No. Nothing to set up beyond your Apify account.

Can I get claps and comments for Medium stories? Yes. Every row has clapCount, voterCount and responseCount. Turn on includeResponses to add the responses themselves (text, author, claps, date) inside each story's row at no extra charge.

How do I get every story in a tag or publication? Enter the tag or publication and leave maxItemsPerSource empty. Big archives hold tens of thousands of stories, so use a date window or a maximum cost per run to collect only what you need. A run stops at its timeout, one hour by default, which covers roughly 40,000 stories; for a bigger archive raise the timeout in the run options or split it into date windows.

Does keyword search stop at 1,000 results? Medium's own search shows at most 1,000 stories per keyword. This actor continues past that; one search for "python" returned 2,500 unique stories.

Can I build a Medium dataset for analysis or AI training? Yes: full text, language, word count, tags, topics and license come with every story. Stories remain their authors' work; check each story's license and Medium's terms before republishing or training on them.

Can I scrape a story on a custom domain or from a friend link? Yes. Paste the link in startUrls: publication domains such as python.plainenglish.io, author domains such as dhh.medium.com and friend links all work.

Can I download Medium articles as PDF? There is no PDF output. contentHtml and contentMarkdown convert to PDF with any document tool.

Why did my run return fewer stories than I expected? Check the log. It names any tag, author, publication or link Medium did not recognize, and says when the run stopped at your maximum cost. Date windows and limits also narrow the results, a reading list skips stories that were removed from Medium, and a story found by two inputs is saved once.

How fresh is the data? Every run collects stories as Medium shows them at that moment, including claps and responses. Schedule the actor to keep a dataset current.

Is it legal to scrape Medium? The actor collects stories Medium shows to any logged-out visitor. How you use them is your responsibility: respect authors' copyright and each story's license, Medium's terms, and data protection law for personal details such as author profiles.

Support

Questions, a missing field or a run that did not return what you expected: open an issue in the Issues tab or email max@mapa.slmail.me. Reports with the run ID get answered fastest. Medium is a trademark of A Medium Corporation. This actor is independent and is not affiliated with or endorsed by Medium.