# Medium Articles Scraper: Tag, Publication & Author Feeds (`arman-bd/medium-articles-scraper`) Actor

Scrape Medium's public RSS feeds: title, author, publish date, tags, canonical URL and article text. Follow a tag, a publication or a specific writer.

- **URL**: https://apify.com/arman-bd/medium-articles-scraper.md
- **Developed by:** [Arman Hossain](https://apify.com/arman-bd) (community)
- **Categories:** News, Social media, Marketing
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.56 / 1,000 article scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Medium Articles Scraper: Tag, Publication & Author Feeds

![Medium Articles Scraper: Articles by tag, publication or author, title, author, publish date, tags and full text when the feed carries it](https://api.apify.com/v2/key-value-stores/ZQOcNAOHrIgTacAmy/records/medium-articles-scraper.jpg)

Reads Medium's public RSS feeds by tag, by publication or by author, and returns one structured record per article. You get the title, author, publish date, tag list, canonical URL, lead image and the article body where the feed carries it.

Medium has no public API, but it does publish RSS for exactly this purpose, and that is what this Actor reads. No browser, no proxy, no login.

Read the limits before you build on it. Medium's feeds return 10 items each with no pagination, and only some of them include the article body. Both constraints are Medium's, not this Actor's, and both are reported per record so you always know what you got. See [Notes](#notes).

**Agent skill: [SKILL.md](https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/medium-articles-scraper.md)**

```
https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/medium-articles-scraper.md
```

### What you get

| Field | What it holds |
|---|---|
| `sourceType`, `sourceName` | `tag`, `publication` or `author`, and the tag, slug or handle it came from |
| `feedTitle`, `feedUrl` | Feed's own title and the exact URL that was fetched |
| `articleId` | Medium's post ID, parsed from the guid. `null` for publications on their own domain |
| `title`, `author` | Article title and the `dc:creator` byline |
| `publishedAt`, `updatedAt` | Publication and last-update timestamps, ISO 8601 |
| `url` | Canonical article URL with Medium's `?source=rss-.` attribution parameter stripped |
| `categories` | Every tag on the article |
| `contentPlain` | Article text with HTML stripped and entities decoded. Full body or one-line teaser, depending on the feed |
| `isFullContent` | `true` when the feed supplied a real body, `false` when `contentPlain` is only a teaser |
| `wordCount`, `readingTimeMinutes` | Both `null` unless `isFullContent` is `true`. Reading time is computed at 200 words per minute, not taken from Medium |
| `imageUrl` | First image in the article, with Medium's view-tracking pixel filtered out |
| `scrapedAt` | Run timestamp |

`RUN_SUMMARY` in the key-value store holds the feeds that were read (`feedsRequested`), the filters you used, and anything that did not go to plan by name: `failures` for feeds that could not be fetched, `entriesRejected` for entries that could never address a feed, `feedsSkipped` for feeds left unread because `maxItems` was already spent, and `duplicatesSkipped` for articles a previous feed had already delivered.

### Use cases

- **Topic monitoring.** Poll `machine-learning` and `programming` hourly and diff on `articleId` to catch new posts.
- **Writer tracking.** Follow a list of authors in your niche, since author feeds are the ones that reliably carry full text.
- **Content-gap analysis.** Collect titles and tag combinations across ten tags to see what is already saturated.
- **Newsletter sourcing.** A de-duplicated, tagged candidate list with lead images ready for a digest layout. An article that appears in six of your tag feeds is returned once.
- **Competitive publication watch.** Track what a rival publication ships, when, and by whom.

### Quick start

One tag, defaults for everything else:

```json
{
 "tags": ["programming"]
}
```

Tags, a publication and an author together, titles filtered:

```json
{
 "tags": ["machine-learning", "data-science"],
 "publications": ["better-programming"],
 "authors": ["@dhh"],
 "searchTerms": ["llm", "rust"],
 "maxItemsPerFeed": 10,
 "includeContent": true
}
```

A metadata-only sweep for change detection:

```json
{
 "tags": ["programming", "javascript", "python", "devops"],
 "includeContent": false,
 "maxItems": 20
}
```

### Input

| Field | Type | Default | Notes |
|---|---|---|---|
| `tags` | array | `[]` | Tag slugs such as `machine-learning`. Reads `medium.com/feed/tag/<tag>`. Teaser text only. |
| `publications` | array | `[]` | Publication slugs such as `better-programming`. A publication on its own domain works too, so paste the site or `/feed` URL. |
| `authors` | array | `[]` | Handles with or without `@`. Reads `medium.com/feed/@<handle>`. |
| `maxItemsPerFeed` | integer | `10` | Cap per feed, counted after de-duplication. Medium serves 10, so higher values change nothing on `medium.com` feeds. 0 means everything the feed returns. |
| `maxItems` | integer | `0` | Ceiling on the articles saved across the whole run. Remaining feeds are skipped once it is reached, and named in `RUN_SUMMARY.feedsSkipped`. 0 means no total cap. |
| `includeContent` | boolean | `true` | Include `contentPlain` and `wordCount`. Turn it off for a much smaller dataset. |
| `searchTerms` | array | `[]` | Keep only titles containing one of these terms, case-insensitive. Empty means all. |

None of the three source fields is individually required, but at least one of `tags`, `publications` or `authors` must be set. The Actor stops with that message if all three are empty.

Both caps are validated before anything is fetched. A negative, fractional or non-numeric value is refused with a message naming the field, rather than being read as "no limit" — an unset field sent as `null` gets the documented default, not the whole feed.

One feed is read once. Entries that resolve to the same feed collapse before any fetch, whether they repeat across the three lists or are just the same publication written several ways, so you are never charged twice for the same article.

### Output example

A real record from a publication feed, with `contentPlain` trimmed:

```json
{
 "sourceType": "publication",
 "sourceName": "better-programming",
 "feedTitle": "Better Programming - Medium",
 "feedUrl": "https://medium.com/feed/better-programming",
 "articleId": "bf37baef8f27",
 "title": "Let a thousand programming publications bloom.",
 "author": "Tony Stubblebine",
 "publishedAt": "2023-11-10T18:18:10.000Z",
 "updatedAt": "2023-11-27T17:53:37.557Z",
 "url": "https://medium.com/better-programming/let-a-thousand-programming-publications-bloom-bf37baef8f27",
 "categories": [],
 "contentPlain": "I’m putting Better Programming on hiatus to make room for other programming publications. …",
 "isFullContent": true,
 "wordCount": 1305,
 "readingTimeMinutes": 7,
 "imageUrl": "https://cdn-images-1.medium.com/max/1024/1*tKikPWjE4MZ5WgdfZuyvEg.png",
 "scrapedAt": "2026-08-06T11:58:36.656Z"
}
```

And a real record from a tag feed. Same keys, teaser instead of body:

```json
{
 "sourceType": "tag",
 "sourceName": "programming",
 "title": "6 Best AI Tools for Software Engineers in 2026 (Compared)",
 "author": "Shahid Bahadur",
 "url": "https://shahidyousafxai.medium.com/6-best-ai-tools-for-software-engineers-in-2026-compared-ddb8992886d2",
 "categories": ["web-development", "programming", "artificial-intelligence", "productivity", "software-engineering"],
 "contentPlain": "AI has become a daily part of software development. Whether you’re debugging code, refacto",
 "isFullContent": false,
 "wordCount": null,
 "readingTimeMinutes": null,
 "imageUrl": "https://cdn-images-1.medium.com/max/1536/1*gGldf-aWBPpS11vTV4p9Pg.png"
}
```

Every record carries the same keys. Only the values change.

### Naming a feed

| You want | Write | Feed fetched |
|---|---|---|
| A tag | `programming` in `tags` | `https://medium.com/feed/tag/programming` |
| A publication | `better-programming` in `publications` | `https://medium.com/feed/better-programming` |
| A publication on its own domain | `https://towardsdatascience.com/feed` in `publications` | `https://towardsdatascience.com/feed` |
| An author | `@dhh` or `dhh` in `authors` | `https://medium.com/feed/@dhh` |

Tag slugs are lower-case and hyphenated: `machine-learning`, not `Machine Learning`. Get one from the URL of any tag page on Medium. Non-English tags such as `programación` work as written.

Publications on their own domain often left Medium and now run WordPress or Ghost. Their feeds still parse, because the Actor handles standard RSS rather than only Medium's flavour, but `articleId` will be `null` and the field mix can differ.

### API example

```bash
curl -X POST "https://api.apify.com/v2/acts/arman-bd~medium-articles-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "tags": ["programming", "machine-learning"],
 "authors": ["@dhh"],
 "maxItemsPerFeed": 10
 }'
```

### JavaScript example

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/medium-articles-scraper').call({
 tags: ['machine-learning'],
 publications: ['better-programming'],
 includeContent: true,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const a of items) {
 console.log(`${a.title} by ${a.author} (${a.publishedAt})${a.isFullContent ? ` [${a.readingTimeMinutes} min]` : ' [teaser only]'}`);
}
```

### Notes

- Every feed returns 10 items with no pagination. Medium's RSS has no page, offset or limit parameter, and `?limit=25` is ignored. To build an archive, schedule the Actor and dedupe on `articleId`, because there is no way to reach back through history in one run.
- Full text is not guaranteed, and `isFullContent` tells you which you got. Tag feeds never include `<content:encoded>`, so `contentPlain` is the one-line teaser and `wordCount` is `null`. Author feeds reliably include it. Publication feeds are inconsistent, and in testing one item in a publication feed carried the full body while the other nine did not. Nothing is fabricated to paper over the gap.
- Reading time is computed, not scraped. Medium's own badge is not in the feed. `readingTimeMinutes` is `ceil(wordCount / 200)`, and it is `null` whenever there is no real body to count.
- Feed furniture is removed. The trailing "Continue reading on … »" line is stripped from teasers, and the `medium.com/_/stat` view-tracking pixel is never returned as `imageUrl`.
- URLs are canonicalised. Medium appends `?source=rss-.` to every link, and that is stripped so the same article produces the same `url` across feeds.
- A bad handle is a 404, recorded in `RUN_SUMMARY.failures` while other feeds carry on. A tag that does not exist returns an empty feed instead, and the log line shows `0 listed`.
- An entry that cannot address a feed at all — `https://`, a slug with spaces in it — is rejected before the run starts, listed in `RUN_SUMMARY.entriesRejected` with the reason, and the remaining feeds are read normally. It never ends the run.
- The same article is returned once per run. Medium files a post under every tag it carries, so overlapping tag feeds repeat heavily; repeats are dropped before they reach the dataset and counted in `RUN_SUMMARY.duplicatesSkipped`. Identity is the Medium post ID, or the canonical URL for feeds that have no post ID.
- One publication is one feed. Scheme, `www.`, host case, a trailing slash and a missing `/feed` suffix all collapse to the same feed URL, so `https://towardsdatascience.com`, `http://www.TowardsDataScience.com/feed/` and `towardsdatascience.com` are read once between them. A non-default port is kept, because that is a different endpoint.
- Requests are spaced 400 ms apart and identify themselves as `apify-medium-articles-scraper/1.0`.
- Public data only. No authentication, no member-only content, no paywall bypass.

### FAQ

**Does it get paywalled member-only stories?** No. The feed gives what Medium chooses to publish there. Member-only posts appear as metadata plus a teaser, and this Actor does not attempt to get past that.

**Can I get more than 10 articles from a tag?** Not in one run. Medium caps its feeds at 10 with no pagination. Run on a schedule and accumulate, since an hourly run on a busy tag captures most of what is posted.

**Why is `contentPlain` one sentence for tag feeds?** Because that is all the tag feed contains. Use author or publication feeds when you need the body, and check `isFullContent`.

**Is there a JSON API I should use instead?** No public one. Medium's internal GraphQL endpoint is undocumented, unauthenticated access is not offered, and it changes without notice. RSS is the supported public surface, which is why this Actor uses it.

**Do claps or responses come through?** No. Neither is in the feed, and nothing is inferred from anywhere else.

**Can I plug it into something else?** Yes. Apify API, the client libraries, webhooks, scheduled runs, dataset exports to JSON, CSV or Excel, or MCP. The output is structured JSON.

# Actor input Schema

## `tags` (type: `array`):

Medium tag slugs to follow, e.g. 'programming' or 'machine-learning'. Reads https://medium.com/feed/tag/<tag>. Tag feeds return titles, authors and tags but no article body.

## `publications` (type: `array`):

Publication slugs, e.g. 'better-programming'. A publication on its own domain works too: paste the site URL or its /feed URL. Scheme, 'www.' and letter case are ignored, so one publication written several ways is fetched and charged once. Publication feeds include the full article body.

## `authors` (type: `array`):

Author handles, with or without the leading @. Reads https://medium.com/feed/@<handle>. Author feeds include the full article body.

## `maxItemsPerFeed` (type: `integer`):

Cap articles saved per feed, counted after de-duplication. Medium serves at most 10 items per feed and offers no pagination, so values above 10 have no effect on medium.com feeds. Set 0 for everything the feed returns. A negative, fractional or non-numeric value is refused before anything is fetched.

## `maxItems` (type: `integer`):

Hard ceiling on the articles saved across the whole run, counted after de-duplication. Once it is reached the remaining feeds are skipped and named in RUN\_SUMMARY. Set 0 for no total cap, leaving only the per-feed cap.

## `includeContent` (type: `boolean`):

Include the article text as plain text plus its word count. Turn off for a much smaller dataset of titles and metadata only.

## `searchTerms` (type: `array`):

Keep only articles whose title contains at least one of these terms (case-insensitive). Leave empty to keep every article.

## Actor input object example

```json
{
  "tags": [
    "programming",
    "machine-learning"
  ],
  "publications": [
    "better-programming",
    "https://towardsdatascience.com/feed"
  ],
  "authors": [
    "@dhh",
    "medium"
  ],
  "maxItemsPerFeed": 10,
  "maxItems": 0,
  "includeContent": true,
  "searchTerms": [
    "rust",
    "kubernetes"
  ]
}
```

# Actor output Schema

## `items` (type: `string`):

Every record the run produced.

## `runsummary` (type: `string`):

The RUN\_SUMMARY record from the run's key-value store.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tags": [
        "programming"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arman-bd/medium-articles-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "tags": ["programming"] }

# Run the Actor and wait for it to finish
run = client.actor("arman-bd/medium-articles-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tags": [
    "programming"
  ]
}' |
apify call arman-bd/medium-articles-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arman-bd/medium-articles-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/LMOrKuhQnNuAJAyW0/builds/LBH8cGENISWEq9BoT/openapi.json