# Medium Scraper - Articles, Claps, Authors, Tags & Publications (`wakachobi/medium-scraper`) Actor

Scrape Medium articles by tag, author, publication or URL: title, author, date, claps, responses, reading time, tags and full public text.

- **URL**: https://apify.com/wakachobi/medium-scraper.md
- **Developed by:** [Cobi Hilliard](https://apify.com/wakachobi) (community)
- **Categories:** News, AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$3.00 / 1,000 articles

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Medium Scraper: articles, claps, responses, authors and full text

Scrape Medium articles by **tag**, **author**, **publication** or **direct URL**. For every article you get the
title, author, publication, publish date, tags and the public text, plus **clap count**, **number of
responses**, reading time and word count whenever Medium serves the article page. No login and no API key.

> Medium often refuses article pages to automated visitors. Those articles are still saved from Medium's RSS
> feed, with title, author, date, tags and text, but without claps and responses (`fromFeed: true`). You pay
> the same per article either way.

### What data you get

- Title, subtitle, URL and article ID
- Author name, username, profile link and follower count
- Publication name and slug
- Published and last-updated dates (ISO 8601)
- Claps, responses, reading time (minutes) and word count
- Tags, language and cover image
- Full article text as plain text. Member-only articles give the public preview, flagged with `textIsPreviewOnly`
- Which input (tag, author, publication or URL) found the article
- `fromFeed`: `true` when the article page could not be loaded and the item was built from the RSS feed instead.
  These items have the title, author, date, tags and text (full text when the feed carries it, otherwise a
  snippet, flagged with `textIsPreviewOnly`), but `claps` and `responses` are `null`

#### Sample result

```json
{
  "id": "a64152b37c35",
  "url": "https://karpathy.medium.com/software-2-0-a64152b37c35",
  "title": "Software 2.0",
  "authorName": "Andrej Karpathy",
  "authorUsername": "karpathy",
  "publishedAt": "2017-11-11T22:18:53.751Z",
  "claps": 61030,
  "responses": 194,
  "readingTimeMinutes": 8.8,
  "tags": ["machine-learning", "artificial-intelligence", "programming", "software-development", "future"],
  "isMemberOnly": false,
  "text": "Software 2.0\n\nI sometimes see people refer to neural networks as just “another tool in your machine learning toolbox”…",
  "source": "author:karpathy"
}
```

### How to use

| Field | What it does |
|---|---|
| `tags` | Latest articles for each tag, e.g. `artificial-intelligence`, `startup`, `javascript` |
| `authors` | Latest articles by each author, e.g. `karpathy` or `@karpathy` |
| `publications` | Latest articles in each publication, e.g. `towards-data-science`, `better-programming` |
| `articleUrls` | Specific articles you already have links for |
| `maxItems` | Stop after this many articles (default 50) |
| `includeFullText` | Turn off to get metadata and stats only |

Example inputs:

```json
{ "tags": ["artificial-intelligence", "startup"], "maxItems": 40 }
```

```json
{ "authors": ["karpathy"], "publications": ["towards-data-science"], "includeFullText": false }
```

```json
{ "articleUrls": [{ "url": "https://karpathy.medium.com/software-2-0-a64152b37c35" }] }
```

Medium's tag, author and publication feeds list each source's **most recent articles** (usually the latest 10 to
25\). For larger sets, add more tags, authors or publications, or pass article URLs directly.

### Pricing

**$3 per 1,000 articles**, charged only for articles saved to your dataset. 200 articles cost about $0.60.

### Use cases

- Track what is being written about your brand, product or market
- Collect articles for AI training sets, RAG pipelines and summarisation
- Find popular writers in a niche by claps and follower counts
- Monitor publications and competitors for new posts
- Content research: which topics and headlines get the most claps and responses

### FAQ

**Is it legal?** The scraper only reads publicly available pages, the same ones any visitor sees without logging
in. It does not get around Medium's paywall: member-only articles return the public preview only. You are
responsible for how you use the data, including personal data and copyright.

**Why did I get fewer articles than `maxItems`?** Feeds only list recent articles. Add more sources or pass URLs.

**Why do some articles have `fromFeed: true` and no claps?** Medium sometimes refuses to serve an article page
(HTTP 403), especially on author subdomains like `karpathy.medium.com`. When the article came from a tag, author or
publication feed, the scraper still saves it using the feed data, so you do not lose it. An article you passed
directly in `articleUrls` has no feed data to fall back on, so if its page is refused it is skipped with a
warning. Using Apify Proxy usually reduces these refusals.

**Can I run it on a schedule?** Yes. Use Apify Schedules to run it daily and get new articles from your tags or
authors, then send them to Google Sheets, Slack, Make, Zapier or a webhook with Apify integrations.

### API

Call it from any language with the Apify API or client libraries (`apify-client` for JavaScript and Python), or
connect it to an AI agent through the Apify MCP server.

# Actor input Schema

## `tags` (type: `array`):

Medium tags, e.g. 'artificial-intelligence' or 'javascript'. Returns the latest articles for each tag.

## `authors` (type: `array`):

Medium usernames (with or without @), e.g. 'karpathy'.

## `publications` (type: `array`):

Publication slugs, e.g. 'towards-data-science' or 'better-programming'.

## `articleUrls` (type: `array`):

Direct links to Medium articles (any medium.com or custom-domain Medium URL).

## `maxItems` (type: `integer`):

Stop after this many articles in total.

## `includeFullText` (type: `boolean`):

Include the article body as plain text. Member-only articles return the public preview only.

## `proxyConfiguration` (type: `object`):

Datacenter proxy works for most runs.

## Actor input object example

```json
{
  "tags": [
    "artificial-intelligence"
  ],
  "maxItems": 20,
  "includeFullText": true,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "tags": [
        "artificial-intelligence"
    ],
    "maxItems": 20,
    "proxyConfiguration": {
        "useApifyProxy": true
    }
};

// Run the Actor and wait for it to finish
const run = await client.actor("wakachobi/medium-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "tags": ["artificial-intelligence"],
    "maxItems": 20,
    "proxyConfiguration": { "useApifyProxy": True },
}

# Run the Actor and wait for it to finish
run = client.actor("wakachobi/medium-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "tags": [
    "artificial-intelligence"
  ],
  "maxItems": 20,
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}' |
apify call wakachobi/medium-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,wakachobi/medium-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RLNWZdoqdPwSwJs4Z/builds/ybpl2wl2mkj4TJF7f/openapi.json