# Medium Scraper Cheapest - Profiles & History (`ahmed_jasarevic/medium-scraper`) Actor

Extract writer profiles, article listings, reading history, and topic/tag discovery from Medium.com via RSS feeds and GraphQL API.

- **URL**: https://apify.com/ahmed\_jasarevic/medium-scraper.md
- **Developed by:** [Ahmed Jasarevic](https://apify.com/ahmed_jasarevic) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.99 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Medium Scraper

Extract **writer profiles**, **article listings**, **reading history**, and **topic/tag discovery** from Medium.com. Built for content marketers, thought-leadership researchers, AI training-data teams, and publication scouts.

[![Try on Apify](https://apify.com/actor-badge.svg)](https://console.apify.com/actors/medium-scraper)

### What does Medium Scraper do?

Medium Scraper collects structured data from Medium using **RSS feeds** (for article listings) and **Cheerio scraping of public pages** — reading each page's embedded JSON (`?format=json`) with an HTML fallback (for detailed profiles and article metadata). It extracts:

- **Writer profiles**: name, username, bio, follower/following count, profile image, verification status, join date, post count, top tags, publications
- **Auto-discovered publication feeds**: when a user owns a publication (e.g. `@dailynewsletter` owns *The Medium Blog*), its article feed is pulled automatically for more results
- **Full user history**: `fetchUserHistory` paginates a user's complete article timeline (up to `maxItemsPerUserHistory`, default 10,000) with claps, responses, reading time and tags — RSS only ever lists ~10
- **Article listings**: title, description, URL, author, publication, tags, clap count, response count, reading time, publish/update dates
- **Topic/tag discovery**: tag names, slugs, post counts from tag feeds
- **Publication feeds**: articles from specific publications

All data is delivered as clean JSON/CSV/Excel via Apify dataset.

### Why use Medium Scraper?

| Use Case | Value |
|----------|-------|
| **Content marketers** | Discover trending topics, top writers, and competitive content in your niche |
| **Thought-leadership researchers** | Track influential voices, their publishing frequency, and engagement metrics |
| **AI training-data teams** | Harvest high-quality, structured article corpora with metadata for LLM fine-tuning |
| **Publication scouts** | Identify rising authors, analyze publication performance, find syndication partners |

### How to use Medium Scraper

1. **Add input** – Specify usernames, tags, and/or publications to scrape
2. **Configure limits** – Set max items per feed and total request cap
3. **Enable details** – Toggle user profiles and/or full article details (requires Apify Proxy)
4. **Run** – Click Start and wait for results
5. **Export** – Download data as JSON, CSV, Excel, or hook into your pipeline via API

### Input

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `usernames` | string\[] | No | Medium usernames (without `@`) to scrape profiles and articles from. Example: `["medium", "towardsdatascience"]` |
| `tags` | string\[] | No | Medium tags to scrape articles from. Example: `["technology", "artificial-intelligence", "programming"]` |
| `publications` | string\[] | No | Medium publication slugs. Example: `["towardsdatascience", "betterhumans", "uxdesign"]` |
| `maxItemsPerFeed` | integer | No (default: 50) | Max articles to fetch per RSS feed (1–1000) |
| `maxItemsPerUserHistory` | integer | No (default: 10000) | Max articles to fetch from each user's full timeline stream (1–100000). Set high (e.g. 10000) to pull essentially every post a user has published |
| `fetchUserProfiles` | boolean | No (default: true) | Fetch detailed profiles (bio, followers, post count, top tags) from the page's embedded JSON, with HTML fallback |
| `fetchUserHistory` | boolean | No (default: true) | Paginate each user's **full article history** (up to `maxItemsPerUserHistory`) via the timeline stream, instead of only the ~10 latest from RSS |
| `fetchArticleDetails` | boolean | No (default: false) | Fetch full article metadata (clap count, responses, reading time) by scraping the public article page |
| `proxyConfiguration` | object | No | Proxy settings (recommended for profile/article pages to avoid IP-based blocks). The history stream and embedded JSON fetch via direct connection to avoid Medium's datacenter-proxy blocking |

**At least one of `usernames`, `tags`, or `publications` is required.**

#### Example Input

```json
{
  "usernames": ["medium", "towardsdatascience"],
  "tags": ["technology", "artificial-intelligence", "programming"],
  "publications": ["betterhumans", "uxdesign"],
  "maxItemsPerFeed": 30,
  "fetchUserProfiles": true,
  "fetchUserHistory": true,
  "fetchArticleDetails": false,
  "maxRequestsPerCrawl": 300,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["BUYPROXIES94952"]
  }
}
```

### Output

Results are stored in the **default dataset**. Each item has a `type` field: `profile`, `article`, or `tag`.

#### User Profile (`type: "profile"`)

```json
{
  "type": "profile",
  "username": "towardsdatascience",
  "name": "Towards Data Science",
  "bio": "Your home for data science. A Medium publication sharing concepts, ideas, and codes.",
  "followerCount": 1250000,
  "followingCount": 42,
  "profileImageUrl": "https://cdn-images-1.medium.com/fit/c/200/200/1*abc123.png",
  "userId": "1234567890abcdef",
  "createdAt": "2016-03-15T10:30:00.000Z",
  "isVerified": true,
  "numberOfPostsPublished": 5230,
  "mediumMemberSince": "2016-03-15T10:30:00.000Z",
  "twitterScreenName": "TDataScience",
  "topTags": [{ "slug": "data-science", "name": "Data Science" }],
  "publications": [{ "slug": "towardsdatascience", "name": "Towards Data Science" }],
  "sourceFeed": "user:@towardsdatascience"
}
```

#### Article (`type: "article"`)

```json
{
  "type": "article",
  "title": "The Difference Between Python That Works and Python That Lasts",
  "description": "Python that works is easy to write but hard to maintain...",
  "url": "https://blog.stackademic.com/the-difference-between-python-that-works-and-python-that-lasts-e1796054bda6",
  "author": "Abdur Rahman",
  "authorUsername": "abdur-rahman",
  "authorProfileUrl": "https://medium.com/@abdur-rahman",
  "publication": "Stackademic",
  "publicationUrl": "https://blog.stackademic.com",
  "tags": ["technology", "python", "data-science", "artificial-intelligence", "programming"],
  "clapCount": 1240,
  "responseCount": 42,
  "readingTimeMinutes": 8,
  "publishedAt": "2026-09-01T12:36:26.000Z",
  "updatedAt": "2026-09-01T12:36:26.000Z",
  "postId": "e1796054bda6",
  "sourceFeed": "tag:technology",
  "sourceFeedType": "tag"
}
```

#### Tag (`type: "tag"`)

```json
{
  "type": "tag",
  "tagName": "Technology",
  "tagSlug": "technology",
  "tagPostCount": 2500000,
  "sourceFeed": "tag:technology"
}
```

### Data table

| Field | Profile | Article | Tag | Description |
|-------|---------|---------|-----|-------------|
| `type` | ✓ | ✓ | ✓ | Item type: `profile`, `article`, or `tag` |
| `username` | ✓ | | | Writer username |
| `name` | ✓ | | | Writer display name |
| `bio` | ✓ | | | Writer bio |
| `followerCount` | ✓ | | | Number of followers |
| `followingCount` | ✓ | | | Number of users followed |
| `profileImageUrl` | ✓ | | | Profile avatar URL |
| `userId` | ✓ | | | Medium internal user ID |
| `createdAt` | ✓ | | | Account creation date |
| `isVerified` | ✓ | | | Verified author badge |
| `numberOfPostsPublished` | ✓ | | | Total posts published |
| `mediumMemberSince` | ✓ | | | Membership start date |
| `twitterScreenName` | ✓ | | | Twitter/X handle |
| `topTags` | ✓ | | | Top author topic tags |
| `publications` | ✓ | | | Publications owned by the user |
| `title` | | ✓ | | Article title |
| `description` | | ✓ | | Article snippet/description |
| `url` | | ✓ | | Canonical article URL |
| `author` | | ✓ | | Author display name |
| `authorUsername` | | ✓ | | Author username |
| `authorProfileUrl` | | ✓ | | Link to author profile |
| `publication` | | ✓ | | Publication name |
| `publicationUrl` | | ✓ | | Publication homepage |
| `tags` | | ✓ | | Article tags/categories |
| `clapCount` | | ✓ | | Number of claps (requires `fetchArticleDetails`) |
| `responseCount` | | ✓ | | Number of responses (requires `fetchArticleDetails`) |
| `readingTimeMinutes` | | ✓ | | Estimated reading time (requires `fetchArticleDetails`) |
| `publishedAt` | | ✓ | | First publication date |
| `updatedAt` | | ✓ | | Last update date |
| `postId` | | ✓ | | Medium internal post ID |
| `tagName` | | | ✓ | Tag display name |
| `tagSlug` | | | ✓ | Tag URL slug |
| `tagPostCount` | | | ✓ | Total posts in tag |
| `sourceFeed` | ✓ | ✓ | ✓ | Origin feed identifier |
| `sourceFeedType` | | ✓ | | `user`, `tag`, or `publication` |

### Pricing / Cost estimation

| Tier | Cost | Limits |
|------|------|--------|
| **Free (Apify Free plan)** | $0 | 20 items total per run |
| **Paid (Apify Starter+)** | $49/mo+ | Unlimited (compute units apply) |

**Compute unit estimates** (approximate):

- RSS feed requests: ~0.01 CU per feed (very cheap)
- Profile page requests: ~0.05 CU per profile
- Article page requests: ~0.1 CU per article

A typical run scraping 10 users × 30 articles + 5 tags × 30 articles + 3 publications × 30 articles = ~540 articles + 10 profiles ≈ **0.5–1 CU** without article details, **~5–10 CU** with full article details.

> **Tip**: Disable `fetchArticleDetails` for large-scale tag/publication scraping. Enable only for targeted deep-dives.

### Tips & Advanced options

- **Proxy recommended for article pages**: `fetchArticleDetails` scrapes Medium's public article HTML, which can block datacenter IPs. Use Apify Proxy group (`BUYPROXIES94952`).
- **Profiles are largely block-resistant**: Profiles are read from the page's embedded JSON (`?format=json`); if that gets blocked the actor automatically retries the plain HTML page, and falls back gracefully instead of failing the run.
- **RSS feeds are free & fast**: Article listings from RSS work without proxy and cost almost nothing.
- **Combine feeds**: Mix usernames, tags, and publications in one run for comprehensive coverage.
- **Deduplicate**: Use `postId` field to deduplicate articles appearing in multiple feeds.
- **Schedule**: Set up daily/weekly runs on Apify to track trends over time.

### FAQ, disclaimers & support

#### Is scraping Medium legal?

This Actor accesses publicly available RSS feeds and public HTML pages. Always review Medium's [Terms of Service](https://policy.medium.com/medium-terms-of-service-9db0094a1e0f) and [Robots.txt](https://medium.com/robots.txt). Use responsibly and respect rate limits.

#### Unknown yet

- `clapCount`, `responseCount`, `readingTimeMinutes` are populated from the timeline stream (with `fetchUserHistory`) or from article-page scraping (with `fetchArticleDetails`)
- `followerCount`, `bio`, `profileImageUrl` require `fetchUserProfiles: true` (profile embedded JSON)
- RSS feeds only provide basic article metadata

#### Known limitations

- Reading history (personal) requires authentication — not supported
- Private/unlisted articles not accessible
- Publication feeds may 404 if publication uses custom domain
- Public page structure (HTML/embedded JSON) may change without notice

#### Support

- Found a bug? [Open an issue](https://console.apify.com/actors/medium-scraper/issues)
- Need custom extraction? [Contact Apify](https://apify.com/contact) for enterprise solutions

***

**Built with** [Crawlee](https://crawlee.dev) + [Apify SDK](https://docs.apify.com/sdk/js/) • **Strategy**: RSS feeds (Cheerio) → embedded page JSON + timeline stream pagination + HTML scraping (Cheerio) • **No browser automation, no internal GraphQL API**

# Actor input Schema

## `usernames` (type: `array`):

Medium usernames (without @) to scrape profiles and articles from. Example: \['medium', 'towardsdatascience', 'username']

## `tags` (type: `array`):

Medium tags to scrape articles from. Example: \['technology', 'artificial-intelligence', 'programming']

## `publications` (type: `array`):

Medium publication slugs to scrape articles from. Example: \['towardsdatascience', 'betterhumans', 'uxdesign']

## `maxItemsPerFeed` (type: `integer`):

Maximum number of articles to fetch per RSS feed (user, tag, or publication). Note: Medium always caps RSS feeds at ~10 items, so this mostly matters for tags and publications.

## `maxItemsPerUserHistory` (type: `integer`):

Maximum number of articles to fetch from each user's FULL article history (paginated timeline stream). Set high (e.g. 10000) to pull every post a user has ever published; the crawl is still bounded by maxRequestsPerCrawl.

## `fetchUserProfiles` (type: `boolean`):

Whether to fetch detailed user profiles from the page's embedded JSON (bio, follower count, profile image, top tags, publications).

## `fetchUserHistory` (type: `boolean`):

Whether to paginate each user's complete article history via the timeline stream (every post they've published), instead of only the ~10 newest from the RSS feed. Bounded by maxItemsPerUserHistory and maxRequestsPerCrawl. Does not use GraphQL.

## `fetchArticleDetails` (type: `boolean`):

Whether to fetch full article metadata (clap count, response count, reading time, tags) by scraping the public article page.

## `maxRequestsPerCrawl` (type: `integer`):

Hard limit on total requests to prevent runaway crawls.

## `proxyConfiguration` (type: `object`):

Proxy settings for accessing Medium. Recommended for user profiles and article details to avoid IP-based blocks.

## Actor input object example

```json
{
  "usernames": [],
  "tags": [],
  "publications": [],
  "maxItemsPerFeed": 50,
  "maxItemsPerUserHistory": 10000,
  "fetchUserProfiles": true,
  "fetchUserHistory": true,
  "fetchArticleDetails": false,
  "maxRequestsPerCrawl": 500,
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": [
      "BUYPROXIES94952"
    ]
  }
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `profiles` (type: `string`):

No description

## `articles` (type: `string`):

No description

## `tags` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("ahmed_jasarevic/medium-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("ahmed_jasarevic/medium-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call ahmed_jasarevic/medium-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ahmed_jasarevic/medium-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/m3nh66B4diD5Yammt/builds/MR5zNfLW9goKV5FQX/openapi.json