Medium API

Extract articles, author profiles, and publication data from Medium.com. This scraper can handle various Medium page types including topic pages, author profiles, and individual articles.

What You Can Extract

Article Data

• Title and full content
• Author information (name, username, profile URL)
• Publication details (name, URL, description)
• Publication date and reading time
• Engagement metrics (claps, responses)
• Tags and categories
• Featured images

Author Profiles

• Name and bio
• Follower count
• Profile URL
• Recent articles

Publications

• Name and description
• URL and follower count
• Published articles

Input Parameters

Parameter	Type	Default	Description
startUrls	Array	-	Required - URLs to start scraping from (articles, author profiles, publications, or topic pages)
scrapeType	String	"articles"	Type of content to scrape ("articles", "authors", "publications", "mixed")
maxArticles	Number	100	Maximum number of articles to scrape (0 = unlimited)
includeAuthorInfo	Boolean	true	Whether to scrape detailed author profile information
includePublicationInfo	Boolean	true	Whether to scrape publication details when available
includeComments	Boolean	false	Whether to scrape article comments
maxRequestsPerCrawl	Number	1000	Maximum number of pages to process (0 = unlimited)
proxyConfiguration	Object	{useApifyProxy: true}	Proxy settings for anti-bot protection

Input Example

{
  "startUrls": [
    {
      "url": "https://medium.com/topic/technology"
    },
    {
      "url": "https://medium.com/topic/data-science"
    }
  ],
  "scrapeType": "articles",
  "maxArticles": 10,
  "includeAuthorInfo": true,
  "includePublicationInfo": true,
  "includeComments": false,
  "maxRequestsPerCrawl": 50,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}

Output

The scraper saves results in two locations:

1. Dataset Items

Each scraped item is saved to the dataset:

{
  "url": "https://medium.com/topic/data-science",
  "title": "The most insightful stories about Data Science - Medium",
  "author": {
    "name": "Eivind Kjosbakken",
    "username": "oieivind",
    "profileUrl": "https://medium.com/@oieivind",
    "bio": "",
    "followers": 0
  },
  "publication": {
    "name": "tag",
    "url": "https://medium.com/tag",
    "description": ""
  },
  "content": "Data Science Collective\n\nData Science Collective\n\nEmmanuel O. Irekponor...",
  "excerpt": "Read stories about Data Science on Medium...",
  "publishedAt": "",
  "imageUrl": "https://miro.medium.com/v2/1*0L5w2b6T1yEVI3_ZYUWONw.png",
  "tags": ["Data Science", "Technology", "Programming", "AI"],
  "scrapedAt": "2025-11-08T09:59:25.186Z",
  "pageType": "article",
  "claps": 0,
  "responses": 0,
  "readingTime": 0
}

2. Statistics

Comprehensive statistics are stored for tracking:

{
  "maxArticles": 5,
  "articlesScraped": 1,
  "includeAuthorInfo": true,
  "includePublicationInfo": true,
  "includeComments": false,
  "startTime": "2025-11-08T09:59:03.729Z",
  "completedAt": "2025-11-08T09:59:25.918Z",
  "errors": [],
  "urlsProcessed": ["https://medium.com/topic/data-science"]
}

Usage Examples

Basic Topic Scraping

{
  "startUrls": [{ "url": "https://medium.com/topic/technology" }],
  "maxArticles": 20
}

Author Profile Scraping

{
  "startUrls": [{ "url": "https://medium.com/@username" }],
  "scrapeType": "authors",
  "includeAuthorInfo": true
}

Publication Scraping

{
  "startUrls": [{ "url": "https://medium.com/publication-name" }],
  "scrapeType": "publications",
  "includePublicationInfo": true
}

Mixed Scraping

{
  "startUrls": [
    { "url": "https://medium.com/topic/technology" },
    { "url": "https://medium.com/@username" },
    { "url": "https://medium.com/publication-name" }
  ],
  "scrapeType": "mixed",
  "maxArticles": 50
}

Use Cases

📊 Content Analysis

• Analyze trending topics and publications
• Track author performance and engagement
• Extract publication statistics

🎯 Research & Journalism

• Gather data for media analysis projects
• Monitor content trends across topics
• Research competitor content strategies

🚀 Content Curation

• Build content aggregation systems
• Create recommendation engines
• Monitor specific authors or publications

📈 SEO & Marketing

• Analyze content performance metrics
• Track brand mentions and coverage
• Research keyword trends and topics

How to Use

1. Configure Input: Set your desired URLs and scraping parameters
2. Run the Scraper: The actor will process all provided URLs
3. Access Results: Download scraped data from the dataset
4. Check Statistics: Review comprehensive scraping statistics

Important Notes

• Rate Limiting: Medium may implement rate limiting. Use proxies for large-scale scraping
• Content Access: Some content may require login for full access
• Terms of Service: Ensure compliance with Medium's Terms of Service
• Data Quality: Content extraction depends on page structure and availability