Substack Publications Scraper

This scraper extracts the latest articles from up to 3 Substack publications in a single run, including detailed author social media profiles.

📊 What You Get

Extract comprehensive article data including:

• Article titles and full content
• Article previews and images
• Author information and profiles
• Author social media links (YouTube, Instagram, X/Twitter, LinkedIn, Facebook, Bluesky)
• Publication dates
• Engagement metrics (likes, comments)
• Direct article links

🎯 Supported Social Platforms

Automatically extracts and categorizes author social links:

• YouTube - Video content and channels
• Instagram - Visual content and stories
• X/Twitter - Real-time updates and thoughts
• LinkedIn - Professional networking
• Facebook - Community engagement
• Bluesky - Decentralized social networking
• Other Links - Any additional social or professional links

📈 Output Example

Here's what you can expect from each scraped article:

{
  "title": "The Future of AI in Content Creation",
  "link": "https://newsletter.theaiedge.io/p/future-ai-content-creation",
  "preview": "Exploring how artificial intelligence is revolutionizing the way we create and consume content...",
  "image_preview": "https://substackcdn.com/image/fetch/w_1456,c_limit/f_webp,q_auto:good/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F...",
  "likes": "127",
  "comments": "23",
  "date": "2025-09-24T10:30:00Z",
  "content": "The landscape of content creation is rapidly evolving with the integration of artificial intelligence...",
  "author": "Sarah Chen",
  "author_link": "https://newsletter.theaiedge.io/about",
  "author_youtube_link": "https://youtube.com/c/sarahchenai",
  "author_instagram_link": null,
  "author_x_link": "https://x.com/sarahchen_ai",
  "author_bluesky_link": null,
  "author_linkedin_link": "https://linkedin.com/in/sarahchen",
  "author_facebook_link": null,
  "author_other_links": ["https://sarahchen.ai", "https://github.com/sarahchen"]
}

🔧 Input Example

Configure your scraper with these parameters:

{
  "list_of_publications": [
    "https://newsletter.theaiedge.io/",
    "https://luizasangalli.substack.com/",
    "https://ketoretotips.substack.com/"
  ]
}

Parameters:

• list_of_publications: Array of Substack publication URLs (maximum 3)
  • Supports both *.substack.com URLs and custom domains
  • Each URL should be the main publication homepage

⚡ Quick Start

1. Add publication URLs - Include up to 3 Substack publications you want to scrape
2. Run the scraper - The scraper will extract the latest articles (up to 24) and author profiles
3. Get social insights - Access comprehensive author social media data
4. Download your data - Export as JSON, CSV, or other formats