Substack Notes Scraper 🔍

Extract valuable content and engagement data from Substack Notes search results. This actor helps you gather insights about discussions, trends, and user engagement around specific topics on Substack's social platform.

🎯 Use Cases

• Monitor discussions around specific keywords or topics
• Track user engagement and content performance
• Analyze visual content trends through attached images
• Research user behavior and community interactions
• Gather social proof and testimonials
• Track competitor activities and engagement

✨ Features

• 🔎 Search-based data extraction
• 👤 Detailed user information (name, handle, bio)
• 📊 Engagement metrics (reactions, restacks, comments)
• 🖼️ Image attachments with dimensions
• ⏱️ Timestamp data for temporal analysis
• 🔄 Automatic pagination handling
• 🚀 Stealth mode to avoid detection

🔢 Output

The actor provides detailed JSON output including:

• User profile information
• Note content and metadata
• Engagement metrics
• Image attachments
• Timestamps
• Search context

💡 Input Parameters

• keywords (Array): List of keywords to search for
• maxItems (Number): Maximum number of items to scrape
• proxyConfiguration (Object): Proxy settings (optional)

📌 Limitations

• Respects Substack's rate limits and terms of service
• Search results are limited to public notes
• Image downloads require additional processing

💪 Tips for Best Results

1. Use specific keywords for targeted results
2. Adjust maxItems based on your needs
3. Consider using proxies for large-scale scraping
4. Allow sufficient run time for complete data collection

Input Example

A full explanation of an input example in JSON.

1{
2    "keywords": [
3        "sale"
4    ],
5    "maxItems": 50
6}

Output sample

The results will be wrapped into a dataset which you can always find in the Storage tab. Here's an excerpt from the data you'd get if you apply the input parameters above:

And here is the same data but in JSON. You can choose in which format to download your data: JSON, JSONL, Excel spreadsheet, HTML table, CSV, or XML.

1[
2    {
3        "keyword": "sale",
4        "entityKey": "c-91995525",
5        "type": "comment",
6        "user": {
7            "id": 310653644,
8            "name": "Chris Sale",
9            "handle": "chrissalephoto",
10            "photoUrl": "https://substack-post-media.s3.amazonaws.com/public/images/b5673547-b3db-4089-9e43-f32170dba61b_96x96.jpeg",
11            "bio": "Landscape photographer based in Cumbria. Sharing stories from the Eden Valley and the North Pennines. "
12        },
13        "comment": {
14            "id": 91995525,
15            "body": "Bothy | Melmerby Fell, North Pennines AONB | February 2025",
16            "date": "2025-02-08T14:19:40.888Z",
17            "reactionCount": 60,
18            "reactions": {
19                "❤": 60
20            },
21            "restacks": 1,
22            "childrenCount": 5
23        },
24        "attachments": [
25            {
26                "id": "5a9aef05-696f-4bb5-a9bf-45807bcaf2da",
27                "type": "image",
28                "imageUrl": "https://substack-post-media.s3.amazonaws.com/public/images/8bc060ec-f0c6-43a4-b4b8-6ca80b54cec9_2160x2700.jpeg",
29                "imageWidth": 2160,
30                "imageHeight": 2700
31            }
32        ],
33        "scrapedAt": "2025-02-10T05:59:15.575Z"
34    },
35    ...
36]