Reddit Search Scraper: Extract Reddit Data by Keywords

Overview

The Reddit Search Scraper is a powerful tool designed to extract detailed post data from Reddit using keyword-based searches. Whether you're conducting sentiment analysis, monitoring trends, or gathering insights for research, this scraper enables efficient and accurate data collection across all of Reddit using targeted keywords.

Features

• Comprehensive Data Extraction: Retrieve essential details such as:

  • Post title
  • Author
  • Creation timestamp
  • Number of comments
  • Score (upvotes)
  • Permalink
  • Image and thumbnail URLs (if available)
  • Post body content
  • Comments

• Flexible Input Parameters: Customize your scraping with options like:

  • Search keyword or phrase
  • Post limit (1-1000)
  • Sort method (relevance, hot, top, new, comments)
  • Time filter (hour, day, week, month, year, all)

• Structured Output: Export the collected data in JSON format for seamless integration with analytical tools.

Usage

Input Configuration

The scraper accepts the following input parameters:

1. Search Keyword:

   • Specify the keyword or phrase to search for across Reddit.
   • Example:

     {
       "keyword": "artificial intelligence",
       "limit": 25,
       "sort": "relevance",
       "time_filter": "week"
     }
     

2. Limit:

   • Set the number of posts to scrape (minimum: 1, maximum: 1000, default: 25).

3. Sort:

   • Choose from "relevance", "hot", "top", "new", or "comments".

4. Time Filter:

   • Specify a time range for filtering posts (hour, day, week, month, year, all).

Output Data

The scraper collects the following information for each post:

• Title
• Author
• Created UTC timestamp
• Number of comments
• Score (upvotes/downvotes)
• Permalink (direct link to the post)
• Subreddit name
• Post URL
• Image URL (if available)
• Thumbnail URL (if available)
• Post body text (selftext)
• Search keyword used
• Detailed comments with nested replies

The output is stored in JSON format, allowing easy data processing.

Quick Start

1. Install the scraper from the Apify Marketplace.
2. Configure the input parameters in the input schema.
3. Run the scraper and download the output in JSON format.

Example Use Cases

• Keyword Trend Monitoring: Track discussions about specific topics, products, or events across all of Reddit.
• Brand Monitoring: Monitor mentions of your brand, products, or competitors across Reddit communities.
• Sentiment Analysis: Analyze user sentiment about specific topics based on post and comment content.
• Research & Data Collection: Gather comprehensive data about specific subjects for academic or market research.
• Content Discovery: Find relevant content and discussions related to your interests or business domain.

Output Storage

The scraper stores results in a structured dataset, allowing easy access to:

• Title
• URL
• Post metadata
• Comments
• Images (if applicable)

Output Example

{
  "title": "The Impact of Artificial Intelligence on Modern Healthcare",
  "author": "healthtech_researcher",
  "permalink": "/r/MachineLearning/comments/1abc234/the_impact_of_artificial_intelligence_on_modern/",
  "score": 847,
  "num_comments": 156,
  "created_utc": 1704067200,
  "subreddit": "MachineLearning",
  "url": "https://www.reddit.com/r/MachineLearning/comments/1abc234/the_impact_of_artificial_intelligence_on_modern/",
  "selftext": "Recent advances in AI have revolutionized diagnostic imaging, drug discovery, and personalized treatment protocols. This comprehensive analysis explores the current state and future potential of AI applications in healthcare settings.",
  "keyword": "artificial intelligence",
  "image_url": "https://preview.redd.it/ai_healthcare_chart_abc123.png?width=960&crop=smart&auto=webp&s=1234567890abcdef",
  "thumbnail_url": "https://b.thumbs.redditmedia.com/ai_healthcare_thumb_abc123.jpg",
  "comments": [
    {
      "author": "medical_ai_expert",
      "body": "This is an excellent overview. I've been working in medical AI for 5 years and can confirm the transformative impact on radiology workflows. The accuracy improvements in detecting early-stage cancers are remarkable.",
      "score": 234,
      "replies": [
        {
          "author": "curious_student",
          "body": "Could you elaborate on specific accuracy improvements? I'm particularly interested in mammography screening applications.",
          "score": 67,
          "replies": [
            {
              "author": "medical_ai_expert",
              "body": "Certainly! Recent studies show AI-assisted mammography reduces false positives by 40% while maintaining 99.2% sensitivity. The Google DeepMind collaboration with Cancer Research UK demonstrated particularly promising results.",
              "score": 89,
              "replies": []
            }
          ]
        },
        {
          "author": "radiologist_practitioner",
          "body": "As a practicing radiologist, I can attest to these improvements. Our department implemented AI screening tools last year and saw a 25% reduction in reading time while improving diagnostic confidence.",
          "score": 156,
          "replies": []
        }
      ]
    },
    {
      "author": "pharma_researcher",
      "body": "The drug discovery section resonates strongly with my experience. AI has compressed our initial compound screening from 18 months to 6 months. The cost savings alone justify the technology investment.",
      "score": 178,
      "replies": [
        {
          "author": "biotech_startup",
          "body": "Which AI platforms are you using for compound screening? We're evaluating options for our pipeline.",
          "score": 43,
          "replies": [
            {
              "author": "pharma_researcher",
              "body": "We primarily use Atomwise for small molecule discovery and BenevolentAI for target identification. Both have shown excellent ROI in our trials.",
              "score": 52,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "author": "ethics_healthcare",
      "body": "While the technological advances are impressive, we must carefully consider the ethical implications of AI decision-making in healthcare. Patient consent, algorithmic bias, and the human element in medical care are crucial considerations.",
      "score": 298,
      "replies": [
        {
          "author": "medical_ethicist",
          "body": "Absolutely critical points. The FDA's recent guidance on AI/ML-based medical devices addresses some concerns, but we need more comprehensive frameworks for ethical AI deployment in clinical settings.",
          "score": 124,
          "replies": []
        },
        {
          "author": "patient_advocate",
          "body": "As someone who has benefited from AI-assisted diagnosis, I appreciate both the technology and the emphasis on ethical considerations. Patients should always understand how AI influences their care decisions.",
          "score": 97,
          "replies": []
        }
      ]
    }
  ]
}

Explore More Actors

✨ Looking for additional solutions? Check out more actors on Apify that can help with your web automation and data extraction needs. Discover a wide range of tools tailored for different scenarios at 🌐 Explore Vulnv's Actors on Apify.

📧 Need higher volume, custom fields, or a managed data pipeline? Email us at apify@vulnv.com — we build bespoke scrapers and managed feeds for teams. Questions and support welcome too.