Reddit Keywords Scraper

An Apify Actor built with Python + Playwright to automatically search and scrape Reddit posts by specific keywords.
Perfect for market research, trend analysis, and content discovery — all without relying on the Reddit API.

Features

• 🔍 Search by Keywords: Find Reddit posts matching specific words or phrases
• 🧩 Multiple Keywords: Search multiple keywords in a single run
• 📊 Detailed Post Data: Extract post titles, authors, scores, timestamps, flairs, and more
• ⚙️ Sorting Options: Sort by relevance, hot, top, new, or comments
• 📈 Result Limit Control: Fetch up to 1000 posts per keyword
• 🌐 Browser-Based Crawling: Avoid API limits using Playwright automation
• 💾 Structured JSON Output: Get clean, ready-to-use JSON data
• 🔄 Pagination Support: Automatically loads additional search results

🧠 Use Cases

Use the Reddit Keyword Scraper to:

• 📢 Perform Market Research — Track discussions around your product, brand, or niche
• 💬 Conduct Sentiment Analysis — Collect Reddit data for NLP or public opinion analysis
• 📰 Discover Trending Content — Identify viral or high-performing posts
• 📈 Monitor Trends — Follow emerging topics across subreddits
• 🕵️ Gather Competitive Intelligence — Find mentions of competitors or products
• 🎓 Support Academic Research — Build Reddit-based datasets for studies

Input Parameters

The actor accepts the following input parameters:

Parameter	Type	Required	Default	Description
keywords	array	Yes	-	List of keywords to search for on Reddit
resultLimit	integer	No	25	Maximum number of results to return per keyword (1-1000)
sort	string	No	relevance	Sort method: relevance, hot, top, new, comments

Example Input

{
  "keywords": ["python programming", "machine learning", "web scraping"],
  "resultLimit": 50,
  "sort": "top"
}

Sorting Options

• relevance - Most relevant to the search query (default)
• hot - Currently trending/hot posts
• top - Top-scoring posts of all time
• new - Newest posts first
• comments - Posts with the most comments

📤 Output Data Fields

Each Reddit post result includes:

Field	Description
keyword	The searched keyword that matched this post
post_id	Unique Reddit post ID (e.g., 1ntdzya)
post_name	Full Reddit post identifier (e.g., t3_1ntdzya)
title	Post title
author	Username of the author ([deleted] if removed)
subreddit	Subreddit name
content	Text content for self/text posts (null for link posts)
score	Post score/upvotes
num_comments	Number of comments
url	Direct link to the post
old_reddit_url	Link to the old Reddit version of the post
thumbnail_image	Thumbnail image URL (if available)
link_flair	Post flair text
created_utc	Unix timestamp when the post was created
created_at	ISO 8601 formatted creation date/time
is_stickied	Boolean indicating if the post is pinned
is_nsfw	Boolean indicating if the post is marked NSFW

🧾 Example Output

{
  "keyword": "putin",
  "post_id": "1ntdzya",
  "post_name": "t3_1ntdzya",
  "title": "All hopes of reasoning with Putin are gone - war is coming",
  "author": "theipaper",
  "subreddit": "r/geopolitics",
  "content": null,
  "score": 743,
  "num_comments": 354,
  "url": "https://reddit.com/r/geopolitics/comments/1ntdzya/all_hopes_of_reasoning_with_putin_are_gone_war_is/",
  "old_reddit_url": "https://old.reddit.com/r/geopolitics/comments/1ntdzya/all_hopes_of_reasoning_with_putin_are_gone_war_is/",
  "thumbnail_image": "https://external-preview.redd.it/gBVLJovs7dZbfvD14Q2tZiSlEMt_26iIluaegGWD-mM.jpeg",
  "link_flair": null,
  "created_utc": 1759140117,
  "created_at": "2025-09-29T10:01:57+00:00",
  "is_stickied": false,
  "is_nsfw": false
}

Usage

Local Development

1. Install dependencies:

   pip install -r requirements.txt
   playwright install chromium
   

2. Set up input:

   Create a .actor/INPUT.json file or use environment variables:

{
  "keywords": ["python programming", "machine learning"],
  "resultLimit": 10,
  "sort": "relevance"
}

3. Run locally:

   $apify run
   

Running on Apify Platform

1. Create an Actor on the Apify Platform
2. Upload the code or connect your Git repository
3. Configure the input in the Actor's input tab
4. Run the Actor and view results in the dataset

Example: Searching Multiple Topics

{
  "keywords": [
    "artificial intelligence",
    "climate change",
    "space exploration",
    "cryptocurrency"
  ],
  "resultLimit": 100,
  "sort": "hot"
}

This will search for 100 hot posts about each topic and return a total of up to 400 posts.

Notes

• The actor uses old.reddit.com for better reliability and performance
• Results are limited to publicly accessible posts
• Rate limiting is handled with delays between searches
• Some very old posts may have incomplete data
• The actor respects Reddit's structure and uses browser automation

Technical Details

• Runtime: Python 3.12
• Browser: Chromium (via Playwright)
• HTML Parser: BeautifulSoup4
• Framework: Apify SDK

Limitations

• Maximum result limit per keyword: 1000 posts
• Only returns posts (not comments) from search results
• Requires stable internet connection
• Processing time increases with result limit and number of keywords

Troubleshooting

No Results Found

• Verify the keywords are spelled correctly
• Try more general or popular keywords
• Check if Reddit is accessible in your region
• Try a different sort method

Incomplete Data

• Some older posts may have missing fields
• Deleted posts will show "[deleted]" as author
• Private/removed posts won't appear in results

Related Actors

• Reddit Comment Scraper - Scrape comments from specific Reddit posts
• Reddit Profile Scraper - Scrape posts from specific user profiles
• Reddit Scraper - General Reddit data scraper

Support

For issues, questions, or contributions, please open an issue in the repository.

License

This actor is provided as-is for use on the Apify platform.

undefined