Reddit Scraper

This tool helps you collect information from Reddit. You can use it to search for topics or to read specific posts and their comments.

How to Use It

The scraper has two main modes you can choose from.

Important Note: The scraper does not support running both modes at the same time. If you accidentally provide inputs for both (e.g., both a topic and a list of URLs), it will only execute the mode you specifically selected in the "Mode" dropdown, and the inputs for the other mode will be ignored.

1. Search Mode

Use this when you want to find posts about a specific topic.

• Topic: Type the word or phrase you want to search for (for example, "data analysis").
• Max Items: Choose how many posts you want to collect. The scraper will scroll down the Reddit search page until it finds that many posts.
• Sort: Choose how to order the search results (options: relevance, hot, top, new, comments). Default is relevance.
• Time Filter: Choose the time range for the search results (options: all, year, month, week, day, hour). Default is all.

2. Post Details Mode

Use this when you have specific Reddit posts you want to read.

• Start URLs: Paste the direct links to the Reddit posts you want to scrape. The tool will visit each link and collect the main post text and all the comments inside it.

---

What the Data Looks Like

The scraper will return a list of results. You can download these results as JSON, CSV, or Excel from Apify. Below is a simple explanation of what information you get in each mode.

When using Search Mode

The tool gives you a list of posts it found in the search. For every post, it collects:

• type: Always says "search_result" so you know what kind of data it is.
• postedTime: When the post was created (like "2 days ago").
• authorProfileUrl: The link to the user who wrote the post.
• title: The title of the post.
• description: A short preview of the text inside the post.
• votes: The number of upvotes the post has.
• comments: The number of comments on the post.
• postUrl: The direct link to the post, which you can use in the "Post Details Mode" later if you want to read the comments.

Exact format example:

[
  {
    "type": "search_result",
    "postedTime": "2 days ago",
    "authorProfileUrl": "/user/johndoe",
    "title": "Best data analysis tools in 2024",
    "description": "I am looking for a comprehensive list of...",
    "votes": "150",
    "comments": "42",
    "postUrl": "https://www.reddit.com/r/data/comments/xyz123/"
  },
  {
    "type": "search_result",
    "postedTime": "3 days ago",
    "authorProfileUrl": "/user/janedoe",
    "title": "Learning Python for beginners",
    "description": "What are the best resources to start...",
    "votes": "85",
    "comments": "12",
    "postUrl": "https://www.reddit.com/r/python/comments/abc789/"
  }
]

When using Post Details Mode

The tool gives you a list containing the full post and all of its comments. For every URL you provide, it collects the following attributes:

• type: Always says "post_details".
• url: The link to the post you provided.
• post: A section containing the main details of the post itself. Inside post, you get:
  • subreddit: The name of the community where the post was created.
  • title: The headline or title of the post.
  • author: The username of the person who wrote the post.
  • selftext: The full body text of the post.
  • score: The number of upvotes the post has.
• comments: A list of all the comments people left. For every single comment, you get:
  • author: The username of the person who wrote the comment.
  • body: The actual text of the comment.
  • score: The number of upvotes the comment has.
  • replies: A list of any replies other people made to that specific comment. These replies have the exact same attributes (author, body, score, and their own replies), which allows you to see the entire conversation tree.

Exact format example:

[
  {
    "type": "post_details",
    "url": "https://www.reddit.com/r/data/comments/xyz123/",
    "post": {
      "subreddit": "data",
      "title": "Best data analysis tools in 2024",
      "author": "johndoe",
      "selftext": "I am looking for a comprehensive list of...",
      "score": 150
    },
    "comments": [
      {
        "author": "janedoe",
        "body": "This is a great question. I highly recommend...",
        "score": 50,
        "replies": [
          {
            "author": "data_nerd",
            "body": "I agree with this completely!",
            "score": 12,
            "replies": []
          }
        ]
      }
    ]
  }
]

Need Help?

If you have any questions or notice any problems, please let us know in the "Issues" tab. We will be happy to help!