Reddit Data Scraper

A Python library to scrape data from Reddit.

Examples

Input:

{
    "limit": 10,
    "query": "python",
    "restrict_sr": "on",
    "result_type": "link",
    "sort": "comments",
    "subreddit_or_username": "webdev",
    "time": "year",
    "type": "subreddit"
}

{
    "limit": 10,
    "query": "python",
    "restrict_sr": "on",
    "result_type": "link",
    "sort": "relevance",
    "subreddit_or_username": "webdev",
    "time": "year",
    "type": "general",
    "include_over_18": "off"
}

Output:

[
  {
    "title": "Sean Cook, founder of the Tea App, only has a 6 month coding bootcamp under his belt.",
    "link": "https://www.reddit.com/r/webdev/comments/1mcx3fl/sean_cook_founder_of_the_tea_app_only_has_a_6/",
    "permalink": "/r/webdev/comments/1mcx3fl/sean_cook_founder_of_the_tea_app_only_has_a_6/",
    "description": "",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1mcx3fl",
    "ups": 4896,
    "downs": 0,
    "upvote_ratio": 0.96,
    "num_comments": 510,
    "num_crossposts": 2,
    "media_type": "image",
    "media_urls": [
      "https://i.redd.it/9td4gnxluxff1.jpeg"
    ],
    "thumbnail": "https://b.thumbs.redditmedia.com/xYbBV-x1dnCzSDLv8W2M6kos6QS_7Koeohj85eFM99c.jpg",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1753849368,
    "author": "Pristine-Elevator198",
    "author_flair_text": null,
    "flair": "News",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "i.redd.it"
  },
  {
    "title": "Massive Failure on the Product",
    "link": "https://www.reddit.com/r/webdev/comments/1iasb06/massive_failure_on_the_product/",
    "permalink": "/r/webdev/comments/1iasb06/massive_failure_on_the_product/",
    "description": "I’ve been working with a team of 4 devs for a year on a major product. Unfortunately, today’s failure was so massive that the product might be discontinued.\n\nDuring the biggest event of the year—a campaign aimed at gaining 20k+ new users—a major backend issue prevented",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1iasb06",
    "ups": 758,
    "downs": 0,
    "upvote_ratio": 0.94,
    "num_comments": 302,
    "num_crossposts": 0,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "self",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1737930204,
    "author": "Yan_LB",
    "author_flair_text": null,
    "flair": "Discussion",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  },
  {
    "title": "Other junior developers are using different IDEs, and it’s causing problems for me. How should I handle this?",
    "link": "https://www.reddit.com/r/webdev/comments/1h26bvs/other_junior_developers_are_using_different_ides/",
    "permalink": "/r/webdev/comments/1h26bvs/other_junior_developers_are_using_different_ides/",
    "description": "We are a group of formerly five developers, all coding in .NET C# with Docker (so YAML files and occasionally some Python and Terraform).\n\nA new junior developer decided to stop using Visual Studio and switched to IntelliJ Rider. Now, after two months, they were tasked",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1h26bvs",
    "ups": 144,
    "downs": 0,
    "upvote_ratio": 0.67,
    "num_comments": 284,
    "num_crossposts": 1,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "self",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1732829685,
    "author": "Abstinence_theonly",
    "author_flair_text": null,
    "flair": null,
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  },
  {
    "title": "I don't understand how they build apps with AI",
    "link": "https://www.reddit.com/r/webdev/comments/1h095wr/i_dont_understand_how_they_build_apps_with_ai/",
    "permalink": "/r/webdev/comments/1h095wr/i_dont_understand_how_they_build_apps_with_ai/",
    "description": "To keep it short. I am modifying an app that uses python flask for backend, and I used SQLAlchemy as an ORM layer to work with my database. I had a model that is already in the db and I needed to add new boolean not nullable field to it. I know how to do it, but decide",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1h095wr",
    "ups": 99,
    "downs": 0,
    "upvote_ratio": 0.79,
    "num_comments": 250,
    "num_crossposts": 1,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "self",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1732618487,
    "author": "NoWeather1702",
    "author_flair_text": null,
    "flair": "Discussion",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  },
  {
    "title": "Site getting around 5000 active users monthly, but I'm still struggling to cover server costs",
    "link": "https://www.reddit.com/r/webdev/comments/1ky5e1d/site_getting_around_5000_active_users_monthly_but/",
    "permalink": "/r/webdev/comments/1ky5e1d/site_getting_around_5000_active_users_monthly_but/",
    "description": "I've been working on a site for the past 2 years. All content is human-written, no AI. It's a micro niche site, a directory of hand-picked open-source web apps.\n\nI got AdSense approval, but the earnings are quite low. I’ve disabled sensitive categories, including 18+ c",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1ky5e1d",
    "ups": 399,
    "downs": 0,
    "upvote_ratio": 0.91,
    "num_comments": 240,
    "num_crossposts": 0,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "https://b.thumbs.redditmedia.com/u45yrweBqitSq5OF4ceCTXUEGU_wdHvdVl6uLKRYCVU.jpg",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1748506619,
    "author": "habib-786",
    "author_flair_text": null,
    "flair": null,
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  },
  {
    "title": "What’s your go-to daily driver browser?",
    "link": "https://www.reddit.com/r/webdev/comments/1hcrprv/whats_your_goto_daily_driver_browser/",
    "permalink": "/r/webdev/comments/1hcrprv/whats_your_goto_daily_driver_browser/",
    "description": "Looking to cut Chrome the RAM destroyer out of my life other than as a x-browser compatibility tool\n\nI’m learning web dev stacks that aren’t Python based so one would imagine that I’ve got a metric shit-ton of tabs open (and I do, much more so than when I’m deving stuf",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1hcrprv",
    "ups": 60,
    "downs": 0,
    "upvote_ratio": 0.73,
    "num_comments": 239,
    "num_crossposts": 0,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "self",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1734028124,
    "author": "husky_whisperer",
    "author_flair_text": null,
    "flair": "Question",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  },
  {
    "title": "I learned to code in prison, then built a Reddit user profile analyzer with modern data visualization",
    "link": "https://www.reddit.com/r/webdev/comments/1if58p7/i_learned_to_code_in_prison_then_built_a_reddit/",
    "permalink": "/r/webdev/comments/1if58p7/i_learned_to_code_in_prison_then_built_a_reddit/",
    "description": "",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1if58p7",
    "ups": 5641,
    "downs": 0,
    "upvote_ratio": 0.97,
    "num_comments": 239,
    "num_crossposts": 1,
    "media_type": "text",
    "media_urls": [
      "https://preview.redd.it/yqf3yaaktige1.png?width=1278&format=png&auto=webp&s=f97412184b6a40fbc69003d343878ae4ff2b3241",
      "https://preview.redd.it/vo6on4rktige1.png?width=1278&format=png&auto=webp&s=ed6a2b9015028c4430e966e99344e301c1f7b733",
      "https://preview.redd.it/yw1viu4ltige1.png?width=1278&format=png&auto=webp&s=82de30524e49fc2454cb704dc25bf5905949614d",
      "https://preview.redd.it/e2ykvmiltige1.png?width=1278&format=png&auto=webp&s=46d66c2bd8e7fef72f12665738c3b991704f6e4d"
    ],
    "thumbnail": "https://b.thumbs.redditmedia.com/iuqZYrByyecaqCzTymMnCZY55YQFcmZJcuLQYYkjx6U.jpg",
    "is_video": false,
    "is_gallery": true,
    "created_utc": 1738412756,
    "author": "MemoryEmptyAgain",
    "author_flair_text": null,
    "flair": "Showoff Saturday",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "reddit.com"
  },
  {
    "title": "isThisTechDead.com : A satirical but data-driven tool to tell you if your stack is dead",
    "link": "https://www.reddit.com/r/webdev/comments/1k89nwj/isthistechdeadcom_a_satirical_but_datadriven_tool/",
    "permalink": "/r/webdev/comments/1k89nwj/isthistechdeadcom_a_satirical_but_datadriven_tool/",
    "description": "**Project**: [IsThisTechDead.com](http://www.isthistechdead.com)   \n  \nA tongue-in-cheek tracker that assigns every language / framework a “Deaditude Score” (0-100 % dead).\n\nThe tone is very satirical so please don't get offended if your favorite framework is dead (it ",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1k89nwj",
    "ups": 868,
    "downs": 0,
    "upvote_ratio": 0.94,
    "num_comments": 238,
    "num_crossposts": 0,
    "media_type": "text",
    "media_urls": [
      "https://preview.redd.it/0ksnra6yl5xe1.png?width=1320&format=png&auto=webp&s=40ecabc5df9c8fe3b776d872baed32e935197e2e",
      "https://preview.redd.it/6gcaw3yzl5xe1.png?width=1320&format=png&auto=webp&s=7c144402547c8bc2e61e05b289096d9f2c36b4db",
      "https://preview.redd.it/71y6jxztl5xe1.png?width=1687&format=png&auto=webp&s=7cdf65f6c23a3b7e1b00158932b0014907566e4f",
      "https://preview.redd.it/6p9b1eqam5xe1.png?width=1031&format=png&auto=webp&s=c519d2fe097aadb552da9d0f28dacc73bb4bcb99"
    ],
    "thumbnail": "https://b.thumbs.redditmedia.com/aq4M5kJjPnCOY9rebYbItAUozdFvdeWNtcyie4GJN1Y.jpg",
    "is_video": false,
    "is_gallery": true,
    "created_utc": 1745662415,
    "author": "jobehi",
    "author_flair_text": null,
    "flair": "Showoff Saturday",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "reddit.com"
  },
  {
    "title": "Is .NET doomed to boring corporate jobs?",
    "link": "https://www.reddit.com/r/webdev/comments/1ildjuw/is_net_doomed_to_boring_corporate_jobs/",
    "permalink": "/r/webdev/comments/1ildjuw/is_net_doomed_to_boring_corporate_jobs/",
    "description": "I've been working as a .NET developer for the last three years, and recently, I've been looking for a new job. But the more I search, the more it feels like every .NET job out there is at some bland consulting company doing enterprise-y projects for banks, insurance fi",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1ildjuw",
    "ups": 280,
    "downs": 0,
    "upvote_ratio": 0.91,
    "num_comments": 237,
    "num_crossposts": 0,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "self",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1739103655,
    "author": "Sad_Mobile3941",
    "author_flair_text": null,
    "flair": null,
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  },
  {
    "title": "AI is getting shittier day after day",
    "link": "https://www.reddit.com/r/webdev/comments/1i3rgbv/ai_is_getting_shittier_day_after_day/",
    "permalink": "/r/webdev/comments/1i3rgbv/ai_is_getting_shittier_day_after_day/",
    "description": "/rant\n\nI've been using GitHub Copilot since its release, mainly on FastAPI (Python) and NextJS. I've also been using ChatGPT along with it for some code snippets, as everyone does.\n\n  \nAt first it was meh, and it got good after getting a little bit of context from my p",
    "subreddit": "r/webdev",
    "subreddit_id": "t5_2qs0q",
    "subreddit_subscribers": 3108891,
    "post_id": "1i3rgbv",
    "ups": 751,
    "downs": 0,
    "upvote_ratio": 0.9,
    "num_comments": 206,
    "num_crossposts": 0,
    "media_type": "text",
    "media_urls": [],
    "thumbnail": "self",
    "is_video": false,
    "is_gallery": false,
    "created_utc": 1737149867,
    "author": "Krigrim",
    "author_flair_text": null,
    "flair": "Discussion",
    "is_original_content": false,
    "awards_count": 0,
    "over_18": false,
    "is_spoiler": false,
    "is_locked": false,
    "is_stickied": false,
    "content_categories": null,
    "allow_live_comments": false,
    "domain": "self.webdev"
  }
]