3 days trial then $10.00/month - No credit card required now

Hacker News Data Scraper

epctex/hackernews-scraper

3 days trial then $10.00/month - No credit card required now

Extract Y Combinator's Hacker News based on any search criteria. Crawl the front page, Show HN, Ask HN, news, job listings, and historical data. Get links, titles, comments, ratings, and more!

Back to issues Create new issue

Does it actually scrape the pages?

Closed

bravura opened this issue

I'd like a structured scrape of the HN pages, with the top text as a field, and all comments (with commentator, date, and text) in a hierarchical way.

beremekdar

Hello Joseph,

Thank you for your request to scrape HN pages in a structured manner, including the top text and hierarchical comments. However, the current public actor is not designed to handle this specific case. If you are interested, we can create a custom project tailored to your requirements. Please let us know if you would like to proceed with a custom solution, and we will be glad to assist you further.

Best.

bravura

Berk, what information is this actor supposed to handle?

tugkan_epctex

Hey Joseph,

This is Tugkan. I am here to answer all the possible use cases of the Hacker News actor within all the technical details. As in the description, Hacker News actor is designed to retrieve post items (story), jobs, and all the possible information inside them which also includes comments. However, the generic use of this actor does not support the hierarchical comments. Let me talk with the Engineering Team if this functionality can be integrated by a small feature flag and let you know about it. However, as Berk mentioned, the most robust solution that fulfills all your needs would be possibly a custom integration. Before thinking about that option, I'll talk with the team and let you know about this request.

Thank you for your patience. Best

tugkan_epctex

Hey Mackie,

I just wanted to let you know that we've integrated a new field called "enableCommentHierarchy". If you enable this property, it will retrieve all the comments and build a hierarchy tree on top of it. I hope this will work for you.

Thank you very much and sorry for the inconvenience. Best

Add comment

Developer

epctex

Actor Metrics

5 monthly users
5 stars
>99% runs succeeded
Created in Mar 2021
Modified a day ago

Categories

News

Other

Hacker News Data Scraper & Activity Monitoring

lucen_data/hacker-news-data-scraper-activity-monitoring

Unofficial Hacker News API to extract data from all Hacker News categories. You can filter posts based on specific keywords, track trends using the unique activity monitoring feature, and receive updates directly in your Slack channel.

Lucen

Hacker News Top Sites Scraper

fearless_sharpener/hacker-news-top-sites-scraper

Unlock even more valuable insights from the popular news website, Hacker News. Our updated Top Sites Scraper extracts titles, scores, links and more, all in one easy-to-use Apify actor. With this powerful tool, you can gather comprehensive data from Hacker News and gain a competitive edge.

Tom

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

59.1k

504

Google Maps Extractor

compass/google-maps-extractor

Extract data from hundreds of places fast. Scrape Google Maps by keyword, category, location, URLs & other filters. Get addresses, contact info, opening hours, popular times, prices, menus & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

17.8k

343

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

15k

190

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.

Vojta Drmota

24.7k

194

Google Maps Scraper

compass/crawler-google-places

Extract data from hundreds of Google Maps locations and businesses. Get Google Maps data including reviews, images, contact info, opening hours, location, popular times, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

78.9k

560

Traffic Generator (Youtube, Web, Etsy, Behance and many more!)

epctex/traffic-generator

Maximize your website's performance and visibility with our Traffic Generator. Drive targeted traffic, simulate page views, and stress-test against potential threats. With the power to generate millions of visits, it's the ultimate solution for boosting your online presence.

epctex

24.4k

172

Facebook Groups Scraper

apify/facebook-groups-scraper

Extract data from one or multiple public Facebook groups. Get group and post URLs, post text, comments, timestamp, likes and comments count, and basic commentator info. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

7.2k