3 days trial then $10.00/month - No credit card required now
Hacker News Data Scraper
3 days trial then $10.00/month - No credit card required now
Extract Y Combinator's Hacker News based on any search criteria. Crawl the front page, Show HN, Ask HN, news, job listings, and historical data. Get links, titles, comments, ratings, and more!
HN Algolia Search
Closed
I'd like to use HN Algolia search URLs for the starting points, e.g.
Can you make this happen?
tugkan
Hey Joseph Turian!
Thank you very much for using our Public actor and suggesting a nice feature like this. We integrated this support into our scraper right away. If you provide Algolia URL into the Start URLs, it'll work as expected.
Please let us know if you have any questions or concerns. Best
bravura
Will it only search the first algolia results page or all of them?
bravura
I tried with this input and got zero results:
{ "extendOutputFunction": "($) => { return {} }", "maxItems": 50, "proxy": { "useApifyProxy": true }, "startUrls": [ { "url": "https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=false&query=monday.com&sort=byPopularity&type=comment" } ] }
tugkan_epctex
Hey Joseph,
Currently, Hacker News Algolia only supports the "stories" type and not comments. It is on our roadmap but yet not integrated. About the pages, if you don't define "maxItems" or "endPage" attributes, the actor will retrieve all of them.
I hope all the answers find you well. Please let us know if you need anything else. Best
bravura
Thank you. Comments is a key feature for us. We want to search for comments and then retrieve the entire page its on.
beremekdar
Hello Joseph,
Thank you for highlighting the importance of comments for your needs. We understand the significance and are happy to assist you with a tailored project specifically designed to search for comments and retrieve the entire page they are on. If you would like to explore this option further, please let us know, and we'll be more than happy to accommodate your requirements.
Best.
tugkan_epctex
Hey Joseph,
This is Tugkan. I am here to answer all the possible use cases of the Hacker News actor within all the technical details. As Berk mentioned, this functionality is not yet supported on the Algolia-based URLs. It is currently on our roadmap but there is no specific timeline about it. I'll keep you posted on the updates though.
Thank you very much. Best
tugkan_epctex
Hey Mackie,
I just wanted to let you know that we've integrated comment support on the Algolia endpoints. I hope this will work for you.
Thank you very much and sorry for the inconvenience. Best
epctex
