Hacker News Data Scraper avatar
Hacker News Data Scraper

Pricing

$10.00/month + usage

Go to Store
Hacker News Data Scraper

Hacker News Data Scraper

Developed by

epctex

Maintained by Community

Extract Y Combinator's Hacker News based on any search criteria. Crawl the front page, Show HN, Ask HN, news, job listings, and historical data. Get links, titles, comments, ratings, and more!

0.0 (0)

Pricing

$10.00/month + usage

6

Monthly users

7

Runs succeeded

>99%

Last modified

13 hours ago

BR

HN Algolia Search

Closed

bravura opened this issue
2 years ago

I'd like to use HN Algolia search URLs for the starting points, e.g.

https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=false&query=llm&sort=byPopularity&type=story

Can you make this happen?

TU

tugkan

2 years ago

Hey Joseph Turian!

Thank you very much for using our Public actor and suggesting a nice feature like this. We integrated this support into our scraper right away. If you provide Algolia URL into the Start URLs, it'll work as expected.

Please let us know if you have any questions or concerns. Best

BR

bravura

2 years ago

Will it only search the first algolia results page or all of them?

BR

bravura

2 years ago

I tried with this input and got zero results:

{ "extendOutputFunction": "($) => { return {} }", "maxItems": 50, "proxy": { "useApifyProxy": true }, "startUrls": [ { "url": "https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=false&query=monday.com&sort=byPopularity&type=comment" } ] }

TE

tugkan_epctex

2 years ago

Hey Joseph,

Currently, Hacker News Algolia only supports the "stories" type and not comments. It is on our roadmap but yet not integrated. About the pages, if you don't define "maxItems" or "endPage" attributes, the actor will retrieve all of them.

I hope all the answers find you well. Please let us know if you need anything else. Best

BR

bravura

2 years ago

Thank you. Comments is a key feature for us. We want to search for comments and then retrieve the entire page its on.

BE

beremekdar

2 years ago

Hello Joseph,

Thank you for highlighting the importance of comments for your needs. We understand the significance and are happy to assist you with a tailored project specifically designed to search for comments and retrieve the entire page they are on. If you would like to explore this option further, please let us know, and we'll be more than happy to accommodate your requirements.

Best.

TE

tugkan_epctex

2 years ago

Hey Joseph,

This is Tugkan. I am here to answer all the possible use cases of the Hacker News actor within all the technical details. As Berk mentioned, this functionality is not yet supported on the Algolia-based URLs. It is currently on our roadmap but there is no specific timeline about it. I'll keep you posted on the updates though.

Thank you very much. Best

TE

tugkan_epctex

2 years ago

Hey Mackie,

I just wanted to let you know that we've integrated comment support on the Algolia endpoints. I hope this will work for you.

Thank you very much and sorry for the inconvenience. Best

Pricing

Pricing model

Rental 

To use this Actor, you have to pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period. You also pay for the Apify platform usage.

Free trial

3 days

Price

$10.00