3 days trial then $10.00/month - No credit card required now

Hacker News Data Scraper

epctex/hackernews-scraper

3 days trial then $10.00/month - No credit card required now

Extract Y Combinator's Hacker News based on any search criteria. Crawl the front page, Show HN, Ask HN, news, job listings, and historical data. Get links, titles, comments, ratings, and more!

Back to issues Create new issue

We got blocked

Closed

bravura opened this issue

What is the best way to avoid this error:

2023-06-20T08:12:38.109Z ERROR BasicCrawler: Request failed and reached maximum retries. Error: We got blocked 2023-06-20T08:12:38.111Z at BasicCrawler.requestHandler (file:///home/myuser/src/main.js:49:19) 2023-06-20T08:12:38.113Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2023-06-20T08:12:38.116Z at async BasicCrawler._runRequestHandler (/home/myuser/node_modules/@crawlee/basic/internals/basic-crawler.js:572:9) 2023-06-20T08:12:38.118Z at async wrap (/home/myuser/node_modules/@apify/timeout/index.js:52:21) {"id":"7VnKzRXCQ8psNG9","url":"https://news.ycombinator.com/item?id=33178562","method":"GET","uniqueKey":"https://news.ycombinator.com/item?id=33178562"} 2

beremekdar

Hello,

Thank you very much for reaching out. Can you please share your Run ID so that we can investigate what went wrong there? Then we can immediately reach out to you about the solutions.

Best

bravura

See this issue: https://console.apify.com/actors/jv1G7XhUz1R34fV2L/issues/De5KmFdg84cg9GMKU/comments

Add comment

Developer

epctex

Actor Metrics

5 monthly users
5 stars
>99% runs succeeded
Created in Mar 2021
Modified 4 hours ago

Categories

News

Other

Hacker News Data Scraper & Activity Monitoring

lucen_data/hacker-news-data-scraper-activity-monitoring

Unofficial Hacker News API to extract data from all Hacker News categories. You can filter posts based on specific keywords, track trends using the unique activity monitoring feature, and receive updates directly in your Slack channel.

Lucen

Hacker News Top Sites Scraper

fearless_sharpener/hacker-news-top-sites-scraper

Unlock even more valuable insights from the popular news website, Hacker News. Our updated Top Sites Scraper extracts titles, scores, links and more, all in one easy-to-use Apify actor. With this powerful tool, you can gather comprehensive data from Hacker News and gain a competitive edge.

Tom

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

59.1k

504

Google Maps Extractor

compass/google-maps-extractor

Extract data from hundreds of places fast. Scrape Google Maps by keyword, category, location, URLs & other filters. Get addresses, contact info, opening hours, popular times, prices, menus & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

17.8k

343

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

15k

190

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.

Vojta Drmota

24.7k

194

Google Maps Scraper

compass/crawler-google-places

Extract data from hundreds of Google Maps locations and businesses. Get Google Maps data including reviews, images, contact info, opening hours, location, popular times, prices & more. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Compass

78.9k

560

Traffic Generator (Youtube, Web, Etsy, Behance and many more!)

epctex/traffic-generator

Maximize your website's performance and visibility with our Traffic Generator. Drive targeted traffic, simulate page views, and stress-test against potential threats. With the power to generate millions of visits, it's the ultimate solution for boosting your online presence.

epctex

24.4k

172

Facebook Groups Scraper

apify/facebook-groups-scraper

Extract data from one or multiple public Facebook groups. Get group and post URLs, post text, comments, timestamp, likes and comments count, and basic commentator info. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

7.2k