3 days trial then $10.00/month - No credit card required now

Hacker News Data Scraper

epctex/hackernews-scraper

3 days trial then $10.00/month - No credit card required now

Extract Y Combinator's Hacker News based on any search criteria. Crawl the front page, Show HN, Ask HN, news, job listings, and historical data. Get links, titles, comments, ratings, and more!

Back to issues Create new issue

We got blocked

Closed

bravura opened this issue

What is the best way to avoid this error:

2023-06-20T08:12:38.109Z ERROR BasicCrawler: Request failed and reached maximum retries. Error: We got blocked 2023-06-20T08:12:38.111Z at BasicCrawler.requestHandler (file:///home/myuser/src/main.js:49:19) 2023-06-20T08:12:38.113Z at process.processTicksAndRejections (node:internal/process/task_queues:95:5) 2023-06-20T08:12:38.116Z at async BasicCrawler._runRequestHandler (/home/myuser/node_modules/@crawlee/basic/internals/basic-crawler.js:572:9) 2023-06-20T08:12:38.118Z at async wrap (/home/myuser/node_modules/@apify/timeout/index.js:52:21) {"id":"7VnKzRXCQ8psNG9","url":"https://news.ycombinator.com/item?id=33178562","method":"GET","uniqueKey":"https://news.ycombinator.com/item?id=33178562"} 2

beremekdar

Hello,

Thank you very much for reaching out. Can you please share your Run ID so that we can investigate what went wrong there? Then we can immediately reach out to you about the solutions.

Best

bravura

See this issue: https://console.apify.com/actors/jv1G7XhUz1R34fV2L/issues/De5KmFdg84cg9GMKU/comments

Add comment

Developer

epctex

Actor Metrics

4 monthly users
5 stars
>99% runs succeeded
Created in Mar 2021
Modified 4 hours ago

Categories

News

Other

Hacker News Data Scraper & Activity Monitoring

lucen_data/hacker-news-data-scraper-activity-monitoring

Unofficial Hacker News API to extract data from all Hacker News categories. You can filter posts based on specific keywords, track trends using the unique activity monitoring feature, and receive updates directly in your Slack channel.

Lucen

Hacker News Top Sites Scraper

fearless_sharpener/hacker-news-top-sites-scraper

Unlock even more valuable insights from the popular news website, Hacker News. Our updated Top Sites Scraper extracts titles, scores, links and more, all in one easy-to-use Apify actor. With this powerful tool, you can gather comprehensive data from Hacker News and gain a competitive edge.

Tom

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

63.9k

656

🏯 Tiktok Profile Scraper (Pay Per Result)

apidojo/tiktok-profile-scraper

Looking for Lead Generation? Intelligence over any TikTok user, or just keep track of any person? The TikTok Profile Scraper is ready for your consumption. This actor is known as the most customized yet stable and effective solution for extracting Profile data from TikTok.com.

API Dojo

274

🏯 Tiktok Scraper (Pay Per Result)

apidojo/tiktok-scraper

The most complete and effective solution for extracting data from TikTok.com. User, Profile, Hashtag, Music, Location, Tag, and many other support is just the beginning. The intelligent algorithm allows TikTok Scraper to retrieve any posts/videos around the platform in the blink of an eye!

API Dojo

768

Linkedin profile scraper

curious_coder/linkedin-profile-scraper

Scrape linkedin profile urls and get complete profile details including basic info, position history, skills, company details, bio, location, etc

Curious Coder

1.9k

📩📍 Google Maps Email Extractor

lukaskrivka/google-maps-with-contact-details

Extract Google Maps contact details. Scrape websites of Google Maps places for contact details and get email addresses, website, location, address, zipcode, phone number, social media links. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Lukáš Křivka

9.3k

258

Facebook Posts Scraper

apify/facebook-posts-scraper

Extract data from hundreds of Facebook posts from one or multiple Facebook pages and profiles. Get post URL, post text, page or profile URL, timestamp, number of likes, shares, comments, and more. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.

Apify

16.1k

221

Contact Details Scraper

vdrmota/contact-info-scraper

Free email extractor and lead scraper to extract and download emails, phone numbers, Facebook, Twitter, LinkedIn, and Instagram profiles from any website. Extract contact information at scale from lists of URLs and download the data as Excel, CSV, JSON, HTML, and XML.