Pricing

$19.99/month + usage

Go to Store

Instagram Hashtag Scraper

Try for free

Developed by

EasyApi

Effortlessly scrape Instagram hashtag data, including top posts and engagement metrics. Perfect for marketers, researchers, and data analysts looking to gain insights from Instagram trends and content performance. 📊

4.6 (2)

Pricing

$19.99/month + usage

Total users

430

Monthly users

Runs succeeded

>99%

Issues response

1.9 days

Last modified

a month ago

Social media

Lead generation

Back to issues Create new issue

Crawling seems to be broken

Open

niklas.klausner opened this issue

Since a couple of hours all requests seem to fail. Proxies are high quality mobiles and nothing has changed from before.

funkybadgerbutt

I'm experiencing this issue too. Here's my output.

2025-06-06T20:01:18.267Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-06T20:01:18.358Z ACTOR: Creating Docker container.
2025-06-06T20:01:18.555Z ACTOR: Starting Docker container.
2025-06-06T20:01:18.782Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-06T20:01:19.973Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-06T20:01:19.975Z   _isPuppeteerCore: true,
2025-06-06T20:01:19.977Z   _changedBrowsers: false,
2025-06-06T20:01:19.979Z   connect: [Function: bound bound connect],
2025-06-06T20:01:19.981Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-06T20:01:19.983Z   configuration: { defaultBrowser: 'chrome' },
2025-06-06T20:01:19.985Z   launch: [Function: bound launch],
2025-06-06T20:01:19.986Z   executablePath: [Function: bound executablePath],
2025-06-06T20:01:19.988Z   defaultArgs: [Function: bound defaultArgs],
2025-06-06T20:01:19.990Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-06T20:01:19.992Z }
2025-06-06T20:01:20.054Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-06T20:01:20.974Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-06T20:01:30.479Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-06T20:02:00.781Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-06T20:02:00.784Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":1}
2025-06-06T20:02:04.376Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-06T20:02:20.974Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":60624,"retryHistogram":[]}
2025-06-06T20:02:21.076Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.019},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.048},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-06T20:02:34.583Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-06T20:02:34.585Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":2}
2025-06-06T20:02:38.569Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-06T20:03:08.585Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-06T20:03:08.587Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":3}
2025-06-06T20:03:17.784Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-06T20:03:21.663Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":120624,"retryHistogram":[]}
2025-06-06T20:03:24.420Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.039},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.163},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-06T20:03:48.250Z ERROR PuppeteerCrawler: Request failed and reached maximum retries. TimeoutError: Navigation timeout of 30000 ms exceeded
2025-06-06T20:03:48.252Z     at new Deferred (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:60:34)
2025-06-06T20:03:48.254Z     at Deferred.create (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/util/Deferred.js:21:16)
2025-06-06T20:03:48.256Z     at new LifecycleWatcher (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/LifecycleWatcher.js:69:60)
2025-06-06T20:03:48.258Z     at CdpFrame.goto (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Frame.js:149:29)
2025-06-06T20:03:48.260Z     at CdpFrame.<anonymous> (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)
2025-06-06T20:03:48.261Z     at CdpPage.goto (/home/myuser/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Page.js:574:43)
2025-06-06T20:03:48.263Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32)
2025-06-06T20:03:48.265Z     at async wrap (/home/myuser/node_modules/@apify/timeout/cjs/index.cjs:54:21) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","method":"GET","uniqueKey":"https://www.instagram.com/explore/search/keyword?q=%23blacklivesmatter"}
2025-06-06T20:03:48.317Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-06T20:03:48.984Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":0,"requestsFailed":1,"retryHistogram":[null,null,null,1],"requestAvgFailedDurationMillis":39435,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":39435,"requestsTotal":1,"crawlerRuntimeMillis":148634}
2025-06-06T20:03:48.986Z INFO  PuppeteerCrawler: Error analysis: {"totalErrors":1,"uniqueErrors":1,"mostCommonErrors":["1x: Navigation timeout of 30000 ms exceeded (file:///home/myuser/src/main.js:71:32)"]}
2025-06-06T20:03:48.988Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 0 succeeded, 1 failed. {"terminal":true}

EasyApi (easyapi)

Hi, fixed, please try now

funkybadgerbutt

I'm still seeing the same error. No configurations were changed aside from the dates.

2025-06-07T00:18:21.850Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-07T00:18:44.289Z ACTOR: Creating Docker container.
2025-06-07T00:18:45.144Z ACTOR: Starting Docker container.
2025-06-07T00:18:45.457Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-07T00:18:46.898Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-07T00:18:46.899Z   _isPuppeteerCore: true,
2025-06-07T00:18:46.900Z   _changedBrowsers: false,
2025-06-07T00:18:46.901Z   connect: [Function: bound bound connect],
2025-06-07T00:18:46.902Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-07T00:18:46.903Z   configuration: { defaultBrowser: 'chrome' },
2025-06-07T00:18:46.904Z   launch: [Function: bound launch],
2025-06-07T00:18:46.905Z   executablePath: [Function: bound executablePath],
2025-06-07T00:18:46.906Z   defaultArgs: [Function: bound defaultArgs],
2025-06-07T00:18:46.907Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-07T00:18:46.908Z }
2025-06-07T00:18:46.959Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-07T00:18:47.763Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-07T00:19:47.763Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":60415,"retryHistogram":[]}
2025-06-07T00:19:47.789Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-07T00:19:49.890Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-07T00:20:19.893Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-07T00:20:19.894Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":1}
2025-06-07T00:20:47.762Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":120414,"retryHistogram":[]}
2025-06-07T00:20:47.792Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-07T00:21:21.768Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-07T00:21:47.762Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":180415,"retryHistogram":[]}
2025-06-07T00:21:47.794Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-07T00:21:51.772Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-07T00:21:51.773Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":2}
2025-06-07T00:22:18.316Z ACTOR: The Actor run was aborted by the user.

funkybadgerbutt

Any updates yet? Here's my latest log. It did start scrolling again but returned no data.

2025-06-08T07:58:14.544Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-08T07:58:23.449Z ACTOR: Creating Docker container.
2025-06-08T07:58:23.638Z ACTOR: Starting Docker container.
2025-06-08T07:58:23.924Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-08T07:58:25.872Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-08T07:58:25.874Z   _isPuppeteerCore: true,
2025-06-08T07:58:25.877Z   _changedBrowsers: false,
2025-06-08T07:58:25.879Z   connect: [Function: bound bound connect],
2025-06-08T07:58:25.881Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-08T07:58:25.883Z   configuration: { defaultBrowser: 'chrome' },
2025-06-08T07:58:25.885Z   launch: [Function: bound launch],
2025-06-08T07:58:25.887Z   executablePath: [Function: bound executablePath],
2025-06-08T07:58:25.889Z   defaultArgs: [Function: bound defaultArgs],
2025-06-08T07:58:25.891Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-08T07:58:25.893Z }
2025-06-08T07:58:25.953Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-08T07:58:26.885Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-08T07:58:41.153Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-08T07:59:11.466Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-08T07:59:11.474Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":1}
2025-06-08T07:59:18.768Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-08T07:59:26.956Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":60550,"retryHistogram":[]}
2025-06-08T07:59:26.968Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.079},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.301},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-08T07:59:48.862Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-08T07:59:48.864Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":2}
2025-06-08T08:00:26.955Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":120550,"retryHistogram":[]}
2025-06-08T08:00:26.970Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.039},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.038},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-08T08:00:30.492Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-08T08:01:00.494Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-08T08:01:00.496Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":3}
2025-06-08T08:01:17.866Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-08T08:01:26.956Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":180551,"retryHistogram":[]}
2025-06-08T08:01:26.973Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.059},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.221},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-08T08:01:39.092Z Scrolling time...1
2025-06-08T08:01:42.323Z Scrolling time...2
2025-06-08T08:01:46.616Z Scrolling time...3
2025-06-08T08:01:50.257Z Scrolling time...4
2025-06-08T08:01:55.152Z Scrolling time...5
2025-06-08T08:01:58.480Z Saved 0 items
2025-06-08T08:02:03.289Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-08T08:02:04.362Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":0,"retryHistogram":[null,null,null,1],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":60966,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":60966,"requestsTotal":1,"crawlerRuntimeMillis":217957}
2025-06-08T08:02:04.365Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 1 succeeded, 0 failed. {"terminal":true}

EasyApi (easyapi)

try now

funkybadgerbutt

Same problem - no results returned.

Using residential proxies:

2025-06-09T00:20:15.662Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-09T00:20:25.605Z ACTOR: Creating Docker container.
2025-06-09T00:20:25.657Z ACTOR: Starting Docker container.
2025-06-09T00:20:25.973Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-09T00:20:28.488Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-09T00:20:28.490Z   _isPuppeteerCore: true,
2025-06-09T00:20:28.492Z   _changedBrowsers: false,
2025-06-09T00:20:28.494Z   connect: [Function: bound bound connect],
2025-06-09T00:20:28.496Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-09T00:20:28.498Z   configuration: { defaultBrowser: 'chrome' },
2025-06-09T00:20:28.499Z   launch: [Function: bound launch],
2025-06-09T00:20:28.501Z   executablePath: [Function: bound executablePath],
2025-06-09T00:20:28.503Z   defaultArgs: [Function: bound defaultArgs],
2025-06-09T00:20:28.504Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-09T00:20:28.506Z }
2025-06-09T00:20:28.546Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-09T00:20:29.310Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-09T00:20:44.440Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-09T00:21:14.751Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-09T00:21:14.754Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":1}
2025-06-09T00:21:22.133Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-09T00:21:29.311Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":60392,"retryHistogram":[]}
2025-06-09T00:21:29.336Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.059},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.26},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-09T00:21:52.431Z WARN  PuppeteerCrawler: Reclaiming failed request back to the list or queue. Navigation timeout of 30000 ms exceeded
2025-06-09T00:21:52.443Z     at PuppeteerCrawler.requestHandler [as userProvidedRequestHandler] (file:///home/myuser/src/main.js:71:32) {"id":"oVznTYS5o5eMy1g","url":"https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter","retryCount":2}
2025-06-09T00:21:56.830Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-09T00:22:19.219Z Scrolling time...1
2025-06-09T00:22:23.315Z Scrolling time...2
2025-06-09T00:22:27.605Z Scrolling time...3
2025-06-09T00:22:29.311Z INFO  PuppeteerCrawler:Statistics: PuppeteerCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":120393,"retryHistogram":[]}
2025-06-09T00:22:29.338Z INFO  PuppeteerCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":3,"systemStatus":{"isSystemIdle":true,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":false,"limitRatio":0.6,"actualRatio":0.023},"cpuInfo":{"isOverloaded":false,"limitRatio":0.4,"actualRatio":0.114},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
2025-06-09T00:22:30.371Z Scrolling time...4
2025-06-09T00:22:35.428Z Scrolling time...5
2025-06-09T00:22:39.520Z Saved 0 items
2025-06-09T00:22:39.773Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-09T00:22:40.629Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":0,"retryHistogram":[null,null,1],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":46766,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":46766,"requestsTotal":1,"crawlerRuntimeMillis":131710}
2025-06-09T00:22:40.631Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 1 succeeded, 0 failed. {"terminal":true}

Using data center proxies

2025-06-09T00:17:30.347Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-09T00:17:30.438Z ACTOR: Creating Docker container.
2025-06-09T00:17:30.481Z ACTOR: Starting Docker container.
2025-06-09T00:17:30.659Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-09T00:17:33.622Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-09T00:17:33.623Z   _isPuppeteerCore: true,
2025-06-09T00:17:33.624Z   _changedBrowsers: false,
2025-06-09T00:17:33.625Z   connect: [Function: bound bound connect],
2025-06-09T00:17:33.626Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-09T00:17:33.627Z   configuration: { defaultBrowser: 'chrome' },
2025-06-09T00:17:33.627Z   launch: [Function: bound launch],
2025-06-09T00:17:33.628Z   executablePath: [Function: bound executablePath],
2025-06-09T00:17:33.629Z   defaultArgs: [Function: bound defaultArgs],
2025-06-09T00:17:33.630Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-09T00:17:33.630Z }
2025-06-09T00:17:33.676Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-09T00:17:34.416Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-09T00:17:39.836Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-09T00:17:55.748Z Scrolling time...1
2025-06-09T00:17:58.549Z Scrolling time...2
2025-06-09T00:18:02.249Z Scrolling time...3
2025-06-09T00:18:07.049Z Scrolling time...4
2025-06-09T00:18:11.582Z Scrolling time...5
2025-06-09T00:18:17.202Z Saved 0 items
2025-06-09T00:18:17.449Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-09T00:18:18.475Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":0,"retryHistogram":[1],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":42652,"requestsFinishedPerMinute":1,"requestsFailedPerMinute":0,"requestTotalDurationMillis":42652,"requestsTotal":1,"crawlerRuntimeMillis":44424}
2025-06-09T00:18:18.476Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 1 succeeded, 0 failed. {"terminal":true}

funkybadgerbutt

Let me know if you need any other of my parameters for debugging

EasyApi (easyapi)

Hi, fixed now, please try again.

funkybadgerbutt

Same issues and output. No data's returned. I am requesting data from 2020.

EasyApi (easyapi)

Hi, could you provide your run log?

funkybadgerbutt

Here's the log using datacenter proxies

2025-06-09T00:51:37.921Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-09T00:51:38.010Z ACTOR: Creating Docker container.
2025-06-09T00:51:38.081Z ACTOR: Starting Docker container.
2025-06-09T00:51:38.289Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-09T00:51:39.433Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-09T00:51:39.435Z   _isPuppeteerCore: true,
2025-06-09T00:51:39.437Z   _changedBrowsers: false,
2025-06-09T00:51:39.439Z   connect: [Function: bound bound connect],
2025-06-09T00:51:39.441Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-09T00:51:39.443Z   configuration: { defaultBrowser: 'chrome' },
2025-06-09T00:51:39.445Z   launch: [Function: bound launch],
2025-06-09T00:51:39.447Z   executablePath: [Function: bound executablePath],
2025-06-09T00:51:39.449Z   defaultArgs: [Function: bound defaultArgs],
2025-06-09T00:51:39.451Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-09T00:51:39.453Z }
2025-06-09T00:51:39.490Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-09T00:51:40.556Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-09T00:51:43.317Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-09T00:51:55.400Z Scrolling time...1
2025-06-09T00:52:01.181Z Scrolling time...2
2025-06-09T00:52:04.819Z Scrolling time...3
2025-06-09T00:52:09.721Z Scrolling time...4
2025-06-09T00:52:12.180Z Scrolling time...5
2025-06-09T00:52:15.520Z Saved 0 items
2025-06-09T00:52:15.879Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-09T00:52:16.743Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":0,"retryHistogram":[1],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":34842,"requestsFinishedPerMinute":2,"requestsFailedPerMinute":0,"requestTotalDurationMillis":34842,"requestsTotal":1,"crawlerRuntimeMillis":36909}
2025-06-09T00:52:16.745Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 1 succeeded, 0 failed. {"terminal":true}

EasyApi (easyapi)

how about without any proxy？

funkybadgerbutt

I had it on any country's proxies, default recommended.

funkybadgerbutt

Here's the output with no proxy:

https://pastebin.com/NG0uuZ6H

niklas.klausner

Two days ago it worked for me once with mobile proxies. Now I am getting a success but the response is empty.

EasyApi (easyapi)

Hi, fixed now, please try again

funkybadgerbutt

Same issues with datacenter proxies or residential proxies. Crawler finishes prematurely and returns no results.

EasyApi (easyapi)

hi, do not use proxy, it's unnecessary, we handled it in backend

funkybadgerbutt

Here's my output without any proxy. Still no data processed. Am I missing any other parameters?

2025-06-10T23:52:23.488Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-10T23:52:34.727Z ACTOR: Creating Docker container.
2025-06-10T23:52:34.894Z ACTOR: Starting Docker container.
2025-06-10T23:52:35.150Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-10T23:52:38.577Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-10T23:52:38.578Z   _isPuppeteerCore: true,
2025-06-10T23:52:38.579Z   _changedBrowsers: false,
2025-06-10T23:52:38.580Z   connect: [Function: bound bound connect],
2025-06-10T23:52:38.581Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-10T23:52:38.582Z   configuration: { defaultBrowser: 'chrome' },
2025-06-10T23:52:38.583Z   launch: [Function: bound launch],
2025-06-10T23:52:38.584Z   executablePath: [Function: bound executablePath],
2025-06-10T23:52:38.584Z   defaultArgs: [Function: bound defaultArgs],
2025-06-10T23:52:38.585Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-10T23:52:38.586Z }
2025-06-10T23:52:38.640Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-10T23:52:39.266Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-10T23:52:41.431Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-10T23:52:48.725Z Scrolling time...1
2025-06-10T23:52:51.984Z Scrolling time...2
2025-06-10T23:52:54.731Z Scrolling time...3
2025-06-10T23:52:58.202Z Scrolling time...4
2025-06-10T23:53:01.596Z Scrolling time...5
2025-06-10T23:53:05.218Z Saved 0 items
2025-06-10T23:53:05.619Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-10T23:53:06.302Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":0,"retryHistogram":[1],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":25733,"requestsFinishedPerMinute":2,"requestsFailedPerMinute":0,"requestTotalDurationMillis":25733,"requestsTotal":1,"crawlerRuntimeMillis":27543}
2025-06-10T23:53:06.303Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 1 succeeded, 0 failed. {"terminal":true}

EasyApi (easyapi)

Hi, try now, I just test your keyword, it's ok:

2025-06-11T03:29:45.207Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter... 2025-06-11T03:29:54.485Z Scrolling time...1 2025-06-11T03:29:58.996Z Scrolling time...2 2025-06-11T03:30:02.685Z Saved 42 items

funkybadgerbutt

So the crawler works fine if I leave date filtering out. The current date filters I have are 2020-05-26T00:00:00Z to 2020-07-01T00:00:00Z. It still isn't returning anything.

Are the dates too far back?

2025-06-11T04:38:11.391Z ACTOR: Pulling Docker image of build 1N6PzRr5C3vor7wT8 from registry.
2025-06-11T04:38:11.394Z ACTOR: Creating Docker container.
2025-06-11T04:38:11.496Z ACTOR: Starting Docker container.
2025-06-11T04:38:11.667Z Will run command: xvfb-run -a -s "-ac -screen 0 1920x1080x24+32 -nolisten tcp" /bin/sh -c npm start --silent
2025-06-11T04:38:14.272Z Warning: Plugin is not derived from PuppeteerExtraPlugin, ignoring. PuppeteerNode {
2025-06-11T04:38:14.274Z   _isPuppeteerCore: true,
2025-06-11T04:38:14.276Z   _changedBrowsers: false,
2025-06-11T04:38:14.278Z   connect: [Function: bound bound connect],
2025-06-11T04:38:14.280Z   defaultBrowserRevision: '131.0.6778.87',
2025-06-11T04:38:14.282Z   configuration: { defaultBrowser: 'chrome' },
2025-06-11T04:38:14.284Z   launch: [Function: bound launch],
2025-06-11T04:38:14.286Z   executablePath: [Function: bound executablePath],
2025-06-11T04:38:14.288Z   defaultArgs: [Function: bound defaultArgs],
2025-06-11T04:38:14.290Z   trimCache: [Function: bound trimCache] AsyncFunction
2025-06-11T04:38:14.293Z }
2025-06-11T04:38:14.361Z INFO  System info {"apifyVersion":"3.4.2","apifyClientVersion":"2.12.4","crawleeVersion":"3.13.4","osType":"Linux","nodeVersion":"v20.19.2"}
2025-06-11T04:38:15.132Z INFO  PuppeteerCrawler: Starting the crawler.
2025-06-11T04:38:17.380Z Processing https://www.instagram.com/explore/search/keyword/?q=%23blacklivesmatter...
2025-06-11T04:38:22.048Z Scrolling time...1
2025-06-11T04:38:25.723Z Scrolling time...2
2025-06-11T04:38:29.122Z Scrolling time...3
2025-06-11T04:38:32.444Z Scrolling time...4
2025-06-11T04:38:35.751Z Scrolling time...5
2025-06-11T04:38:39.350Z Saved 0 items
2025-06-11T04:38:40.243Z INFO  PuppeteerCrawler: All requests from the queue have been processed, the crawler will shut down.
2025-06-11T04:38:41.126Z INFO  PuppeteerCrawler: Final request statistics: {"requestsFinished":1,"requestsFailed":0,"retryHistogram":[1],"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":24297,"requestsFinishedPerMinute":2,"requestsFailedPerMinute":0,"requestTotalDurationMillis":24297,"requestsTotal":1,"crawlerRuntimeMillis":26546}
2025-06-11T04:38:41.128Z INFO  PuppeteerCrawler: Finished! Total 1 requests: 1 succeeded, 0 failed. {"terminal":true}

funkybadgerbutt

Using another hashtag from with time filtering from 2023-04-01T00:00:00Z to 2023-05-01T00:00:00Z, it scrolled but returned 4 results.

niklas.klausner

getting timeouts now...

EasyApi (easyapi)

Hi, fixed now

niklas.klausner

WARN PuppeteerCrawler: Reclaiming failed request back to the list or queue. Detected a session error, rotating session... 2025-06-23T07:07:26.491Z net::ERR_TUNNEL_CONNECTION_FAILED

This used to be a proxy issue on my side but seems to be not the case this time. Can you take a look? Thank you

Add comment

Instagram Hashtag Scraper Pro

scraping_solutions/instagram-hashtag-scraper-pro

Instagram Hashtag Scraper API: Automatically extract public Instagram posts by hashtag. Perfect for trend analysis, campaign monitoring, and real-time content collection.

Scraping Solutions

Instagram Hashtag Scraper

coderx/instagram-hashtag-scraper

This scraper extracts users from Instagram based on a hashtag. Just enter a hashtag and set the number of usernames you need. It collects user details like usernames, IDs, and captions, making it perfect for social media analysis and research. 🚀

CoderX

115

Instagram Hashtag Media Scraper

powerai/instagram-hashtag-media-scraper

Extract Instagram posts for a specific hashtag with detailed media and metadata. Supports auto-pagination and custom result limits. No login required.

PowerAI

5.0

Instagram Hashtag Username Scraper

coderx/instagram-hashtag-username-scraper

This actor scrapes Instagram posts based on a specified hashtag and extracts usernames, profile names, and user IDs. It helps track engaged users, analyze social media trends, and gather audience insights. Ideal for influencer research, marketing, and hashtag-based user analysis.

CoderX

372

5.0

Instagram Related Hashtag Stats Scraper

apify/instagram-hashtag-stats

Extract detailed Instagram hashtag statistics fast. Get total post count, posts per day, top and latest posts, related hashtags (literal and semantic), and their usage frequency. Export scraped stats, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

1.2K

4.5

Instagram Posts Scraper

easyapi/instagram-posts-scraper

Effortlessly scrape Instagram posts data from multiple users. Extract comprehensive details including likes, comments, captions, hashtags, and more. Perfect for social media analysis, market research, and content monitoring.

EasyApi

105

Instagram Keywords Intelligence Tool 🎯

easyapi/instagram-keywords-intelligence-tool

🔍 Discover trending hashtags and influential users on Instagram with real-time keyword analysis. Get comprehensive insights into hashtag popularity and user engagement. Perfect for social media marketers, influencers, and researchers looking to optimize their Instagram strategy.

EasyApi

3.0

Instagram Scraper

apify/instagram-scraper

Scrape and download Instagram posts, profiles, places, hashtags, photos, and comments. Get data from Instagram using one or more Instagram URLs or search queries. Export scraped data, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

113K

3.7

Instagram Hashtag Scraper

apify/instagram-hashtag-scraper

Scrape Instagram hashtags data. Just add one or more hashtags and extract posts, images, URLs, comments, likes, users, locations, timestamps, and more. Export scraped datasets, run the scraper via API, schedule and monitor runs or integrate with other tools.

Apify

25K

4.5

Instagram Posts Scraper

pratikdani/instagram-posts-scraper

The Instagram Post Scraper extracts detailed data from any valid post URL, including content, engagement metrics, user details, and media. It delivers structured output for social media analysis, monitoring, and research.