TikTok Scraper

Since July 9, we have seen a significant number of job failures.

12024-07-11T15:13:00.905Z WARN  PlaywrightCrawler: Reclaiming failed request back to the list or queue. page.waitForSelector: Timeout 30000ms exceeded.
22024-07-11T15:13:00.907Z Call log:
32024-07-11T15:13:00.909Z   - waiting for locator('script[id=__UNIVERSAL_DATA_FOR_REHYDRATION__]')
42024-07-11T15:13:00.910Z   -   locator resolved to hidden <script type="application/json" id="__UNIVERSAL_DATA_FOR…>{"__DEFAULT_SCOPE__":{"webapp.app-context":{"lang…</script>
52024-07-11T15:13:00.911Z
62024-07-11T15:13:00.912Z     at dealWithProfileDetails (/home/myuser/dist/src/routes/profile.js:13:16) {"id":"DvOnhdJOW92rZ1Q","url":"https://www.tiktok.com/@anth.inathens","retryCount":1}
72024-07-11T15:13:10.806Z INFO  [Status message]: Scraped 0/1 profiles
82024-07-11T15:13:21.239Z INFO  [Status message]: Scraped 0/1 profiles
92024-07-11T15:13:21.408Z INFO  PlaywrightCrawler:Statistics: PlaywrightCrawler request statistics: {"requestAvgFailedDurationMillis":null,"requestAvgFinishedDurationMillis":null,"requestsFinishedPerMinute":0,"requestsFailedPerMinute":0,"requestTotalDurationMillis":0,"requestsTotal":0,"crawlerRuntimeMillis":61521,"retryHistogram":[]}
102024-07-11T15:13:22.613Z INFO  PlaywrightCrawler:AutoscaledPool: state {"currentConcurrency":1,"desiredConcurrency":1,"systemStatus":{"isSystemIdle":false,"memInfo":{"isOverloaded":false,"limitRatio":0.2,"actualRatio":0},"eventLoopInfo":{"isOverloaded":true,"limitRatio":0.6,"actualRatio":0.692},"cpuInfo":{"isOverloaded":true,"limitRatio":0.4,"actualRatio":1},"clientInfo":{"isOverloaded":false,"limitRatio":0.3,"actualRatio":0}}}
112024-07-11T15:13:22.615Z WARN  Setting status message timed out after 1s
122024-07-11T15:13:22.699Z WARN  PlaywrightCrawler: Setting status message timed out after 1s
132024-07-11T15:13:24.604Z WARN  Getting the current run timed out after 1s
142024-07-11T15:13:31.711Z INFO  [Status message]: Scraped 0/1 profiles
152024-07-11T15:13:33.243Z WARN  Setting status message timed out after 1s
162024-07-11T15:13:33.414Z WARN  PlaywrightCrawler: Setting status message timed out after 1s
172024-07-11T15:13:34.627Z WARN  Getting the current run timed out after 1s
182024-07-11T15:13:41.631Z INFO  [Status message]: Scraped 0/1 profiles
192024-07-11T15:13:43.008Z WARN  Setting status message timed out after 1s
202024-07-11T15:13:43.013Z WARN  PlaywrightCrawler: Setting status message timed out after 1s
212024-07-11T15:13:52.199Z INFO  [Status message]: Scraped 0/1 profiles
222024-07-11T15:13:53.309Z WARN  Setting status message timed out after 1s
232024-07-11T15:13:53.311Z WARN  PlaywrightCrawler: Setting status message timed out after 1s
242024-07-11T15:13:54.905Z WARN  Getting the current run timed out after 1s
252024-07-11T15:14:02.133Z INFO  [Status message]: Scraped 0/1 profiles
262024-07-11T15:14:04.930Z WARN  PlaywrightCrawler: Reclaiming failed request back to the list or queue. page.waitForSelector: Timeout 30000ms exceeded.
272024-07-11T15:14:04.931Z Call log:
282024-07-11T15:14:04.933Z   - waiting for locator('script[id=__UNIVERSAL_DATA_FOR_REHYDRATION__]')
292024-07-11T15:14:04.934Z
302024-07-11T15:14:04.935Z     at dealWithProfileDetails (/home/myuser/dist/src/routes/profile.js:13:16) {"id":"DvOnhdJOW92rZ1Q","url":"https://www.tiktok.com/@anth.inathens","retryCount":2}
312024-07-11T15:14:12.207Z INFO  [Status message]: Scraped 0/1 profiles
322024-07-11T15:14:13.860Z ACTOR: The Actor run has reached the timeout of 120 seconds, aborting it. You can increase the timeout in Input > Run options.