Amazon Product Scraper

Hi Massi, thanks for opening this issue!

I can see that you've resurrected the run 4 times, did the issue occur more than once? Unfortunately your run is so big, that the logs are cut off early, but I can see one occurrence of an unhandled error:

12024-07-03T01:43:04.722Z node:events:491
22024-07-03T01:43:04.725Z       throw er; // Unhandled 'error' event
32024-07-03T01:43:04.727Z       ^
42024-07-03T01:43:04.729Z 
52024-07-03T01:43:04.731Z Error: Client network socket disconnected before secure TLS connection was established
62024-07-03T01:43:04.733Z     at connResetException (node:internal/errors:705:14)
72024-07-03T01:43:04.735Z     at TLSSocket.onConnectEnd (node:_tls_wrap:1594:19)
82024-07-03T01:43:04.737Z     at TLSSocket.emit (node:events:525:35)
92024-07-03T01:43:04.739Z     at endReadableNT (node:internal/streams/readable:1358:12)
102024-07-03T01:43:04.741Z     at processTicksAndRejections (node:internal/process/task_queues:83:21)
112024-07-03T01:43:04.743Z Emitted 'error' event on TLSSocket instance at:
122024-07-03T01:43:04.745Z     at emitErrorNT (node:internal/streams/destroy:157:8)
132024-07-03T01:43:04.747Z     at emitErrorCloseNT (node:internal/streams/destroy:122:3)
142024-07-03T01:43:04.749Z     at processTicksAndRejections (node:internal/process/task_queues:83:21) {
152024-07-03T01:43:04.751Z   code: 'ECONNRESET',
162024-07-03T01:43:04.753Z   path: undefined,
172024-07-03T01:43:04.755Z   host: undefined,
182024-07-03T01:43:04.757Z   port: undefined,
192024-07-03T01:43:04.759Z   localAddress: undefined
202024-07-03T01:43:04.761Z }

Apparently there was a short outage of one of our Workers for a few minutes, which caused Actor runs to suddenly stop. Although I'm not sure when that exactly that happened. It looks like a likely explanation. Thankfully, your run was not affected, as this Actor is built to be stopped and resurrected at any moment.

Btw, looking at your run, your input has a lot of duplicate URLs. Not to worry, they are deduplicated by the crawler, however you might want to clean them in your own list. Also, there are a lot non-existed/deleted products in your list. Those cannot be simply removed from the crawl, and a lot of resources are wasted on them.

Please let us know if you encounter this again, thanks and happy scraping!