2 hours trial then $20.00/month - No credit card required now
Fiverr Scraper
2 hours trial then $20.00/month - No credit card required now
Scrapes Fiverr gigs
Use of uploaded txt-file for start-urls not working
Closed
I the format of urls in url_list.txt wrong?
log:
2024-02-01T14:54:52.154Z ACTOR: Pulling Docker image of build T9ERS7oBtxhv9eqEd from repository.
2024-02-01T14:54:52.613Z ACTOR: Creating Docker container.
2024-02-01T14:54:52.720Z ACTOR: Starting Docker container.
2024-02-01T14:54:56.257Z INFO System info {"apifyVersion":"3.1.13","apifyClientVersion":"2.8.4","crawleeVersion":"3.6.2","osType":"Linux","nodeVersion":"v16.20.2"}
2024-02-01T14:54:56.537Z url: [
2024-02-01T14:54:56.538Z {
2024-02-01T14:54:56.538Z requestsFromUrl: 'https://apify-uploads-prod.s3.us-east-1.amazonaws.com/4sD6Sa2mZhG2oztvD-url_list.txt'
2024-02-01T14:54:56.538Z }
2024-02-01T14:54:56.539Z ]
2024-02-01T14:54:56.751Z /usr/src/app/node_modules/ow/dist/index.js:33
2024-02-01T14:54:56.752Z (0, test_1.default)(value, () => (0, infer_label_1.inferLabel)(stackFrames), labelOrPredicate);
2024-02-01T14:54:56.753Z ^
2024-02-01T14:54:56.753Z
2024-02-01T14:54:56.753Z ArgumentError: (array requests) Any predicate failed with the following errors:
2024-02-01T14:54:56.754Z - Expected values to be of type string but received type Object
2024-02-01T14:54:56.754Z - Expected property url to be of type string but received type undefined in object values
2024-02-01T14:54:56.754Z - Expected property requestsFromUrl to be of type string but received type undefined in object values
2024-02-01T14:54:56.755Z at ow (/usr/src/app/node_modules/ow/dist/index.js:33:28)
2024-02-01T14:54:56.755Z at RequestQueue.addRequestsBatched (/usr/src/app/node_modules/@crawlee/core/storages/request_provider.js:304:26)
2024-02-01T14:54:56.755Z at RequestQueue.addRequestsBatched (/usr/src/app/node_modules/@crawlee/core/storages/request_queue.js:266:22)
2024-02-01T14:54:56.756Z at CheerioCrawler.addRequests (/usr/src/app/node_modules/@crawlee/basic/internals/basic-crawler.js:608:29)
2024-02-01T14:54:56.756Z at processTicksAndRejections (node:internal/process/task_queues:96:5)
2024-02-01T14:54:56.757Z at async file:///usr/src/app/src/main.js:43:1 {
2024-02-01T14:54:56.757Z validationErrors: Map(1) {
2024-02-01T14:54:56.757Z 'requests' => Set(1) {
2024-02-01T14:54:56.758Z '(array requests) Any predicate failed with the following errors:\n' +
2024-02-01T14:54:56.758Z ' - Expected values to be of type string but received type Object\n' +
2024-02-01T14:54:56.759Z ' - Expected property url to be of type string but received type undefined in object values\n' +
2024-02-01T14:54:56.759Z ' - Expected property requestsFromUrl to be of type string but received type undefined in object values'
2024-02-01T14:54:56.760Z }
2024-02-01T14:54:56.760Z }
2024-02-01T14:54:56.761Z }
I'm not aware that we have this feature to allow using a txt of list of urls. You might have to write some custom codes to handle converting the txt to a list of urls that we accept. BTW we are happy to take any custom projects to tailor to your needs.
newpo- 12 monthly users
- 100.0% runs succeeded
- 0.15 days response time
- Created in Feb 2023
- Modified 5 months ago
