No credit card required
Scrapy Books Example
No credit card required
Example of Python Scrapy project. It scrapes book data from https://books.toscrape.com/.
Works perfectly in CLI :-), but not in Apify console :-(... ?!?
Open
Hello, everyone! I am currently following the tutorial: https://www.youtube.com/watch?v=4nxStxC1BJM&list=PLObrtcm1Kw6PEnu5BpeEFb8XEoQXMw0g7&index=11
And at this part, dealing with the deployment of ‘booksscraper’: 05:33 Demo: building an Actor using templates and tools
I observe the following problem: It works perfectly in CLI, but bugs as an Actor in the Apify console…
Yet, I do get: Succeeded Success! Everything went according to plan.
While the log file shows: https://prnt.sc/ub4nDgcrr3ig
Of course, no result...
Would you have any idea?
Hi, on the screenshot there is only a part of the log, I don't even see the whole traceback. Could you please provide the entire traceback and your source code and Actor configuration? As a text (not a screenshot) please, so I can reproduce it. Thanks.
Hi, Vlada.
First, thanks for your quick answer.
And of course, here the full log:
https://api.apify.com/v2/logs/N5dmgi7HGIgFqzYuc
And the .actor/actor.json:
1{ 2 "actorSpecification": 1, 3 "name": "booksscraper", 4 "title": "Getting started with Python and Scrapy", 5 "description": "Python Scrapy project wrapped using the Apify CLI.", 6 "version": "0.0", 7 "meta": { 8 "templateId": "python-scrapy" 9 }, 10 "input": "./input_schema.json", 11 "dockerfile": "./Dockerfile" 12}
Note I think I have correctly followed the process as video in youtube...
To soon
Oh, I see the problem :).
builtins.NameError: Module 'apify.scrapy.middlewares' doesn't define any object named 'ApifyRetryMiddleware'
A custom ApifyRetryMiddleware is not needed anymore and was removed in the later versions of SDK.
So, you should just remove it from your settings.py file - I suppose you referencing it there.
You can also compare your (settings) code with our reference scrapy project -> https://github.com/apify/actor-scrapy-books-example.
Hope it helps!
Vlada Dusek- 1 monthly user
- 2 stars
- 100.0% runs succeeded
- 3.6 days response time
- Created in Dec 2023
- Modified about 1 month ago
