Scrapy Books Example avatar
Scrapy Books Example
Try for free

No credit card required

View all Actors
Scrapy Books Example

Scrapy Books Example

vdusek/scrapy-books-example
Try for free

No credit card required

Example of Python Scrapy project. It scrapes book data from https://books.toscrape.com/.

User avatar

Works perfectly in CLI :-), but not in Apify console :-(... ?!?

Open

GC7 (GC7) opened this issue
a month ago

Hello, everyone! I am currently following the tutorial: https://www.youtube.com/watch?v=4nxStxC1BJM&list=PLObrtcm1Kw6PEnu5BpeEFb8XEoQXMw0g7&index=11

And at this part, dealing with the deployment of ‘booksscraper’: 05:33 Demo: building an Actor using templates and tools

I observe the following problem: It works perfectly in CLI, but bugs as an Actor in the Apify console…

Yet, I do get: Succeeded Success! Everything went according to plan.

While the log file shows: https://prnt.sc/ub4nDgcrr3ig

Of course, no result...

Would you have any idea?

User avatar

Hi, on the screenshot there is only a part of the log, I don't even see the whole traceback. Could you please provide the entire traceback and your source code and Actor configuration? As a text (not a screenshot) please, so I can reproduce it. Thanks.

User avatar

GC7 (GC7)

a month ago

Hi, Vlada.

First, thanks for your quick answer.

And of course, here the full log:

https://api.apify.com/v2/logs/N5dmgi7HGIgFqzYuc

And the .actor/actor.json:

1{
2    "actorSpecification": 1,
3    "name": "booksscraper",
4    "title": "Getting started with Python and Scrapy",
5    "description": "Python Scrapy project wrapped using the Apify CLI.",
6    "version": "0.0",
7    "meta": {
8        "templateId": "python-scrapy"
9    },
10    "input": "./input_schema.json",
11    "dockerfile": "./Dockerfile"
12}

Note I think I have correctly followed the process as video in youtube...

To soon

User avatar

Oh, I see the problem :).

builtins.NameError: Module 'apify.scrapy.middlewares' doesn't define any object named 'ApifyRetryMiddleware'

A custom ApifyRetryMiddleware is not needed anymore and was removed in the later versions of SDK.

So, you should just remove it from your settings.py file - I suppose you referencing it there.

You can also compare your (settings) code with our reference scrapy project -> https://github.com/apify/actor-scrapy-books-example.

Hope it helps!

Developer
Maintained by Apify
Actor metrics
  • 2 monthly users
  • 2 stars
  • 100.0% runs succeeded
  • 3.6 days response time
  • Created in Dec 2023
  • Modified 8 days ago