1 day trial then $20.00/month - No credit card required now
Substack Scraper
1 day trial then $20.00/month - No credit card required now
The Substack Author Scraper is a powerful Apify actor that makes it easy for content creators to scrape and retrieve all posts from their favorite Substack authors. With structured data presented in a user-friendly format, analyzing and processing valuable information has never been easier.
The subscription baed substack accounts are impossible to scrape
Closed
this is one of the subsctiption based accounts I tried to scrape:
2024-04-03T09:07:38.678Z ACTOR: Pulling Docker image of build 79Jhj6dXdLhgwsEkQ from repository. 2024-04-03T09:07:41.656Z ACTOR: Creating Docker container. 2024-04-03T09:07:41.741Z ACTOR: Starting Docker container. 2024-04-03T09:07:44.037Z INFO Initializing actor... 2024-04-03T09:07:44.038Z INFO System info ({"apify_sdk_version": "1.0.0", "apify_client_version": "1.0.0", "python_version": "3.11.6", "os": "linux"}) 2024-04-03T09:07:44.126Z 2024-04-03 09:07:44 [INFO] substack_scraper.main - Scraping 10 most recent posts from asiancenturystocks, (scrape article content: True) - (main.py:20) 2024-04-03T09:07:44.128Z 2024-04-03 09:07:44 [INFO] substack_scraper.posts_scraper - Scraping posts from author asiancenturystocks using url https://asiancenturystocks.substack.com (profile: https://asiancenturystocks.substack.com) - (posts_scraper.py:32) 2024-04-03T09:07:44.235Z ERROR Actor failed with an exception 2024-04-03T09:07:44.236Z Traceback (most recent call last): 2024-04-03T09:07:44.237Z File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 971, in json 2024-04-03T09:07:44.238Z return complexjson.loads(self.text, **kwargs) 2024-04-03T09:07:44.239Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-03T09:07:44.239Z File "/usr/local/lib/python3.11/json/init.py", line 346, in loads 2024-04-03T09:07:44.240Z return _default_decoder.decode(s) 2024-04-03T09:07:44.241Z ^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-03T09:07:44.241Z File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode 2024-04-03T09:07:44.242Z obj, end = self.raw_decode(s, idx=_w(s, 0).end()) 2024-04-03T09:07:44.243Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-03T09:07:44.244Z File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode 2024-04-03T09:07:44.244Z raise JSONDecodeError("Expecting value", s, err.value) from None 2024-04-03T09:07:44.245Z json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2024-04-03T09:07:44.246Z 2024-04-03T09:07:44.247Z During handling of the above exception, another exception occurred: 2024-04-03T09:07:44.248Z 2024-04-03T09:07:44.248Z Traceback (most recent call last): 2024-04-03T09:07:44.249Z File "/usr/src/app/src/main.py", line 43, in main 2024-04-03T09:07:44.250Z data = await scrap(actor_input) 2024-04-03T09:07:44.251Z ^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-03T09:07:44.251Z File "/usr/src/app/src/main.py", line 23, in scrap 2024-04-03T09:07:44.252Z posts: list[Post] = PostsScraper.scrap(author, limit) 2024-04-03T09:07:44.253Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-04-03T09:07:44.253Z File "/usr/src/app/src/substack_scraper/posts_scraper.py", line 50, in scrap 2024-04-03T09:07:44.254Z data = response.json() 2024-04-03T09:07:44.255Z ^^^^^^^^^^^^^^^ 2024-04-03T09:07:44.255Z File "/usr/local/lib/python3.11/site-packages/requests/models.py", line 975, in json 2024-04-03T09:07:44.256Z raise RequestsJSONDecodeError(e.msg, e.doc, e.pos) 2024-04-03T09:07:44.257Z requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2024-04-03T09:07:44.257Z INFO Exiting actor ({"exit_code": 91})
Hello,
It's because the input require the author of the article. In your case the author name is "fritz"
Please give it a try and let us know if that resolve your issue
Regards, Yoann
Hello, the issue has been fixed
QPS- 7 monthly users
- 3 stars
- 71.9% runs succeeded
- 4.8 hours response time
- Created in Apr 2023
- Modified 4 months ago
