Empty Python project
Empty template with basic structure for the Actor with Apify SDK that allows you to easily add your own functionality.
src/main.py
src/__main__.py
1"""This module defines the main entry point for the Apify Actor.
2
3Feel free to modify this file to suit your specific needs.
4
5To build Apify Actors, utilize the Apify SDK toolkit, read more at the official documentation:
6https://docs.apify.com/sdk/python
7"""
8
9from apify import Actor
10
11
12async def main() -> None:
13 """Main entry point for the Apify Actor.
14
15 This coroutine is executed using `asyncio.run()`, so it must remain an asynchronous function for proper execution.
16 Asynchronous execution is required for communication with Apify platform, and it also enhances performance in
17 the field of web scraping significantly.
18 """
19 async with Actor:
20 Actor.log.info('Hello from the Actor!')
21 # Write your code here
Python empty template
Start a new web scraping project quickly and easily in Python with our empty project template. It provides a basic structure for the Actor with Apify SDK and allows you to easily add your own functionality.
Included features
- Apify SDK for Python - a toolkit for building Apify Actors and scrapers in Python
- Input schema - define and easily validate a schema for your Actor's input
- Request queue - queues into which you can put the URLs you want to scrape
- Dataset - store structured data where each object stored has the same attributes
How it works
Insert your own code to async with Actor:
block. You can use the Apify SDK with any other Python library.
Resources
- Python tutorials in Academy
- Video guide on getting data using Apify API
- Integration with Make, GitHub, Zapier, Google Drive, and other apps
- A short guide on how to build web scrapers using code templates:
This example Scrapy spider scrapes page titles from URLs defined in input parameter. It shows how to use Apify SDK for Python and Scrapy pipelines to save results.
Scrape single page with provided URL with HTTPX and extract data from page's HTML with Beautiful Soup.
Example of a web scraper that uses Python HTTPX to scrape HTML from URLs provided on input, parses it using BeautifulSoup and saves results to storage.
Crawler example that uses headless Chrome driven by Playwright to scrape a website. Headless browsers render JavaScript and can help when getting blocked.
Scraper example built with Selenium and headless Chrome browser to scrape a website and save the results to storage. A popular alternative to Playwright.
Template with basic structure for an Actor using Standby mode that allows you to easily add your own functionality.