Universal Avito Ads Crawler (All Categories) / Универсальный скрепер объявлений Авито (все категории)

Introduction / Введение

Universal Avito Ads Crawler is an Apify Actor that systematically crawls Avito (Авито) – Russia’s largest classifieds marketplace – to extract structured ads data from all categories (vehicles, real estate, jobs, electronics, services, etc.). It uses a custom proprietary framework to navigate Avito’s web pages and retrieve information on each listing.

This tool automates the collection of Avito (Авито) marketplace data – from all categories including vehicles (авто), real estate (недвижимость), jobs (вакансии), electronics (электроника), and more. By providing specific search URLs, users can build a comprehensive dataset of Avito ads (объявления) for analytics or integration.

Features ✨ / Возможности

• All Categories: Scrapes Avito across all categories – cars, rentals, jobs, services, etc. – to compile a broad ads database.
• Custom Filters: Supports keyword search, category selection, and city/region filters by using customized start URLs.
• Detail Extraction: Captures rich data fields from each ad: listing ID, title, description, price, currency, location (city/region), posting date, seller name, contact info, images URLs, and other metadata.
• Pagination & Limits: Handles pagination automatically and respects the maxResults limit, so you control how many listings to scrape per run.
• Anti-blocking Measures: Uses advanced techniques like TLS fingerprinting, intelligent proxy rotation, and randomized delays to avoid IP bans.
• Dataset Output: Saves all extracted data into an Apify dataset, ready for export in JSON, CSV, Excel, or other formats.

How It Works / Как это работает

The actor is built on a high-performance, custom proprietary framework written in Go. It does not run a headless browser; instead, it sends HTTP requests directly, which makes it incredibly fast and efficient.

It loads the initial search pages from the startUrls you provide, parses the listings, and then creates new requests to scrape the details from each ad page. The framework automatically manages the entire scraping flow, including pagination, until it reaches the maxResults limit or there are no more pages.

To avoid being blocked, the actor employs several advanced anti-scraping techniques:

• TLS Fingerprint Spoofing: It mimics the TLS handshake of real browsers to avoid detection by advanced firewalls.
• Intelligent Proxy and User-Agent Rotation: It automatically rotates proxies and browser headers for each request.
• Dynamic Rate Limiting: The actor intelligently adjusts its request rate to avoid overwhelming the target server.

All collected data is consolidated into an Apify dataset, which you can easily download or integrate with other tools.

Input Configuration / Настройка входных данных

The actor requires the following inputs to run:

startUrls (Required)

An array of one or more URLs to begin scraping from. These URLs should be from avito.ru and can be customized with search queries, filters, and categories.

How to get startUrls:

1. Go to Avito.ru.
2. Use the search bar and filters to find the listings you want to scrape (e.g., search for "iPhone" in "Moscow").
3. Copy the URL from your browser's address bar.
4. Paste it into the startUrls field. You can add multiple URLs.

Example:

[
  { "url": "https://www.avito.ru/moskva?q=iPhone" },
  { "url": "https://www.avito.ru/sankt-peterburg/kvartiry/prodam-ASgBAgICAUSSA8YQ" }
]

maxResults

The maximum number of ads to scrape. The crawler will stop once this limit is reached. If set to 0 or left empty, it will scrape all available results.

Example: 1000

proxyConfiguration (Required)

Configure the proxies to be used for scraping. Due to Avito's strong anti-crawling measures, using a reliable proxy is essential.

• Apify Proxy: The recommended and easiest option. It automatically uses Apify's pool of residential proxies.
• BrightData Proxy: If you have a BrightData account, you can use it by providing your credentials in the brightDataProxyConfig field. This actor was tested using BrightData proxies.
• Custom Proxy URLs: You can provide your own list of proxy URLs.

Proxy Recommendations:

• Cost-Efficiency: For the best balance of cost and performance, Datacenter proxies are recommended.
• IP Pool Size: Avito has complex anti-crawling measures. If you are using your own proxies (either via proxyUrls or a custom BrightData setup), you will need a very large number of IPs to avoid getting blocked. A small pool of proxies will quickly be detected and banned.

brightDataProxyConfig

A JSON object containing your BrightData proxy credentials. This is only required if you select BrightData as your proxy provider.

Example:

{
  "host": "brd.superproxy.io",
  "port": 22225,
  "username": "brd-customer-YOUR_CUSTOMER_ID-zone-YOUR_ZONE",
  "password": "YOUR_PASSWORD",
  "customer_id": "YOUR_CUSTOMER_ID",
  "zone_name": "YOUR_ZONE"
}

Output / Вывод

The actor produces a dataset of JSON records, one per Avito listing. Each record includes the listing ID, title, full description, category/subcategory, price (and currency), location (city/region), posting date, seller name (and type/status), contact info (if available), image URLs, and other ad attributes.

Here is an example of a single scraped ad:

{
  "AdID": "№ 4306588198",
  "Address": "Москва, Большая кольцевая линияСавёловскаядо 5 мин.Менделеевская16–20 мин.Петровский парк21–30 мин.",
  "Description": "Мы более 10 лет на рынке. В связи с увеличением заказов, требуются отделочники универсалы...",
  "ImageURLs": null,
  "Location": "Москва",
  "Params": {
    "График": "Фиксированный",
    "Занятость": "Полная",
    "Сфера деятельности компании": "Ремонт и отделка помещений"
  },
  "PostedDate": "· сегодня в 18:04",
  "Price": "от 9 000 ₽ на руки",
  "SellerName": "Cherry-remont",
  "SellerType": "Работодатель",
  "Title": "Отделочник универсал в Москве",
  "URL": "/moskva/vakansii/otdelochnik_universal_4306588198",
  "Views": "1345 просмотров"
}

Use Cases / Примеры использования

• Market Research & Price Monitoring: Analyze Avito ads to track pricing trends, supply-demand, and popular items across regions.
• Competitor Analysis: Monitor competing sellers and their offerings to inform sales and pricing strategies.
• Lead Generation: Extract seller contact info and business listings to build outreach lists for sales or marketing.
• Business Intelligence: Feed Avito data into dashboards or AI models to forecast market shifts or optimize inventory.
• Industry Analysis: Gain insights in specific sectors – for example, real estate analysts can track rental/housing ads, and auto dealers can monitor used car listings.

Is it legal to scrape Avito? / Легально ли парсить Авито?

Our scrapers are designed to be ethical and do not extract any private user data. They only extract data that is publicly available on the website. We believe that our scrapers, when used for ethical purposes, are safe. However, you should be aware that your results could contain personal data, which is protected by regulations like GDPR. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure, consult with your lawyers. You can also read our blog post on the legality of web scraping.

Need More Data? / Нужно больше данных?

If you require additional data fields or encounter any issues with data extraction, please open an issue. We will review your request and work to incorporate the necessary changes.