# Profi.ru specialists scraper (`deep_eruption/profi-ru-scraper`) Actor

- **URL**: https://apify.com/deep\_eruption/profi-ru-scraper.md
- **Developed by:** [Via K](https://apify.com/deep_eruption) (community)
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Profi.ru scraper — headless Apify actor

A browser-based (Playwright) actor for Profi.ru — a direct HTTP scrape is
anti-bot-blocked, so this runs a real browser over Apify residential proxies. Its
output matches `src/modules/profi/parser.py::parse_apify_item` (`url`, `name`,
`specialty`, `location`, `rate`, `phone`).

### Deploy (one time)

```bash
npm i -g apify-cli
apify login                       # paste your Apify API token
cd apify-actors/profi-scraper
apify push                        # builds + uploads; prints the actor id (user/name)
```

Then in the app's `.env`:

```
PROFI_APIFY_ACTOR=<your-user>/profi-ru-scraper
```

and recreate the `app` container. The Profi `apify` mode will then appear in the
search panel.

### How it scrapes

Three phases: resolve the query to a taxonomy path, read the PROFILES listing, then open
each card's profile for what the listing does not carry (the full review list and the
education/experience block).

The listing serves 20 cards and appends the next 20 per click on
`button[data-shmid="pagination_next"]` ("Показать ещё 20") — there is no page-number URL,
so `maxItems` above 20 is only reachable by clicking through. Cards hydrate one by one,
so the actor waits for the count to stop climbing before it extracts or paginates;
extracting on the first card that appeared used to return 3 of 20.

**Prices come from the card, not from the profile.** The card lists only the services
filed under the searched trade and prints the real total next to them ("Все услуги и цены
(15)" — that link opens the full profile). The profile page lists every trade the
specialist sells, led by whatever they do most, which is how a search for "электрик" came
back with a plumbing price list. So `prices` is the card's rows and `pricesTotal` the count
from that link.

The profile pass also clicks "Все услуги и цены (34)" — it expands the table in place, no
modal and no navigation — and returns every row as `pricesAll`, each tagged with the trade
it sits under (`group`). The app shows `prices` by default and `pricesAll` behind a toggle,
so the full portfolio is available without the searched category drowning in it.

Selectors are taken from the live markup and use `data-shmid` / `data-testid` hooks
(`profileCard`, `profileIndicators_ratingBlock`, `profileOIO`, `priceRow`, `reviewItem`,
`pagination_next`, …). Class names are hashed CSS modules and are never matched whole.

### Output schema

Two files, because the Console counts them separately:

- `.actor/dataset_schema.json` — what the dataset looks like: two table views
  (specialist overview, prices/reviews) instead of raw JSON. It ships `fields` without
  `required`, so a specialist whose profile omits half the fields still lands in the
  dataset.
- `.actor/output_schema.json` — what the run's **Output** tab points at
  (`{{links.apiDefaultDatasetUrl}}/items`). This is the one the Store's "Add Output
  schema" gate checks; a dataset schema alone leaves it red.

### Cost and speed

One specialist = one profile page load in a headless browser over residential proxies:
a 50-item run took 8 minutes and $0.40. The app therefore caps Profi at 20 items
(`src/modules/profi/parser.py::MAX_ITEMS`) — about $0.16 and one "show more" click.

Images, media, fonts and the analytics hosts are aborted in `preNavigationHooks`: the
scraper reads image URLs from the markup and never the pixels, so downloading them only
spends residential-proxy GB — the quota a browser actor exhausts first. `fullProfiles:
false` is the bigger lever: listing only, no per-specialist page, a run in well under a
minute, at the cost of the full review list and the education block.

### Known limits

**Relevance is Profi's, not ours.** Profi returns whoever it matched to the category,
and the price row for the searched service is generated even when the specialist does
something else — a violin tutor comes back for "тестировщики" with a "Тестировщики"
price row. Cards carrying the "Нет отзывов по вашей услуге" warning are the tell; that
warning is deliberately not surfaced as a trust badge.

**Phones are not scrapeable.** Profi gates contacts behind task creation, so no phone
appears anywhere and `phone` is always null.

**The query is resolved by Profi, not by us.** Profi has no keyword-search endpoint —
every trade is a taxonomy path — and `smart-search` cannot be called directly, since the
page before it hands it the answer. So the actor types the query into Profi's own search
box and reads the path off wherever the site navigates. That keeps resolution correct
when Profi reshuffles its taxonomy; the only thing to maintain is `SEARCH_BOX`, and the
log line `search box resolved "…" to …` says whether it worked.

`/smart-search/` in the log is a waiting room, not a result: Profi keeps matching there
for about ten seconds and then redirects to the wizard with `?pathname=<category>`. The
actor waits for that redirect (or a taxonomy path in the URL), which is what the log line
`search box resolved "…" to …` reports. A query Profi cannot classify parks at
`/raznoe/raznoe_other/` — treated as a failed resolve, since its listing is every trade
at once.

`CATEGORY_BY_KEYWORD` is a fallback used only when the search box cannot be driven, and
`categoryPath` overrides everything:

```bash
apify run -i '{ "query": "сварщик", "categoryPath": "/remont/svarochnye-raboty/", "region": "Кемерово" }'
```

**Region mapping is partial.** `region` is matched against the cities the app targets
(see `GEO_BY_CITY`); anything unknown falls back to Moscow, Profi's own default locale.

### Local test

```bash
apify run -i '{ "query": "электрик", "region": "Москва", "maxItems": 10 }'
```

# Actor input Schema

## `query` (type: `string`):

Search keywords, e.g. 'электрик'

## `region` (type: `string`):

City/region, e.g. 'Москва'

## `maxItems` (type: `integer`):

Stop after this many specialists

## `categoryPath` (type: `string`):

Profi taxonomy path, e.g. /remont/electromontajnye-raboty/ — overrides keyword mapping

## `profileUrls` (type: `array`):

Scrape these profile pages directly instead of searching (for testing)

## Actor input object example

```json
{
  "region": "",
  "maxItems": 50,
  "profileUrls": []
}
```

# Actor output Schema

## `specialists` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("deep_eruption/profi-ru-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("deep_eruption/profi-ru-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call deep_eruption/profi-ru-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,deep_eruption/profi-ru-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/64aotwTpy9uxPWnJo/builds/8BBaTAAOt66YwaDbl/openapi.json