# Algolia API Scraper (`rl1987/algolia-api-scraper`) Actor

Generalised scraper for any Algolia-powered search index: scrape hits or discover facets and their values directly via the Algolia Search REST API.

- **URL**: https://apify.com/rl1987/algolia-api-scraper.md
- **Developed by:** [R.L.](https://apify.com/rl1987) (community)
- **Categories:** Other
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 scraped items

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

A **generalised [Algolia](https://www.algolia.com/) API scraper** for extracting search results and discovering facets from any website that uses Algolia as its search backend — e-commerce catalogs, documentation sites, marketplaces, directories, and more. Instead of scraping rendered HTML, this Actor talks directly to the site's Algolia index over the [Algolia Search REST API](https://www.algolia.com/doc/rest-api/search/), so extraction is fast, reliable, and returns the exact structured data the site itself uses. Run it on the [Apify platform](https://apify.com) for scheduling, API access, and monitoring without managing any infrastructure.

### Why use Algolia API Scraper?

Thousands of sites (documentation portals, online stores, job boards, real estate listings) use Algolia under the hood for instant search. Once you have the site's Algolia `applicationId`, a search-only `apiKey`, and the `indexName` (all visible in the site's browser network requests), you can:

- Pull the full catalog of a site without crawling pages one by one.
- Apply the exact same filters, facets, and query syntax the site's own search UI uses.
- Discover which attributes are filterable/facetable on an index — and what values exist for each — before building a larger scraping or monitoring pipeline.
- Feed structured, ready-to-use JSON straight into spreadsheets, databases, or downstream automations via Apify's integrations.

### How to use Algolia API Scraper

1. Open the site you want to extract data from and inspect its network requests (DevTools → Network → filter for `algolia.net` or `algolianet.com`) while performing a search.
2. Note down the **Application ID** (from the request host, e.g. `ABC123DEF-dsn.algolia.net`), the **API Key** (header `X-Algolia-API-Key`), and the **index name** (in the request URL or body).
3. Paste those three values into the Actor's input, optionally along with a search query, filters, and facets.
4. Click **Start** and let the Actor page through the index and push every hit to the dataset — or enable **Discover facets only** to get a fast overview of the index's facetable attributes first.

### Input

Configure the Actor via the **Input** tab. Key fields:

| Field | Description |
| --- | --- |
| `algoliaAppId` | Algolia Application ID |
| `algoliaApiKey` | Search-only Algolia API key |
| `indexName` | Name of the index to query |
| `searchQuery` | Free-text query (empty = match all) |
| `filters` | Algolia filter expression, e.g. `category:Shoes AND price < 50` |
| `facetFilters` / `numericFilters` | Structured filter arrays |
| `facets` | Facet attributes to get counts for; `["*"]` discovers all of them |
| `discoverFacetsOnly` | Skip item scraping, only return facet values/counts |
| `hitsPerPage` / `maxItems` | Pagination size and overall item cap |
| `extraQueryParams` | Any other raw Algolia `SearchParams` field, for advanced use |

Example input for facet discovery:

```json
{
    "algoliaAppId": "ABC123DEF",
    "algoliaApiKey": "aaaabbbbccccdddd11112222",
    "indexName": "products",
    "facets": ["*"],
    "discoverFacetsOnly": true
}
```

### Output

Each scraped record is pushed to the dataset exactly as Algolia returns it (plus its `objectID`). Example item:

```json
{
    "objectID": "12345",
    "title": "Running Shoes",
    "category": "Shoes",
    "price": 79.99
}
```

When `discoverFacetsOnly` is enabled, the dataset instead contains a single item with the discovered facets:

```json
{
    "facets": {
        "category": { "Shoes": 512, "Shirts": 340 },
        "brand": { "Nike": 210, "Adidas": 180 }
    },
    "facets_stats": {},
    "nbHits": 852
}
```

The same facet data is also saved to the key-value store under the `FACETS` key for quick reference. You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

### Data table

| Field | Description |
| --- | --- |
| `objectID` | Algolia's unique identifier for the record |
| *(all other fields)* | Whatever attributes the source index stores — varies per site |

### Cost estimation

This Actor makes lightweight JSON API calls (no browser rendering), so it is inexpensive to run. A full scrape of an index with tens of thousands of records typically finishes in well under a minute of compute time. Facet discovery runs a single request and is essentially free. Apify's free tier is generally enough for occasional runs; larger or scheduled scrapes will consume more compute units proportional to the number of pages fetched.

### Tips and advanced options

- Set `hitsPerPage` to the maximum (1000) to minimize the number of requests.
- Use `maxItems` to cap a run when you only need a sample.
- Use `discoverFacetsOnly` first on a new index to understand what filters are available, then build a targeted `filters`/`facetFilters` query for the real scrape.
- Pass any additional Algolia parameter (e.g. `distinct`, `restrictSearchableAttributes`, `optionalFilters`) via `extraQueryParams`.

### FAQ, disclaimers, and support

This Actor only queries indexes using credentials you supply — it does not bypass authentication or access private data. Only use it against indexes and API keys you are authorized to query, and respect the target site's Terms of Service. Algolia may change or rotate search-only keys; if the Actor stops working, re-check the site's current `applicationId`/`apiKey`/`indexName`. For issues or feature requests, use the Actor's Issues tab.

# Actor input Schema

## `algoliaAppId` (type: `string`):

The Algolia Application ID (find it in the site's network requests as `X-Algolia-Application-Id` or in the request URL host `<appId>-dsn.algolia.net`). Defaults to Hacker News Search's own public Algolia integration (hn.algolia.com).

## `algoliaApiKey` (type: `string`):

A search-only Algolia API key (header `X-Algolia-API-Key`). Most public sites expose this in their frontend JS bundle or network requests. Defaults to Hacker News Search's public search-only key.

## `indexName` (type: `string`):

Name of the Algolia index to query. Defaults to Hacker News Search's "Item\_dev" index (stories and comments).

## `searchQuery` (type: `string`):

Free-text query string. Leave empty to match all records (browse mode).

## `filters` (type: `string`):

Algolia filter expression, e.g. `category:Shoes AND price < 50`. See https://www.algolia.com/doc/api-reference/api-parameters/filters/

## `facetFilters` (type: `array`):

Algolia facetFilters array, e.g. \["category:Shoes", \["color:Red", "color:Blue"]]. Each entry is a string or a nested array (OR group).

## `numericFilters` (type: `array`):

Algolia numericFilters array, e.g. \["price>=10", "price<=100"].

## `facets` (type: `array`):

Facet attribute names to request counts for. Use \["\*"] to discover every facetable attribute and its values.

## `maxValuesPerFacet` (type: `integer`):

Maximum number of facet values returned per facet.

## `discoverFacetsOnly` (type: `boolean`):

When enabled, the Actor makes a single hitsPerPage=0 request and only outputs the discovered facets and their values/counts, without scraping any search result items.

## `attributesToRetrieve` (type: `array`):

Limit which attributes are returned per hit. Leave empty to retrieve all attributes (`*`).

## `hitsPerPage` (type: `integer`):

Number of hits fetched per Algolia request page.

## `maxItems` (type: `integer`):

Maximum number of items to scrape in total. 0 = unlimited (scrape all pages Algolia returns).

## `extraQueryParams` (type: `object`):

Advanced: any additional Algolia SearchParams fields (e.g. distinct, optionalFilters, restrictSearchableAttributes) merged directly into the request body.

## Actor input object example

```json
{
  "algoliaAppId": "UJ5WYC0L7X",
  "algoliaApiKey": "28f0e1ec37a5e792e6845e67da5f20dd",
  "indexName": "Item_dev",
  "searchQuery": "arcteryx",
  "filters": "",
  "facetFilters": [],
  "numericFilters": [],
  "facets": [
    "*"
  ],
  "maxValuesPerFacet": 100,
  "discoverFacetsOnly": false,
  "attributesToRetrieve": [],
  "hitsPerPage": 1000,
  "maxItems": 0,
  "extraQueryParams": {}
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "algoliaAppId": "UJ5WYC0L7X",
    "algoliaApiKey": "28f0e1ec37a5e792e6845e67da5f20dd",
    "indexName": "Item_dev",
    "searchQuery": "arcteryx"
};

// Run the Actor and wait for it to finish
const run = await client.actor("rl1987/algolia-api-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "algoliaAppId": "UJ5WYC0L7X",
    "algoliaApiKey": "28f0e1ec37a5e792e6845e67da5f20dd",
    "indexName": "Item_dev",
    "searchQuery": "arcteryx",
}

# Run the Actor and wait for it to finish
run = client.actor("rl1987/algolia-api-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "algoliaAppId": "UJ5WYC0L7X",
  "algoliaApiKey": "28f0e1ec37a5e792e6845e67da5f20dd",
  "indexName": "Item_dev",
  "searchQuery": "arcteryx"
}' |
apify call rl1987/algolia-api-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rl1987/algolia-api-scraper"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/r1TznFP4fd6AGdJNZ/builds/sgoA1kjfBLl2tN0x0/openapi.json