# NeuroScraper (`arcolab/neuroscraper`) Actor

NeuroScraper: An AI-powered web scraper that parses any website without brittle regex or selectors. By leveraging advanced LLMs to extract data directly from structured HTML, it completely bypasses modern code obfuscation, layout shifts, and anti-scraping traps.

- **URL**: https://apify.com/arcolab/neuroscraper.md
- **Developed by:** [Artur Cosicov](https://apify.com/arcolab) (community)
- **Categories:** Automation, Developer tools, AI
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $3.00 / 1,000 results

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## 🤖 NeuroScraper: Next-Gen Universal AI Web Scraper

**NeuroScraper** is a production-grade, AI-powered data extraction engine built for the modern web. It completely eliminates the need for fragile CSS selectors, brittle regex patterns, or custom scraping code.

By leveraging advanced Large Language Models (LLMs) and a custom semantic HTML compression pipeline, NeuroScraper reads web pages exactly like a human being. It easily bypasses layout changes, code obfuscation, randomized Tailwind/React classes, and anti-scraping mechanisms.

***

### 🚀 Key Features

- **100% Code Obfuscation Immune**: Random IDs or dynamic class names don't matter anymore. The AI understands the context and meaning of the data natively.
- **Dual Processing Engine**:
  - ⚡ **Fast Mode (HTTP)**: Lightning-fast, cost-effective extraction for standard websites and lightweight pages.
  - 🌐 **Advanced Mode (Headless Browser)**: Uses a powerful Playwright-driven hidden browser to render heavy JavaScript-based applications and crack anti-bot shields (Amazon, Booking, eBay, etc.).
- **94% Semantic HTML Compression**: Heavy pages from retail giants contain megabytes of hidden JSON telemetry. Our custom filters strip the boilerplate noise down to ~50-70KB, making extraction extremely token-efficient and blazing fast.
- **Lazy-Loading Protection**: The browser engine automatically triggers dynamic content rendering and lazy-loaded items (like gallery photos) before feeding code to the AI.
- **Dynamic Visual Fields Builder**: Zero programming required. No hardcoded JSON strings. Just define your custom key and describe what you are looking for in plain English.

***

### 🛠 How to Use

1. **Set URLs**: Provide the link(s) to the specific pages you want to scrape (e.g., a product page or hotel listing).
2. **Choose Parser Mode**: Select **Fast Mode** for simple layouts or **Advanced Mode** for heavy, protected JavaScript platforms.
3. **Define Fields to Extract**: Use the visual schema table to create your data map. For example:
   - Field Name: `title` ➡️ What to look for: `Full product name`
   - Field Name: `price` ➡️ What to look for: `Current buying price as integer`
   - Field Name: `photos` ➡️ What to look for: `Array of main product image URLs`
4. **Run and Download**: Click **Start**. Once completed, grab your clean, structured dataset in **JSON, CSV, or Excel** format.

***

### 💳 Pricing & Consumption

NeuroScraper uses a highly cost-effective **Pay-per-result** monetization structure. You only pay for successful extractions:

- **Actor Start**: `$0.00005` (Virtually free initialization)
- **Per Successful Result**: `$0.003` (Only **$3.00 per 1,000 successfully scraped pages**)

*Note: In addition to the result price, standard Apify platform compute unit costs apply based on the chosen mode (Fast Mode consumes significantly fewer resources than Advanced Browser Mode).*

***

### 📊 Sample Output

No matter how chaotic the source HTML code is, NeuroScraper converts it into clean, typed, and structured fields

***

### 🔒 Enterprise Infrastructure

NeuroScraper runs on a dedicated high-throughput server farm powered by private **NVIDIA RTX GPUs with ultra-fast VRAM clusters and native vLLM optimization**. Thanks to continuous Automatic Prefix Caching (APC), concurrent batch processing takes seconds, delivering enterprise-grade extraction speeds even under heavy multi-tenant scale.

*Brought to you by ArCoLab Studio. Scraping the web with pure neural intelligence.*

# Actor input Schema

## `startUrls` (type: `array`):

List of URLs to scrape

## `crawlerType` (type: `string`):

Choose 'Fast (HTTP)' for regular websites, or 'Advanced (Browser)' for JavaScript-heavy or protected sites.

## `extractionFields` (type: `array`):

Define what data you want the AI to extract from the page.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "crawlerType": "cheerio",
  "extractionFields": [
    {
      "fieldName": "title",
      "description": "The main title of the page"
    }
  ]
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://apify.com"
        }
    ],
    "crawlerType": "cheerio",
    "extractionFields": [
        {
            "fieldName": "title",
            "description": "The main title of the page"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("arcolab/neuroscraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://apify.com" }],
    "crawlerType": "cheerio",
    "extractionFields": [{
            "fieldName": "title",
            "description": "The main title of the page",
        }],
}

# Run the Actor and wait for it to finish
run = client.actor("arcolab/neuroscraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://apify.com"
    }
  ],
  "crawlerType": "cheerio",
  "extractionFields": [
    {
      "fieldName": "title",
      "description": "The main title of the page"
    }
  ]
}' |
apify call arcolab/neuroscraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,arcolab/neuroscraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/cKODA4Cpm2MhxMgyH/builds/PhDmMPc2vYgmDtGSB/openapi.json