# SourceLens AI: Extract Any Data From Websites With Your Own LLM (`onsq/sourcelens-ai`) Actor

Describe the fields you want in plain English and get clean JSON from any website. Bring your own OpenAI or Claude key. Every value is backed by a quote from the page, so made-up answers are dropped.

- **URL**: https://apify.com/onsq/sourcelens-ai.md
- **Developed by:** [Owen Eskew](https://apify.com/onsq) (community)
- **Categories:** AI, Developer tools
- **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 ai-extracted pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## SourceLens AI: Extract Any Data From Websites With Your Own LLM

**Stop writing scrapers. Describe the data you want in plain English, and SourceLens AI returns it as clean JSON from any public website.** No CSS selectors, no XPath, no code, and nothing breaks when a site changes its layout.

Bring your own **OpenAI** or **Anthropic (Claude)** API key. You choose the model and you pay the AI provider directly at their prices, with no markup. SourceLens AI handles fetching, cleaning, crawling, validation and delivery.

**Every value is backed by evidence.** The AI must quote the exact page text each answer came from, and SourceLens AI checks that quote against the real page. Values it can't back up are dropped instead of invented.

Built and maintained by **ONSQ Enterprises**.

### Why SourceLens AI?

- 🗣️ **Plain-English fields.** Write `"price": "Price as a number, without the currency sign"` and you're done. It works on any page layout, including ones you've never seen.
- 🛡️ **Hallucination guard.** Each value comes with a source quote that is verified against the page. You can see exactly why every value is there.
- 📋 **Lists become rows.** Turn on **Each page lists several items** and a product grid, job board or team page becomes one clean row per item.
- 🕸️ **Crawl whole sections.** Follow links on the same site, for example from a category page to every product page, with depth, page limits and URL filters.
- 🔑 **Your key, your model, your cost.** Use GPT or Claude, from small fast models to the most capable ones. Your key is stored encrypted and is only ever sent to the provider you pick.
- 💸 **Fair billing.** You pay a small fee per page that returns data. Pages that fail, are blocked, or contain none of your fields are free.
- 📤 **Ready to use anywhere.** Export to JSON, CSV, Excel or Google Sheets, or call it from Make, Zapier, n8n, LangChain or the Apify API.

### What people use it for

- **E-commerce and pricing:** product names, prices, stock status and ratings from any store, even without a dedicated scraper.
- **Lead generation:** company names, services, locations and contact pages from business websites and directories.
- **Recruiting:** job titles, salaries, locations and requirements from career pages.
- **Real estate:** listing prices, addresses, bedrooms and features.
- **Research and monitoring:** facts, dates, figures and claims from articles, reports and documentation, each with a quote to cite.
- **AI agents:** give your agent a reliable "read this page and give me X" tool through the Apify API or MCP.

### How to use it

1. Click **Try for free**.
2. Paste the URLs of the pages you want data from.
3. List the fields you want, each with a name and an optional description and type.
4. Turn on **Each page lists several items** if a page shows many products, jobs or people.
5. Choose **OpenAI** or **Anthropic**, and paste your API key.
6. Click **Start**.

#### Example: every book on a category page

```json
{
  "urls": ["https://books.toscrape.com/"],
  "multipleItems": true,
  "fields": [
    { "name": "title", "description": "Book title" },
    { "name": "price", "description": "Price as a number, without the currency sign", "type": "number" },
    { "name": "inStock", "description": "True if the book is in stock", "type": "boolean" }
  ],
  "provider": "anthropic",
  "apiKey": "sk-ant-..."
}
```

#### Example: company facts from many websites

```json
{
  "urls": ["https://www.python.org/psf/", "https://www.apache.org/"],
  "fields": [
    { "name": "organization", "description": "Name of the organization" },
    { "name": "mission", "description": "One-sentence mission statement" },
    { "name": "founded", "description": "Year founded", "type": "integer" },
    { "name": "programs", "description": "Main programs or projects", "type": "list" }
  ],
  "provider": "openai",
  "apiKey": "sk-..."
}
```

Field types are `string` (the default), `number`, `integer`, `boolean`, and `list`, which is a list of text values. You can add **Extra instructions** such as "Prices are in euros" or "Ignore sponsored listings".

### Output

Each row holds the extracted fields in `data`, with the supporting quotes in `evidence`:

```json
{
  "url": "https://books.toscrape.com/",
  "status": "ok",
  "itemIndex": 0,
  "data": { "title": "A Light in the Attic", "price": 51.77, "inStock": true },
  "evidence": {
    "title": { "quote": "A Light in the Attic", "quoteFound": true, "valueInQuote": true },
    "price": { "quote": "£51.77", "quoteFound": true, "valueInQuote": true },
    "inStock": { "quote": "In stock", "quoteFound": true, "valueInQuote": false }
  },
  "aiProvider": "anthropic",
  "aiModel": "claude-opus-5-5",
  "billed": true
}
```

- `quoteFound` means the quote really appears on the page.
- `valueInQuote` means the value itself appears inside the quote. It is `false` for values the AI had to interpret, like `true` for "In stock".
- Values whose quote can't be found are set to `null`, and a note is added to `warnings`.
- The **Extracted data** table view shows each field as its own column, and CSV and Excel exports do the same.

A run summary with page counts and your total AI token usage is saved in the key-value store as `SUMMARY`.

### How much does it cost?

There are two separate costs.

1. **SourceLens AI fee.** A small charge for each page that returns data, shown on the **Pricing** tab. A listing page with 50 items counts as one page. Failed and empty pages are free.
2. **Your AI provider's charge.** Billed to your own OpenAI or Anthropic account at their normal token prices. SourceLens AI adds no markup.

Your provider charge depends mostly on the model you pick and how much page text is sent. **Max page characters sent to the AI** defaults to 30,000 characters, which is roughly 7,500 tokens. Rough provider cost per page at that size:

| Model | Approximate cost per page |
| --- | --- |
| `claude-haiku-5-5` or `gpt-6-luna` | about $0.001 |
| `gpt-6.1-sol` (OpenAI default) | about $0.02 |
| `claude-opus-5-5` (Anthropic default) | about $0.05 |

These are estimates. Check your provider's current pricing, and set a spending limit in your provider account.

### Is my API key safe?

- The key field is a **secret input**. Apify stores it encrypted, and it is hidden from the actor's developer and from your run's public input.
- The key is only sent to `api.openai.com` or `api.anthropic.com`, never anywhere else.
- It is never written to logs, results or the run summary.
- As good practice, create a separate key just for SourceLens AI with a monthly spending limit.

The key is checked before any page is fetched. A wrong key stops the run right away with a clear message. Running out of credit or naming a model that doesn't exist also stops the run immediately instead of failing page by page.

### Limits and good to know

- SourceLens AI reads the HTML a server returns. It does not run JavaScript, so pages that only build their content in the browser may come back nearly empty.
- Sites with strong bot protection may refuse the request. Those pages are free error rows.
- Quote checking proves a value is grounded in the page text. It doesn't prove the AI interpreted that text correctly, so spot-check results for important work.
- Very long pages are cut to your character limit before they are sent, so items near the end of a huge page can be missed. Raise the limit, or crawl the individual pages instead.

### FAQ

**Which model should I use?**
Start with the default. For large, simple jobs like product grids, a small model such as `claude-haiku-5-5` or `gpt-6-luna` is far cheaper and usually just as accurate. Move to a larger model for messy pages or fields that need judgment.

**Do I need SourceLens AI or plain SourceLens?**
If you just need page text as Markdown for an LLM or RAG pipeline, or you already know the CSS selectors, [SourceLens](https://apify.com/onsq/sourcelens) is cheaper and doesn't need an AI key. Use SourceLens AI when you want specific fields without writing selectors.

**Why is a value empty when I can see it on the page?**
Either the AI couldn't find a quote for it, or the page builds that part with JavaScript. Look at `warnings` for that row. For fields that summarise or classify a page, turn off **Require a page quote for every value**.

**Is it legal to scrape websites?**
SourceLens AI only fetches public pages and respects robots.txt by default. Results can still contain personal or copyrighted data. Make sure you have a legitimate reason to collect it, follow each website's terms, and consult a lawyer if you are unsure.

**Something is broken or missing.**
Open an issue on the **Issues** tab. Feature requests are welcome.

# Actor input Schema

## `urls` (type: `array`):

Public webpages to extract from, up to 1,000 per run. Missing https:// is added for you.

## `fields` (type: `array`):

Describe each field in plain English. Each needs a name; description and type are optional. Types: string (default), number, integer, boolean or list (a list of text values).

## `multipleItems` (type: `boolean`):

Turn on for listing pages such as products, jobs, people or articles. You get one row per item instead of one row per page.

## `provider` (type: `string`):

Whose API key you are using. Model costs are billed by that provider to your own account.

## `apiKey` (type: `string`):

Your OpenAI key (starts with sk-) or Anthropic key (starts with sk-ant-). Stored encrypted.

## `model` (type: `string`):

Optional. Leave empty for the default: gpt-6.1-sol on OpenAI or claude-opus-5-5 on Anthropic. Cheaper choices include gpt-6-luna and claude-haiku-5-5.

## `instructions` (type: `string`):

Optional guidance for the AI, for example: Prices are in euros. Ignore sponsored listings.

## `requireEvidence` (type: `boolean`):

Recommended. Values the AI cannot back with a quote found on the page are set to empty. Turn off only for fields that summarise or classify the page.

## `maxItemsPerPage` (type: `integer`):

Only used when each page lists several items.

## `includeMarkdown` (type: `boolean`):

Adds the clean Markdown of each page to the results.

## `maxCrawlDepth` (type: `integer`):

0 extracts only the URLs above. 1 also follows links found on those pages, and so on. Crawling stays on the same website.

## `maxCrawlPages` (type: `integer`):

Upper limit on extra pages added by crawling, on top of your start URLs.

## `includeUrlPattern` (type: `string`):

Optional regular expression. Example: /product/

## `excludeUrlPattern` (type: `string`):

Optional regular expression. Example: /(login|cart)/

## `maxInputChars` (type: `integer`):

Longer pages are cut to this length before they are sent, which caps your model cost per page.

## `concurrency` (type: `integer`):

How many pages to process at the same time. Lower it if your AI provider rate-limits you.

## `timeoutSeconds` (type: `integer`):

Give up fetching a page after this many seconds.

## `respectRobotsTxt` (type: `boolean`):

Skip pages that the website's robots.txt asks crawlers not to visit.

## Actor input object example

```json
{
  "urls": [
    "https://books.toscrape.com/"
  ],
  "fields": [
    {
      "name": "title",
      "description": "Book title"
    },
    {
      "name": "price",
      "description": "Price as a number, without the currency sign",
      "type": "number"
    },
    {
      "name": "inStock",
      "description": "True if the book is in stock",
      "type": "boolean"
    }
  ],
  "multipleItems": true,
  "provider": "openai",
  "requireEvidence": true,
  "maxItemsPerPage": 50,
  "includeMarkdown": false,
  "maxCrawlDepth": 0,
  "maxCrawlPages": 50,
  "maxInputChars": 30000,
  "concurrency": 3,
  "timeoutSeconds": 30,
  "respectRobotsTxt": true
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://books.toscrape.com/"
    ],
    "fields": [
        {
            "name": "title",
            "description": "Book title"
        },
        {
            "name": "price",
            "description": "Price as a number, without the currency sign",
            "type": "number"
        },
        {
            "name": "inStock",
            "description": "True if the book is in stock",
            "type": "boolean"
        }
    ],
    "multipleItems": true
};

// Run the Actor and wait for it to finish
const run = await client.actor("onsq/sourcelens-ai").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://books.toscrape.com/"],
    "fields": [
        {
            "name": "title",
            "description": "Book title",
        },
        {
            "name": "price",
            "description": "Price as a number, without the currency sign",
            "type": "number",
        },
        {
            "name": "inStock",
            "description": "True if the book is in stock",
            "type": "boolean",
        },
    ],
    "multipleItems": True,
}

# Run the Actor and wait for it to finish
run = client.actor("onsq/sourcelens-ai").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://books.toscrape.com/"
  ],
  "fields": [
    {
      "name": "title",
      "description": "Book title"
    },
    {
      "name": "price",
      "description": "Price as a number, without the currency sign",
      "type": "number"
    },
    {
      "name": "inStock",
      "description": "True if the book is in stock",
      "type": "boolean"
    }
  ],
  "multipleItems": true
}' |
apify call onsq/sourcelens-ai --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,onsq/sourcelens-ai"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/fULhSWMja4cbBRTMj/builds/wfrC0ejzY96GyXYvG/openapi.json