# Instant Web Scraper - Any List or Table to Rows (`memo23/instant-web-scraper`) Actor

Paste any URL and get the page's main list or table back as rows. Columns are detected and named for you, next pages are followed automatically, and one run scrapes many URLs. No selectors, no code, no browser. Cookies work for logged-in pages. Export to JSON, CSV or Excel.

- **URL**: https://apify.com/memo23/instant-web-scraper.md
- **Developed by:** [Muhamed Didovic](https://apify.com/memo23) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.50 / 1,000 row scrapeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Instant Web Scraper: turn any list or table into rows

Paste a URL. The Actor finds the page's main repeating list (product grid, search results, directory, ranking, HTML table), names the columns, follows the next pages, and hands you clean rows in JSON, CSV or Excel. You don't write any selectors or code, and there's no browser to babysit.

### Why use Instant Web Scraper

- **One run, done.** Other instant scrapers make you run a "find tables" pass, write down column numbers, then run again. This one picks the best list and names the columns on the first run.
- **Readable column names.** Columns are named from the page itself: table headers (`teamName`, `wins`), class names (`price`, `rating`), and link, image and title attributes (`linkUrl`, `thumbnail`, `linkTitle`).
- **Pagination without setup.** It follows `rel=next` links, "Next" / "›" / "»" buttons in about 20 languages, and numbered pagers. It also counts up `?page=N` and `/page/N` when the page shows no link.
- **Many URLs per run.** Each start URL is scraped and paginated on its own, five at a time.
- **Logged-in pages.** Paste a Cookie-Editor export and the cookies go out with every request to that domain.
- **Cheap and fast.** It reads plain HTML with a real browser TLS fingerprint and never starts a browser. A page takes about a second.

### Overview

| | |
|---|---|
| Input | One or more page URLs |
| Output | One row per list item, plus `pageUrl`, `pageNumber`, `position` |
| Pagination | Automatic (next links, numbered pagers, page-number URLs) or your own CSS selector |
| Login | Cookies (Cookie-Editor JSON, `name=value; ...` string, or `{name: value}`) |
| Rendering | Server-rendered HTML only (see FAQ) |

### Supported inputs

Any page whose list is in the HTML the server sends, for example:

- E-commerce search and category pages (`/search?q=...`, `/category/shoes`)
- Directories and listings (business, lawyer, job, real estate)
- Rankings, leaderboards, Wikipedia and other HTML tables
- Blog archives, news indexes, forum thread lists
- WordPress, Shopify and other CMS listing pages

### Use cases

- Pull a competitor's catalogue (name, price, URL, image) into a spreadsheet
- Build a lead list from a directory page
- Track a ranking or a price table every day with a scheduled run
- Grab a table from a reference page without copy-paste clean-up
- Feed product or listing rows to an AI agent through the Apify API or MCP

### How it works

1. Downloads the page with a browser-grade TLS fingerprint, through a proxy.
2. Scores every group of sibling elements that share a shape (same tag and same shared classes). More items, more text, more fields, images and links score higher. Menus, footers, pagers, filter lists and reference lists score lower.
3. Reads every item of the winning list: element text, link URLs, link titles, image URLs (lazy-loaded `data-src` and `srcset` included), alt text, `datetime` values and table cells.
4. Lines the values up into columns across items and names them.
5. Finds the next page and repeats, matching the same list on each page. It stops at your page or row limit, or when a page brings no new rows.

### Input configuration

```json
{
    "startUrls": ["https://books.toscrape.com/"],
    "maxItems": 100,
    "maxPages": 5
}
```

| Field | What it does |
|---|---|
| `startUrls` | Pages to scrape. Each one is paginated separately. |
| `action` | `scrape` (rows) or `findTables` (one preview row per list on each page, not charged per row). |
| `maxItems` | Max rows per start URL. Default 1000. |
| `maxPages` | Max pages per start URL, start page included. Default 20. |
| `tableNumber` | Scrape the 2nd, 3rd, ... detected list instead of the best one. |
| `itemSelector` | CSS selector for one row, e.g. `li.product`. Skips detection. |
| `renameColumns` | Rename detected columns (`linkTitle` → `title`). |
| `keepOnlyRenamedColumns` | Output only the columns you renamed. |
| `dropConstantLabels` | Drop columns holding the same short label in every row ("Read more"). Default on. |
| `nextPageSelector` | CSS selector of the next-page link, when auto-detection picks the wrong one. |
| `autoIncrementPage` | Count up `?page=N` / `/page/N` when there is no next link. Default on. |
| `delayMs` | Pause between pages of the same URL. |
| `cookies` | Cookies for logged-in pages. |
| `customHeaders` | Extra request headers. |
| `proxy` | Pick a proxy country or group. Leave empty for the default route. |

#### Picking a different list

Run once with `"action": "findTables"`. Each output row describes one list: its `tableNumber`, `itemCount`, `columns`, three `sampleRows` and a ready-made `itemSelector`. Then run again with `tableNumber` (or paste the `itemSelector`).

### Output overview

One dataset row per list item. The column names come from the page, so they differ from site to site. Every row also carries:

| Field | Meaning |
|---|---|
| `pageUrl` | The page the row was read from |
| `pageNumber` | 1 for the start page, 2 for the next, ... |
| `position` | Position of the item on its page |

### Output samples

From `https://books.toscrape.com/`, with `linkTitle → title`, `priceColor → price`, `linkUrl → url` and `keepOnlyRenamedColumns` on:

```json
{
    "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
    "title": "A Light in the Attic",
    "price": "£51.77",
    "pageUrl": "https://books.toscrape.com/",
    "pageNumber": 1,
    "position": 1
}
```

From an HTML table (`https://www.scrapethissite.com/pages/forms/`), page 3:

```json
{
    "teamName": "Los Angeles Kings",
    "year": "1992",
    "wins": "39",
    "losses": "35",
    "win": "0.464",
    "goalsForGf": "338",
    "goalsAgainstGa": "340",
    "column9": "-2",
    "pageUrl": "https://www.scrapethissite.com/pages/forms/?page_num=3",
    "pageNumber": 3,
    "position": 1
}
```

### Key output fields

| Column name pattern | Comes from |
|---|---|
| Table header in camelCase (`teamName`) | `<th>` text of the column |
| Class name in camelCase (`price`, `ratingCount`) | Text of an element with that class |
| `...Url` | `href` of a link |
| `...Title` | `title` attribute of a link, often the untruncated name |
| `...Image` / class like `thumbnail` | Image URL (lazy-load attributes first) |
| `...Alt` | Image alt text |
| `datetime` | `datetime` attribute of a `<time>` |
| `linkText`, `text`, `title`, `description` | Element text when there is no useful class |

When two columns would get the same name, the second becomes `name2`, then `name3`.

### FAQ

**The page I want returns no rows.** The site most likely builds the list in the browser with JavaScript, so the HTML holds no list. The run says so in its status message. This Actor does not run a browser, which keeps it fast and cheap. If the site has a JSON API behind the page, scraping that API is usually the better route.

**It picked the wrong list.** Run with `action: findTables`, look at the sample rows, then set `tableNumber` or paste the `itemSelector` it prints.

**Column names look technical.** They come from the page's own class names. Use `renameColumns` with `keepOnlyRenamedColumns` to get exactly the columns you want under your own names.

**Pagination stopped early.** Set `nextPageSelector` to the CSS selector of the next button, or raise `maxPages`. Infinite-scroll and "load more" buttons that fetch with JavaScript are not followed.

**The site blocks me.** Pick a residential proxy and a country in `proxy`. For logged-in content, add `cookies`.

**How am I charged?** Per row in the dataset. "Find tables" preview rows are free.

### Support

Open an issue on the Actor's Issues tab with the URL you tried and what you expected. Most fixes for a specific site ship within a few days.

### Additional services

Need a dedicated scraper for one site (detail pages, logins, APIs, scheduled exports)? Reach out through the Issues tab.

### Explore more scrapers

Browse the rest of the portfolio at [apify.com/memo23](https://apify.com/memo23).

### 🤖 For AI agents & LLM apps

Call this Actor through the Apify API or the Apify MCP server with `{"startUrls": ["<url>"], "maxItems": 50}`. It returns flat JSON rows with stable keys for a given site, which suits tool calls that need "the list on this page" without writing a scraper. Use `action: "findTables"` first when you need to choose between several lists on a page.

### ⚠️ Disclaimer

This Actor reads publicly available web pages, or pages you can access with your own cookies. You are responsible for following each website's terms of service, robots rules and applicable law (including GDPR and copyright) when you scrape and use the data. Do not use it to collect personal data without a lawful basis.

### SEO Keywords

instant data scraper, instant web scraper, scrape any website, list scraper, table scraper, html table to csv, web page to excel, auto detect data, no code scraper, pagination scraper, product list scraper, directory scraper, website data extractor, web scraping api, data scraper cloud

# Actor input Schema

## `startUrls` (type: `array`):

Pages that show a list or table: search results, category pages, directories, rankings, HTML tables. Each URL is scraped on its own and its next pages are followed.

## `action` (type: `string`):

"Scrape rows" returns the best list on each page as rows. "Find tables" returns one preview row per list found on each page (number, size, column names, 3 sample rows), so you can pick a different one with Table number. Preview runs are not charged per row.

## `maxItems` (type: `integer`):

Stop after this many rows from each start URL.

## `maxPages` (type: `integer`):

How many pages to follow from each start URL, the start page included. Set 1 to read only the page you entered.

## `tableNumber` (type: `integer`):

Which detected list to scrape: 1 is the best match. Run "Find tables" first to see the numbers. Ignored when Item selector is set.

## `itemSelector` (type: `string`):

CSS selector that matches one row of the list, for example "li.product" or "table#results tbody tr". Skips auto-detection. "Find tables" prints a working selector for every list it finds.

## `renameColumns` (type: `array`):

Rename detected columns. Key: the detected column name (as shown in the output or in "Find tables"). Value: the name you want.

## `keepOnlyRenamedColumns` (type: `boolean`):

Output only the columns listed in Rename columns (plus pageUrl, pageNumber, position).

## `dropConstantLabels` (type: `boolean`):

Leave out columns that hold the same short text in every row, such as "Read more" or "Add to cart".

## `nextPageSelector` (type: `string`):

CSS selector of the next-page link. Leave empty: the Actor finds rel=next links, "Next" / "›" buttons and numbered pagers on its own.

## `autoIncrementPage` (type: `boolean`):

When a page has no next link but its URL holds a page number (?page=2, /page/2), try the following number. Stops when a page brings no new rows.

## `delayMs` (type: `integer`):

Pause before each next page of the same URL. Raise it for sites that rate-limit.

## `cookies` (type: `array`):

For pages that need you to be logged in. Export your cookies with a browser extension such as Cookie-Editor (JSON export) and paste them here. Only cookies whose domain matches the URL are sent.

## `customHeaders` (type: `object`):

Headers added to every request, for example {"accept-language": "de-DE"}.

## `proxy` (type: `object`):

Leave empty and the Actor picks its own route. Pick a proxy country or group here when a site shows different content per country or blocks datacenter IPs.

## Actor input object example

```json
{
  "startUrls": [
    "https://wordpress.org/plugins/?plugin_business_model=commercial"
  ],
  "action": "scrape",
  "maxItems": 100,
  "maxPages": 20,
  "tableNumber": 1,
  "keepOnlyRenamedColumns": false,
  "dropConstantLabels": true,
  "autoIncrementPage": true,
  "delayMs": 0
}
```

# Actor output Schema

## `rows` (type: `string`):

One row per list item: the detected columns plus pageUrl, pageNumber and position.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        "https://wordpress.org/plugins/?plugin_business_model=commercial"
    ],
    "maxItems": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("memo23/instant-web-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": ["https://wordpress.org/plugins/?plugin_business_model=commercial"],
    "maxItems": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("memo23/instant-web-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    "https://wordpress.org/plugins/?plugin_business_model=commercial"
  ],
  "maxItems": 100
}' |
apify call memo23/instant-web-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,memo23/instant-web-scraper"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vpz95dHAIr3knSnsv/builds/eVEZ3TKeM0RUhcFCo/openapi.json