# REST API Paginator: Every Page of a JSON API (`ucchiii/rest-api-paginator`) Actor

Fetch every page of a REST or JSON API in one run: page number, offset/limit, cursor, Link header or next-URL pagination. One dataset row per item (or per page), with your headers and API keys kept secret. Internal addresses are blocked. Empty input = free demo.

- **URL**: https://apify.com/ucchiii/rest-api-paginator.md
- **Developed by:** [Actor Factory](https://apify.com/ucchiii) (community)
- **Categories:** Developer tools, Automation, Other
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.31 / 1,000 fetched pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**Get every page of a JSON API without writing a pagination loop**

**For developers, data and ops teams who need all records from a REST API, not just the first page.** Give it one
endpoint (URL, method, headers, query or JSON body) and it follows the API's pagination to the end: **page numbers,
offset/limit, cursors or next tokens, Link headers (`rel="next"`) or a next-page URL in the response**. You get **one
dataset row per item** with `_page` and `_sourceUrl`, ready for CSV, Excel, Google Sheets or the next Actor — or one row
per page if you prefer. **$3.30 per 1,000 pages.** Leave the input empty for a free demo.

### Use cases

- Export all orders, tickets, contacts or products from a SaaS API that returns 50 or 100 records per page
- Pull a full list from a public JSON API (search results, catalogs, open-data endpoints) into one dataset
- Turn a paginated API into a CSV or spreadsheet: run this, then export the dataset or use [JSON to CSV](https://apify.com/ucchiii/json-to-csv)
- Schedule a nightly full sync of an API listing without maintaining a script

### How it works

- **Pagination types:**
  - `page` — `page=1, 2, 3 ...` (parameter name and first page number are configurable)
  - `offset` — `offset=0, 20, 40 ...` or `skip`/`start`; the offset grows by the number of items actually received
  - `cursor` — reads the next cursor from a dot path in the response (`meta.next_cursor`, `nextPageToken`, `response_metadata.next_cursor` ...) and sends it back as `cursor`, `pageToken`, `start_cursor` or any name you choose. `@last.id` takes it from the last item (`starting_after` style)
  - `link` — follows the `Link` header with `rel="next"` (GitHub-style APIs)
  - `nextUrl` — follows a next-page URL in the body (`next`, `links.next`, `paging.next`, `@odata.nextLink`, `{ "href": ... }` objects, relative URLs ...)
  - `auto` (default) — Link header first, then a next-page URL in the body; `none` — a single request
- **Items:** set `itemsPath` (a dot path such as `data.items`, `results` or `data[0].rows`) or leave it empty: the first page is scanned for a top-level array or the first array of objects (`data`, `items`, `results`, `records` ...). Plain values become `{ "value": ... }` rows
- **GET or POST/PUT/PATCH** with a JSON body; pagination values can go in the query string or into the body (`paramLocation: "body"`, dot paths allowed) for search endpoints that read them from the body
- `pageSize` with your page-size parameter (`limit`, `per_page`, `page_size` ...), `hasMorePath` (e.g. `has_more`) to stop when the API says so
- **Stops by itself** on an empty page, a missing or repeated cursor, a repeated next link, a page identical to the previous one (an API that ignores the page parameter), `maxPages` (up to 10,000), `maxItems`, or the maximum charge you set for the run
- **Polite by default:** one request at a time, `delaySecs` between requests (default 0.3 s), `timeoutSecs`, and `retries` for 429 and 5xx. **HTTP 429 `Retry-After` is honoured** (seconds or a date, waits up to 120 s); otherwise backoff of 1, 2, 4 ... s
- Run totals — pages, items, why it stopped, detected items path, a per-page log — are saved as `SUMMARY` in the key-value store

### Your API keys stay secret

- **Headers** and **Secret query parameters** are secret input fields: Apify stores them encrypted
- Values of headers whose names contain key, token, auth, secret, cookie or password, and of query parameters with such names (`api_key`, `access_token` ...), are **never written to the log, the dataset or the key-value store**: they appear as `[REDACTED]`. If the API echoes your key back in its response, it is redacted in the items too
- Logs show only the host name and per-page counts
- Secret headers and secret query parameters are **not sent to another host** — neither on redirects nor when a next-page link points to a different domain
- **Internal addresses are never contacted.** Host names are resolved and localhost, private ranges, link-local (such as the cloud metadata address 169.254.169.254) and similar are refused, also at every redirect hop

### Pricing

**$0.0033 per page that returned items** (a 2xx response with a JSON body containing at least one item). One page with 100 items costs the same as one with 1 item.
**Not charged:** failed requests (4xx/5xx after retries, timeouts, DNS errors), non-JSON answers, requests blocked by the internal-address guard, empty last pages, pages that repeat the previous page, retries, and the free demo on empty input.
On the Apify Free plan, the $5 of monthly platform credit covers about **1,500 pages** a month — 150,000 items at 100 items per page.

### Input

```json
{
  "url": "https://api.example.com/v1/orders",
  "headers": { "Authorization": "Bearer YOUR_TOKEN" },
  "queryParams": { "status": "open" },
  "paginationType": "cursor",
  "cursorPath": "meta.next_cursor",
  "cursorParam": "cursor",
  "limitParam": "limit",
  "pageSize": 100,
  "itemsPath": "data",
  "maxPages": 500,
  "delaySecs": 0.5
}
```

Offset example (the prefilled one): `{ "url": "https://dummyjson.com/products", "paginationType": "offset", "offsetParam": "skip", "limitParam": "limit", "pageSize": 20, "itemsPath": "products", "maxPages": 5 }`.

### Output

One row per item (default):

```json
{ "id": 21, "title": "Cucumber", "price": 1.49, "category": "groceries", "_page": 2,
  "_sourceUrl": "https://dummyjson.com/products?select=title,price,category&limit=20&skip=20" }
```

With `"outputMode": "pages"`, one row per page:

```json
{ "_page": 1, "_sourceUrl": "https://api.example.com/v1/orders?limit=100&api_key=[REDACTED]", "statusCode": 200,
  "itemCount": 100, "items": [ { "...": "..." } ], "next": "eyJpZCI6MTAwfQ", "error": null, "elapsedMs": 212 }
```

The `SUMMARY` record has `pagesFetched`, `pagesCharged`, `items`, `stopReason` (e.g. `empty_page`, `no_next_cursor`,
`repeated_cursor`, `max_pages`, `request_failed`), `lastError`, `itemsPath` and a per-page log.

### Free demo

With empty input the Actor fetches 3 pages of products from [DummyJSON](https://dummyjson.com), a free public fake
REST API made for testing and prototyping (no sign-up, no key). Demo rows have `_demo: true` and nothing is charged.

### Notes

You are responsible for what you call: only use APIs you are allowed to access, with your own credentials, and follow
their terms of service and rate limits. The Actor sends only the requests needed to page through the one endpoint you
give; it does not crawl or follow other links. Each response is read up to 8 MB.
Known limit: the internal-address check happens just before connecting; an attacker-controlled DNS server that changes its answer between the check and the connection (DNS rebinding) is not covered.

If this Actor saved you time, a rating on the Store helps others find it.

### More tools from the same developer

- [Bulk HTTP Request](https://apify.com/ucchiii/bulk-http-request) — send many HTTP requests (GET/POST/...) from a list and collect status, headers and body
- [JSON to CSV Converter](https://apify.com/ucchiii/json-to-csv) — turn nested JSON files or datasets into one flat CSV file
- [Flatten JSON to CSV](https://apify.com/ucchiii/flatten-json) — flatten nested JSON into flat rows with the separator you choose
- [Dataset Transform](https://apify.com/ucchiii/dataset-transform) — flatten, filter, rename and convert any Apify dataset, export CSV / JSON / JSONL with a link
- [Bulk URL Status Checker](https://apify.com/ucchiii/url-status-checker) — bulk HTTP status and redirect-chain checker for URL lists

# Actor input Schema

## `url` (type: `string`):

The endpoint to paginate, for example https://api.example.com/v1/orders?status=open. Query parameters already in the URL are kept on every page. Leave empty for a free demo.

## `method` (type: `string`):

HTTP method. Use POST for search or query endpoints that take a JSON body (the same body is sent for every page).

## `headers` (type: `object`):

Request headers as a JSON object, for example {"Authorization": "Bearer YOUR_TOKEN"} or {"X-API-Key": "..."}. Stored encrypted. Values of headers whose names contain key, token, auth, secret, cookie or password are never logged and are shown as \[REDACTED] in the output; if the API echoes them back, they are redacted there too. They are not sent to another host (redirects or next-page links pointing elsewhere).

## `queryParams` (type: `object`):

Extra query parameters for the first request (and every request for page, offset and cursor pagination), for example {"status": "open", "sort": "created"}. Do not put API keys here; use Secret query parameters.

## `secretQueryParams` (type: `object`):

Query parameters that hold credentials, for example {"api_key": "..."} or {"appid": "..."}. Stored encrypted, added to every request to the same host (also to next-page links that drop them) and shown as \[REDACTED] in \_sourceUrl and logs.

## `body` (type: `object`):

Optional JSON body for POST, PUT or PATCH (ignored for GET). Sent as application/json. With Put pagination parameters in = body, the page, offset, cursor and page-size values are written into this body.

## `paginationType` (type: `string`):

How the API moves to the next page. Auto follows a Link header rel="next" or a next-page URL in the body (next, links.next, @odata.nextLink, paging.next ...) and stops when there is none.

## `itemsPath` (type: `string`):

Dot path to the array of items in each response, for example data.items, results or data\[0].rows. Leave empty to detect it automatically from the first page (a top-level array, or the first array of objects such as data, items or results).

## `pageSize` (type: `integer`):

Items per page to ask for. Sent in the page-size parameter below; leave empty to use the API's default.

## `limitParam` (type: `string`):

Name of the page-size parameter, for example limit, per_page, page_size or pageSize. Used only when Page size is set.

## `pageParam` (type: `string`):

Name of the page-number parameter, for example page or p.

## `startPage` (type: `integer`):

Page number of the first request (1 for most APIs, 0 for some).

## `offsetParam` (type: `string`):

Name of the offset parameter, for example offset, skip or start. The offset grows by the number of items actually received on each page.

## `startOffset` (type: `integer`):

Offset of the first request.

## `cursorPath` (type: `string`):

Dot path to the next cursor or token in each response, for example meta.next_cursor, nextPageToken or response_metadata.next_cursor. Start with @last. to take it from the last item, for example @last.id (for starting_after style APIs). Pagination stops when the cursor is missing or repeats.

## `cursorParam` (type: `string`):

Name of the parameter that sends the cursor back, for example cursor, pageToken, starting_after or start_cursor.

## `hasMorePath` (type: `string`):

Optional dot path to a true/false field that says whether more pages exist, for example has_more or meta.hasNextPage. When it is false, pagination stops.

## `nextUrlPath` (type: `string`):

Dot path to the next-page URL in the body, for example links.next or paging.next. Leave empty to look in the usual places. Relative URLs are resolved against the current page.

## `paramLocation` (type: `string`):

Where the page, offset, cursor and page-size values go. Body is for APIs like search endpoints that read them from a POST body (dot paths such as paging.cursor are allowed).

## `outputMode` (type: `string`):

One row per item (each with \_page and \_sourceUrl), or one row per page with its items array, status and next cursor/link.

## `maxPages` (type: `integer`):

Stop after this many requests (1 to 10,000).

## `maxItems` (type: `integer`):

Stop once this many items are saved (0 = no limit). The last page is cut to fit.

## `stopOnShortPage` (type: `boolean`):

Page and offset types: stop when a page returns fewer items than Page size, saving one request. Leave off if the API may return fewer items than asked for on pages that are not the last.

## `delaySecs` (type: `number`):

Pause before each request after the first, to stay within the API's rate limit. Pages are fetched one at a time.

## `timeoutSecs` (type: `integer`):

Timeout for each request attempt.

## `retries` (type: `integer`):

Retries per page for HTTP 429 (honouring Retry-After, up to 120 s), 5xx and network errors. POST, PUT and PATCH are retried only for 429 or when no connection was made. Retries are not charged.

## Actor input object example

```json
{
  "url": "https://dummyjson.com/products",
  "method": "GET",
  "queryParams": {
    "select": "title,price,category"
  },
  "paginationType": "offset",
  "itemsPath": "products",
  "pageSize": 20,
  "limitParam": "limit",
  "pageParam": "page",
  "startPage": 1,
  "offsetParam": "skip",
  "startOffset": 0,
  "cursorParam": "cursor",
  "paramLocation": "query",
  "outputMode": "items",
  "maxPages": 5,
  "maxItems": 0,
  "stopOnShortPage": false,
  "delaySecs": 0.3,
  "timeoutSecs": 30,
  "retries": 3
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://dummyjson.com/products",
    "queryParams": {
        "select": "title,price,category"
    },
    "paginationType": "offset",
    "itemsPath": "products",
    "pageSize": 20,
    "limitParam": "limit",
    "offsetParam": "skip",
    "maxPages": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("ucchiii/rest-api-paginator").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "url": "https://dummyjson.com/products",
    "queryParams": { "select": "title,price,category" },
    "paginationType": "offset",
    "itemsPath": "products",
    "pageSize": 20,
    "limitParam": "limit",
    "offsetParam": "skip",
    "maxPages": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("ucchiii/rest-api-paginator").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://dummyjson.com/products",
  "queryParams": {
    "select": "title,price,category"
  },
  "paginationType": "offset",
  "itemsPath": "products",
  "pageSize": 20,
  "limitParam": "limit",
  "offsetParam": "skip",
  "maxPages": 5
}' |
apify call ucchiii/rest-api-paginator --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,ucchiii/rest-api-paginator"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/1EjKZHqJwlO3kqfwF/builds/TbC1U4k3POhiosNBB/openapi.json
