Pricing

Pay per usage

Universal Web Scraper Toolkit

Extract structured data from any URL — metadata, links, headlines, images, tables, clean text, or custom CSS selectors. Batch up to 10 URLs.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

2x lazymac

Actor stats

Bookmarked

Total users

Monthly active users

15 hours ago

Last modified

Categories

Developer tools

You can access the Universal Web Scraper Toolkit programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

{
  "openapi": "3.0.1",
  "info": {
    "version": "1.0",
    "x-build-id": "WvlI8TCU0L615qBqc"
  },
  "servers": [
    {
      "url": "https://api.apify.com/v2"
    }
  ],
  "paths": {
    "/acts/lazymac~web-scraper-toolkit/run-sync-get-dataset-items": {
      "post": {
        "operationId": "run-sync-get-dataset-items-lazymac-web-scraper-toolkit",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK"
          }
        }
      }
    },
    "/acts/lazymac~web-scraper-toolkit/runs": {
      "post": {
        "operationId": "runs-sync-lazymac-web-scraper-toolkit",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor and returns information about the initiated run in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/runsResponseSchema"
                }
              }
            }
          }
        }
      }
    },
    "/acts/lazymac~web-scraper-toolkit/run-sync": {
      "post": {
        "operationId": "run-sync-lazymac-web-scraper-toolkit",
        "x-openai-isConsequential": false,
        "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
        "tags": [
          "Run Actor"
        ],
        "requestBody": {
          "required": true,
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/inputSchema"
              }
            }
          }
        },
        "parameters": [
          {
            "name": "token",
            "in": "query",
            "required": true,
            "schema": {
              "type": "string"
            },
            "description": "Enter your Apify token here"
          }
        ],
        "responses": {
          "200": {
            "description": "OK"
          }
        }
      }
    }
  },
  "components": {
    "schemas": {
      "inputSchema": {
        "type": "object",
        "required": [
          "urls"
        ],
        "properties": {
          "urls": {
            "title": "URLs to scrape",
            "type": "array",
            "description": "List of URLs to scrape",
            "items": {
              "type": "string"
            }
          },
          "mode": {
            "title": "Extraction mode",
            "enum": [
              "full",
              "metadata",
              "links",
              "headlines",
              "images",
              "tables",
              "text",
              "custom"
            ],
            "type": "string",
            "description": "What to extract from each page",
            "default": "full"
          },
          "selector": {
            "title": "CSS Selector (custom mode only)",
            "type": "string",
            "description": "CSS selector for custom mode extraction"
          }
        }
      },
      "runsResponseSchema": {
        "type": "object",
        "properties": {
          "data": {
            "type": "object",
            "properties": {
              "id": {
                "type": "string"
              },
              "actId": {
                "type": "string"
              },
              "userId": {
                "type": "string"
              },
              "startedAt": {
                "type": "string",
                "format": "date-time",
                "example": "2025-01-08T00:00:00.000Z"
              },
              "finishedAt": {
                "type": "string",
                "format": "date-time",
                "example": "2025-01-08T00:00:00.000Z"
              },
              "status": {
                "type": "string",
                "example": "READY"
              },
              "meta": {
                "type": "object",
                "properties": {
                  "origin": {
                    "type": "string",
                    "example": "API"
                  },
                  "userAgent": {
                    "type": "string"
                  }
                }
              },
              "stats": {
                "type": "object",
                "properties": {
                  "inputBodyLen": {
                    "type": "integer",
                    "example": 2000
                  },
                  "rebootCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "restartCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "resurrectCount": {
                    "type": "integer",
                    "example": 0
                  },
                  "computeUnits": {
                    "type": "integer",
                    "example": 0
                  }
                }
              },
              "options": {
                "type": "object",
                "properties": {
                  "build": {
                    "type": "string",
                    "example": "latest"
                  },
                  "timeoutSecs": {
                    "type": "integer",
                    "example": 300
                  },
                  "memoryMbytes": {
                    "type": "integer",
                    "example": 1024
                  },
                  "diskMbytes": {
                    "type": "integer",
                    "example": 2048
                  }
                }
              },
              "buildId": {
                "type": "string"
              },
              "defaultKeyValueStoreId": {
                "type": "string"
              },
              "defaultDatasetId": {
                "type": "string"
              },
              "defaultRequestQueueId": {
                "type": "string"
              },
              "buildNumber": {
                "type": "string",
                "example": "1.0.0"
              },
              "containerUrl": {
                "type": "string"
              },
              "usage": {
                "type": "object",
                "properties": {
                  "ACTOR_COMPUTE_UNITS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_WRITES": {
                    "type": "integer",
                    "example": 1
                  },
                  "KEY_VALUE_STORE_LISTS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_INTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_EXTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_SERPS": {
                    "type": "integer",
                    "example": 0
                  }
                }
              },
              "usageTotalUsd": {
                "type": "number",
                "example": 0.00005
              },
              "usageUsd": {
                "type": "object",
                "properties": {
                  "ACTOR_COMPUTE_UNITS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATASET_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "KEY_VALUE_STORE_WRITES": {
                    "type": "number",
                    "example": 0.00005
                  },
                  "KEY_VALUE_STORE_LISTS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_READS": {
                    "type": "integer",
                    "example": 0
                  },
                  "REQUEST_QUEUE_WRITES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_INTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "DATA_TRANSFER_EXTERNAL_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                    "type": "integer",
                    "example": 0
                  },
                  "PROXY_SERPS": {
                    "type": "integer",
                    "example": 0
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Universal Web Scraper Toolkit OpenAPI definition

OpenAPI is a standard for designing and describing RESTful APIs, allowing developers to define API structure, endpoints, and data formats in a machine-readable way. It simplifies API development, integration, and documentation.

OpenAPI is effective when used with AI agents and GPTs by standardizing how these systems interact with various APIs, for reliable integrations and efficient communication.

By defining machine-readable API specifications, OpenAPI allows AI models like GPTs to understand and use varied data sources, improving accuracy. This accelerates development, reduces errors, and provides context-aware responses, making OpenAPI a core component for AI applications.

You can download the OpenAPI definitions for Universal Web Scraper Toolkit from the options below:

OpenAPI.json

If you’d like to learn more about how OpenAPI powers GPTs, read our blog post.

You can also check out our other API clients:

Universal Web Scraper Toolkit API in Python

Universal Web Scraper Toolkit API in JavaScript

Universal Web Scraper Toolkit API through CLI

Universal Web Scraper Toolkit API

Universal Apify Email & Metadata Scraper (Puppeteer + Crawlee)

lucrateresults/universal-apify-email-metadata-scraper-puppeteer-crawlee

Description: A production-ready Apify actor built with PuppeteerCrawler (Crawlee) to extract emails and metadata from public websites. Optimized for parallel crawling, JavaScript rendering, and IP rotation. Disclaimer: Scrape only public data. Respect each site’s terms.

Lucrate Results

Universal Website Scraper (Python)

fortuitous_inch/my-actor

Scrape structured data from any website URL using Python and BeautifulSoup. Extract titles, links, and page content for research and automation.

Amol Pandgale

Universal Remote Job Scraper

kmltmr00/universal-remote-job-scraper

Get the latest remote jobs from global platforms automatically. Perfect for job seekers and recruiters.

kmltmr

Universal Website to API – Any Site → JSON

inquisitive_zeppelin/universal-website-to-api---any-site---json

Multi-URL Web Scraper is a fast, reliable, and highly flexible scraping tool designed to extract structured data from multiple web pages in a single run.

Hamza Ahmed

Universal Website Scraper

jowi/universal-website-scraper

The Universal Website Scraper is a powerful all-purpose crawler that extracts structured data from ANY website, including dynamic pages, product listings, articles, tables, blogs, and more. No coding needed just enter a URL and let the actor do the rest.

Fred

My Actor

universal_scraping/my-actor

adsadsdsa

Michael Novak

Aladia Teachers Profile Scraper

getdataforme/aladia-teachers-profile-scraper

This Apify actor automates scraping of comprehensive teacher profiles from Aladia, extracting bios, skills, experiences, education, and more into structured JSON. Perfect for research, recruitment, and analysis, it delivers high-quality, scalable data with ethical practices and customizable exports.

GetDataForMe

Universal AI Web Scraper

stanvanrooy6/universal-ai-web-scraper

Turn any website into an API. Extract structured data using plain English. Features anti-bot bypass, dynamic rendering, and web search. No coding needed.

Stan Van Rooy

Scrape GPT - Universal AI Web Scraper Agent

paradox-analytics/scrape-gpt---universal-ai-web-scraper-agent

AI-powered universal web scraper that works on ANY website without configuration. Extract data from e-commerce, news sites, social media, and more using intelligent LLM-based field mapping. Features JSON-first extraction, automatic pagination, anti-bot bypass, and cost-effective caching.

Paradox Analytics

Universal Web Scraper & Data Extractor – Fast No-Code Tool

motivational_nickel/my-actor

Universal web scraper that extracts structured data from almost any website. Detect and scrape webpage content into clean datasets (CSV, Excel, JSON) without coding. Ideal for web scraping, research, lead generation, automation pipelines, and large-scale data extraction.