General Purpose Web Scraping and Metadata Extraction

Pricing

$20.00/month + usage

General Purpose Web Scraping and Metadata Extraction

This project uses the Apify platform to scrape data from web pages, collect metadata, and store results in an Apify dataset. It features functions for managing date ranges, encoding identifiers, and handling large datasets, aiming to efficiently extract and store structured data for analysis.

Pricing

$20.00/month + usage

Rating

0.0

(0)

Developer

Jamshaid Arif

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

18 days ago

Last modified

Categories

Automation

Developer tools

You can access the General Purpose Web Scraping and Metadata Extraction programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in Integrations settings in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = { "startUrls": [{ "url": "https://www.airbnb.co.uk/rooms/7431972" }] }
9
10# Run the Actor and wait for it to finish
11run = client.actor("moving_beacon-owner1/my-actor-10").call(run_input=run_input)
12
13# Fetch and print Actor results from the run's dataset (if there are any)
14print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
15for item in client.dataset(run["defaultDatasetId"]).iterate_items():
16    print(item)
17
18# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

General Purpose Web Scraping and Metadata Extraction API in Python

The Apify API client for Python is the official library that allows you to use General Purpose Web Scraping and Metadata Extraction API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

General Purpose Web Scraping and Metadata Extraction API in JavaScript

General Purpose Web Scraping and Metadata Extraction API through CLI

General Purpose Web Scraping and Metadata Extraction OpenAPI definition

General Purpose Web Scraping and Metadata Extraction API

Extract Website With URL

mrahil/extract-website-with-url

The Extract Website with URL API allows users to extract structured data from any webpage by providing a URL. It retrieves HTML, metadata, tables, and images, returning data in JSON format. Ideal for web scraping, SEO analysis, and content extraction. Use it for e-commerce data, news scraping

Mohammed Rahil

187

Web Scraper and AI processor

scraping_samurai/web-scraper-and-ai-processor

Adaptive AI controller classifies page quality from fast HTTP fetches and selectively triggers headless rendering, then converts raw text into structured JSON from natural-language extraction prompts. Optimizes cost vs. accuracy with AI-guided escalation, retry, and thin/blocked content heuristics.

Scraping Samurai

Dynamic Web Scraper

josejet/dynamic-web-scraper

Dynamic Web Scraper is an Apify Actor that gathers information online by simulating user browsing behavior on the web. It reduces the time and amount of scraped web pages by using a model (ChatGPT) to make decisions regarding browser navigation and results evaluation.

Pepa o͜ﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞ-o J : o͜ﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞﱞ-o

284

Single page web scraping

krishnapada.m.99/single-page-web-scraping

Scrapes the <title> tag or H1 tag from a single webpage provided by the user. Useful for SEO audits or content previews.

Somnath Mandal

PromptBase Scraper

crawlerbros/PromptBase

Extract comprehensive data from PromptBase.com, the world's largest AI prompt marketplace with 220k+ prompts for AI models. This actor scrapes detailed prompt content, pricing data , creator profiles, AI model classifications, and high-quality prompt images.

Crawler Bros

5.0

Smartcontext AI Web Crawler

bluelightco/smartcontext-ai-crawler

Scrape any website and extract structured data using AI-powered instructions. Provide URLs and a natural language prompt to get tailored JSON outputs.

Bluelight

5.0

Universal AI GPT Scraper

louisdeconinck/ai-gpt-scraper

Transform any website into structured data with AI-powered extraction. This versatile tool combines advanced web scraping with intelligent content analysis to deliver clean, customized JSON output - perfect for automating data collection from any web source.

Louis Deconinck

145

5.0

Ai Web Scraper - Extract Data With Ease

eloquent_mountain/ai-web-scraper-extract-data-with-ease

Ai Web Scraper enables scraping for everyone, including non-techies! It uses Google's Gemini LLM to scrape websites with natural language commands. It dynamically extracts data, no selector input needed, handles dynamic content and cookie consent, avoids bot detection, outputs JSON or other formats.