Deprecated

This Actor is deprecated

This Actor is unavailable because the developer has decided to deprecate it. Would you like to try a similar Actor instead?

See alternative Actors

AI Web Scraper

vulnv/ai-web-scraper

Scrape structured data effortlessly - just describe what you need in plain language, and get precise results tailored to your request. Simplify data extraction with a tool designed for ease and accuracy, no coding required.

Effortlessly extract structured data from web pages by simply describing what you need. This scraper is designed for precision and ease of use, allowing you to customize your data extraction with natural language prompts.

Features

Start URLs: Specify the URLs to begin your scraping.
Natural Language Prompts: Define the desired output by describing it in plain language.
Custom Depth: Configure the scraping depth to suit your needs.
Flexible AI Model Selection: Choose from multiple models for AI processing.
Secure API Key Integration: Seamlessly connect using your API key.

Configuration

Input Schema

The actor accepts the following input parameters:

Field	Type	Description	Default Value
start_urls	Array	List of URLs to start scraping from.	`[{"url": "https://apify.com"}]`
prompt	String	Natural language description of the desired scraping output.	`"List me all the features with their description."`
api_key	String	OpenAI API key. This field is required for AI processing.	(Required)
model	String	The model to use for AI processing. Options: `gpt-4o`, `gpt-4o-mini`, `o1`.	`gpt-4o-mini`
max_depth	Integer	The maximum depth for recursive scraping.	`0`
initial_cookies	String	Cookies that will be pre-set to all pages the scraper opens.	`[]`
proxy_configuration	Object	Select proxies to be used by your crawler.	`{ "useApifyProxy": false }`
save_html_to_key_value_store	Boolean	If enabled, stores full transformed HTML of all pages found to the default key-value store.	`false`
save_markdown_to_key_value_store	Boolean	If enabled, converts the transformed HTML of all pages found to Markdown, and stores it under the markdown field in the output dataset.	`false`

Example Input

1{
2    "start_urls": [
3        { "url": "https://apify.com" }
4    ],
5    "prompt": "List me all the features with their description.",
6    "api_key": "your-openai-api-key",
7    "model": "gpt-4o-mini",
8    "max_depth": 0
9}

How to Use

Set the start URLs to specify the pages you want to scrape.
Write a prompt to describe your desired output.
Add your OpenAI API key for natural language processing.
Choose a model for AI handling, if needed.
Set the maximum depth to control recursive scraping.
Run the actor and get structured results based on your input!

Output

The actor outputs structured data in JSON format, tailored to your provided prompt.

🚀 Looking for a hassle-free web scraping solution? 🚀 This next-generation actor, available on the Apify platform requires no API keys or configurations, simplifies web scraping like never before. Explore advanced features like CAPTCHA bypass, JavaScript rendering, and more. 🌐 Learn more and start using it at AI Web Scraper. 🔍 💡 🌐

Explore More Actors

✨ Looking for additional solutions? Check out more actors on Apify that can help with your web automation and data extraction needs. Discover a wide range of tools tailored for different scenarios at 🌐 Explore Vulnv's Actors on Apify.

📧 For inquiries or support, feel free to reach out to us at apify@vulnv.com.

Developer

VulnV

Categories

Automation

Integrations

Ai Web Scraper - Extract Data With Ease

eloquent_mountain/ai-web-scraper-extract-data-with-ease

Ai Web Scraper enables scraping for everyone, including non-techies! It uses Google's Gemini LLM to scrape websites with natural language commands. It dynamically extracts data, no selector input needed, handles dynamic content, avoids bot detection, and outputs JSON or other formats.

Paco

AI Web Scraper [No API Key Needed]

vulnv/ai-web-scraper-no-api-key-needed

VulnV

AI Markdown Web Scraper

jordanchalupka/ai-markdown-web-scraper

Turn any webpage into a clean markdown format. Perfect to use as input to your LLM.

Jordan

AI Vision Scraper

zscrape/ai-vision-scraper

AI Vision Scraper automates web tasks, navigating sites, solving CAPTCHAs, and extracting data on demand using a single prompt. From competitor tracking to form submissions, it streamlines workflows and automation across industries like e-commerce, sales, and recruiting.

ZScrape Solutions

Website Content to Markdown Scraper for LLM Training

easyapi/website-content-to-markdown-scraper-for-llm-training

🚀 Transform web content into clean, LLM-ready Markdown! 📘 Scrape multiple pages, extract main content, and convert to Markdown format. Perfect for AI researchers, data scientists, and LLM developers. Fast, efficient, and customizable. Supercharge your AI training data today! 🌐📝🧠

EasyApi

Website Content to Markdown for LLM Training

easyapi/website-content-to-markdown-for-llm-training

EasyApi

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

32.6k

889

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, AI overviews, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule runs, or integrate with other tools.

Apify

52.5k

308

🔥 LinkedIn Jobs Scraper

bebity/linkedin-jobs-scraper

ℹ️ Designed for both personal and professional use, simply enter your desired job title and location to receive a tailored list of job opportunities. Try it today!