AI Web Search Tool: Real-Time Search Results for Agents and Apps

AI Web Search Tool searches several engines, including DuckDuckGo, Google, Brave, Yahoo and Bing, with automatic fallback when one is unavailable, and returns web pages, images, videos, news or books as structured JSON. Feed it into an AI agent, a RAG pipeline or your own app, then read the results from the dataset, or pull the complete object, including run metadata, from the key-value store.

Why This Scraper?

• Multiple engines, automatic fallback. Web and news searches draw on several engines, including DuckDuckGo, Google, Brave, Yahoo and Bing, so one engine being unavailable does not stop the run.
• Five result types. Web pages, images, videos, news articles or books, switchable with a single field.
• Locale and freshness control. Pick a region and language code, and optionally restrict results to the last day, week, month or year.
• Safe search built in. Off, moderate or strict, set per run.
• Structured for machines. Every result is a flat JSON record ready for an LLM prompt, a RAG index or a spreadsheet, with no HTML to parse.
• Two ways to read the output. Individual records in the dataset, or one combined object with results and run metadata in the key-value store.
• Field shaping on demand. Trim each record down to only the fields you need with dot notation paths, no post-processing required.

Use Cases

• AI agents and RAG pipelines: feed live web, news or image results into an LLM's context so it can answer with current information.
• Fact checking and research: pull recent news and multiple sources on a claim before writing it up.
• Content and media discovery: find images, videos or news items on a topic for social posts, blogs or newsletters.
• Monitoring and reporting: run the same query on a schedule and export results into your own pipeline or spreadsheet.
• App and chatbot backends: power a "search the web" feature in your own product through the API.

Data You Get

Sample shape: values are illustrative placeholders, not from a live record.

Field	Example
position	1
title	"Understanding Artificial Intelligence"
link	"https://example.com/articles/understanding-ai"
snippet	"Artificial intelligence refers to computer systems that perform tasks typically requiring human intelligence."
domain	"example.com"
query	"artificial intelligence trends"
image	"https://example.com/media/photo.jpg"
thumbnail	"https://example.com/media/photo-thumb.jpg"
source	"example.com"
height	720
width	1280
description	"A short clip explaining the topic."
duration	"5:32"
published	"2024-06-01"
publisher	"Sample Publisher"
date	"2024-06-01"
author	"Sample Author"
info	"English [en], .epub, 0.5MB"

Which fields show up depends on Result type: web results carry snippet and domain, news results carry snippet, date and source, images carry image, thumbnail, source, height and width, videos carry description, thumbnail, duration, published and publisher, and books carry author, publisher, info and thumbnail. position and query are on every record.

How to Use

1. Enter a Search query, the only required input.
2. Pick a Result type: web, images, videos, news or books.
3. Adjust Max results, Region, Safe search and Time range as needed.
4. Click Start, then read the results from the dataset, or fetch the combined object from the key-value store.

Basic web search:

{
  "query": "artificial intelligence trends",
  "maxResults": 10,
  "resultType": "web"
}

Recent news with a time filter:

{
  "query": "renewable energy policy",
  "resultType": "news",
  "timeRange": "w",
  "maxResults": 20
}

Image search with region and safe search:

{
  "query": "mountain landscape photography",
  "resultType": "images",
  "region": "uk-en",
  "safeSearch": "strict",
  "maxResults": 30
}

Web search through Apify Proxy:

{
  "query": "machine learning datasets",
  "resultType": "web",
  "maxResults": 50,
  "proxy": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
  }
}

Run it from your code

Python:

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("abotapi/ai-web-search-tool").call(run_input={"query": "artificial intelligence trends", "resultType": "web"})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["title"], item["link"])

JavaScript:

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('abotapi/ai-web-search-tool').call({ query: 'artificial intelligence trends', resultType: 'web' });
const { items } = await client.dataset(run.defaultDatasetId).listItems();

Or connect it to Make, Zapier, n8n, Google Sheets or webhooks from the Integrations tab.

Shaping and reusing output

• dev_transform_fields trims every record down to only the field paths you list, using dot notation for nested values, so you don't have to post-process the dataset yourself.
• dev_dataset_name saves a run into a named dataset instead of the default one, with {ACTOR}, {DATE} and {TIME} placeholders for organizing repeated runs.
• dev_dataset_clear empties that dataset before the new results are inserted, useful when a named dataset is reused as a "latest results" table.

Input Parameters

Parameter	Type	Default	Description
query	string	"Tesla"	The search query to run. Required.
maxResults	integer	50	Maximum number of results to return (1 to 500).
region	string	"us-en"	Region and language code for results, for example us-en, uk-en, de-de, fr-fr, jp-jp.
safeSearch	string	"moderate"	Safe search filter level: off, moderate or strict.
timeRange	string	""	Restrict results by recency: empty for any time, d (past day), w (past week), m (past month) or y (past year).
resultType	string	"web"	Type of results to return: web, images, videos, news or books.
proxy	object	(none)	Optional connection settings. Left empty, the search runs without a proxy.
dev_transform_fields	array	[]	Advanced. Keep only the listed field paths in each output record, using dot notation for nested objects.
dev_dataset_name	string	"default"	Advanced. Save results into a named dataset instead of the default one. Supports {ACTOR}, {DATE} and {TIME} placeholders.
dev_dataset_clear	boolean	false	Advanced. Clear the named dataset before inserting the new results.

Output Example

Sample shape: values are illustrative placeholders, not from a live record.

{
  "position": 1,
  "title": "Understanding Artificial Intelligence",
  "link": "https://example.com/articles/understanding-ai",
  "snippet": "Artificial intelligence refers to computer systems that perform tasks typically requiring human intelligence.",
  "domain": "example.com",
  "query": "artificial intelligence trends"
}

Plan Requirement

This actor runs under Apify's standard usage based pricing. Most searches work without Apify Proxy; for higher volume or repeated runs, turning on Apify Proxy under Connection can improve reliability. Check the actor's Pricing tab for exact rates.

FAQ

How much does it cost?

You pay under Apify's usage based pricing for the compute time each run uses. The Pricing tab on the actor page shows the current rate, and Max results caps how large a single run can get.

Is it legal to search DuckDuckGo, Bing, Google and the other engines this way?

This actor reads the same public result pages a normal search returns to any visitor, and it does not extract content from the sites those results link to. You are responsible for how you use the results: follow each search engine's own terms and the laws that apply to you, especially before republishing or redistributing what you find.

Can I schedule this to get fresh results automatically?

Yes, schedule it from the Schedules tab like any Apify actor. There is no incremental mode built in, so every scheduled run returns a fresh, complete set of results for the query rather than only what changed since the last run.

Why does the Search query field show "Tesla" by default?

That's a sample value so the form has something to run out of the box. query is required, so replace it with your own search before a real run; leaving the sample in place will search for "Tesla".

Why did my run fail instead of returning an empty dataset?

The run stops with an error when query is empty, or when no search engine returns any result for it, whether the engines could not be reached or the query simply matched nothing. In both cases nothing is written to the dataset. Try a broader query, a different Region or Time range, or run it again in a few minutes.

Can I use this with AI agents or MCP?

Yes. Call it from your agent framework's tool interface or through the Apify API or client shown above, then feed the returned JSON straight into your prompt or retrieval index. It does not currently include extra connectors that push results into other apps.

🔗 Want more AI data?

Pair this actor with these related scrapers from the same team:

🤖 AI Agent Web Fetcher An advanced web fetcher that can fetch almost all websites and convert them to...	📱 Lemon8 Profile Scraper Scrape Lemon8 user profiles with automated multi-profile discovery. Extract profile data...
🧩 Yandex SERP Scraper Scrape Yandex web, image and video results with URLs, snippets and organic rankings for...	👗 Lemon8 Feed Scraper Scrape Lemon8 feeds across 22 categories and 10+ regions. Extract posts, images, videos...
🤖 Web Scraper For Llms Stealth web scraping engine built for LLMs. Converts any web page to clean markdown or...	📇 Yellow Pages AU From $1/1K. Scrape business listings from Yellow Pages Australia by type and location...

👉 Browse all abotapi scrapers

💬 Support & custom scrapers

• 🐞 Found a bug or a missing field? Open a ticket on the Issues tab. We usually reply within hours.
• 🛠️ Need another site, extra fields or a private build? Email abotapi@proton.me or message Telegram @abotapi.
• ⭐ Enjoying it? A quick review on the actor page helps other users find it.