Bing Copilot Scraper 🚀

Extract generative AI answers from Bing Copilot with ease.

✨ Features

• Resilient Extraction: Handles complex dynamic content by searching through all iframes to find the generative response.
• Clean Output: Automatically strips markdown bolding (**) and noisy citation markers (e.g., [^1^]) for a clean reading experience.
• Structured Citations: Parses source citations into a dedicated, easy-to-use array of objects (index, URL).
• Intelligent Waiting: Automatically waits for the generative AI to finish its response before extraction.
• Residential Proxy Support: Defaults to and highly recommends residential proxies to ensure stable access to Bing's AI features.

🛠️ Usage

Input Schema

Field	Type	Description
queries	array	Required. List of questions/queries to ask Bing Copilot.
maxRequestsPerCrawl	integer	Maximum number of queries to process (default: 100).
proxyConfiguration	object	Specifies proxy servers (Residential proxies are prefilled and highly recommended).

Example Input

{
  "queries": ["How to optimize TypeScript for performance?"],
  "proxyConfiguration": {
    "useApifyProxy": true
  }
}

📊 Output

The Actor results in a structured dataset. Each item contains:

Field	Description
url	The initial search URL.
query	The query that was asked.
answer	The clean, parsed answer text.
citations	An array of source links used by the AI.
timestamp	ISO timestamp of the crawl.

Example Output item

{
  "url": "https://www.bing.com/copilotsearch?q=TypeScript+performance",
  "query": "How to optimize TypeScript for performance?",
  "answer": "To optimize TypeScript performance, you should focus on several key areas including project structure, compiler settings, and coding patterns...",
  "citations": [
    { "index": "1", "url": "https://www.typescriptlang.org/docs/handbook/2/performance.html" },
    { "index": "2", "url": "https://blog.logrocket.com/how-to-optimize-typescript-performance/" }
  ],
  "timestamp": "2026-01-26T14:42:26.073Z"
}

🚀 Why use this scraper?

Bing Copilot's UI is notoriously difficult to scrape because it uses deeply nested iframes and dynamic content. Most traditional scrapers fail when the DOM structure shifts. Our strategy provides a layer of abstraction that makes this scraper significantly more resilient to UI changes than any selector-based approach.

🏃 Local Development

If you want to run this actor locally, ensure you have the Apify CLI installed:

1. Clone the repository.
2. Install dependencies: npm install.
3. Run locally: npm run start:dev.