Competitor Website Analyzer

Turn any company website into a structured competitive, SEO or sales brief in under a minute: free in extractive mode, with an optional AI-written brief. Built for founders, product marketers, SEO specialists, sales reps and agencies who need a fast first look at a competitor or prospect.

Quick start

1. Paste a company website into Website URL (a competitor or a prospect).
2. Pick a Brief type: competitive, SEO or sales. Leave Generate AI-written brief off for the free extractive brief, or switch it on for an AI narrative (no API key needed).
3. Click Start, then open the Output tab or download the result as JSON, CSV or Excel.

Sample output

{
  "ok": true,
  "inputUrl": "https://www.example.com/",
  "briefType": "competitive",
  "mode": "llm",
  "brief": "## Positioning\n- Live dashboards for small teams, sold on fast setup...\n\n_AI-generated by openai/gpt-4o-mini on 2026-10-17. May contain errors; check facts before relying on them._",
  "aiGenerated": true,
  "aiModel": "openai/gpt-4o-mini",
  "aiNotice": "AI-generated; may contain errors. Check facts before relying on them.",
  "aiGeneratedAt": "2026-10-17T09:00:12.000Z",
  "aiDigitalSourceType": "http://cv.iptc.org/newscodes/digitalsourcetype/trainedAlgorithmicMedia",
  "aiError": null,
  "structured": {
    "company": "Acme Analytics",
    "tagline": "Acme Analytics turns spreadsheets into live dashboards in minutes.",
    "keyMessaging": ["Dashboards your whole team understands", "Fast setup", "Enterprise-grade security"],
    "pricingSignals": ["$29/mo", "$99/mo"],
    "detectedTechStack": ["Segment", "Google Analytics", "Stripe"],
    "callsToAction": ["Start free trial", "Book a demo"],
    "pagesAnalyzed": ["https://www.example.com/", "https://www.example.com/pricing"]
  },
  "pages": [
    { "pageUrl": "https://www.example.com/", "title": "Acme Analytics - Dashboards for small teams", "tdmReserved": false, "tdmPolicy": null }
  ],
  "skippedSubpages": [],
  "generatedAt": "2026-10-17T09:00:13.000Z"
}

pages is shortened here. In the free extractive mode mode is "extractive", brief is null, aiGenerated is false and the other AI labels are null; all structured signals are still there.

How much does it cost?

What you get	Price
Extractive brief (no AI)	Free
AI-written brief (event "Brief generated")	$0.05 per brief on the free plan, as low as $0.035 on higher Apify plans
Actor start (automatic)	$0.00005 per GB of run memory, once per run

• 100 AI briefs: $5.00 (as low as $3.50). 1,000 AI briefs: $50.00 (as low as $35.00).
• 100 or 1,000 extractive briefs: $0.00, plus the start fee of each run ($0.00005 at 1 GB).

Platform usage is included in the price. An AI brief is charged once, only after it has been written and saved, and costs the same $0.05 whether it is written through Apify's AI gateway or with your own OpenAI key (OpenAI then bills you for the tokens). Nothing is charged for unreachable or blocked sites, for sites whose robots.txt doesn't allow the Actor, or when the AI step is skipped or fails and you get the free extractive brief instead. Model usage on Apify's AI gateway may be billed to your Apify account separately; see AI and your data.

Pricing update on 16 November 2026. The prices above take effect on 16 November 2026 at 14:41 UTC (Apify's 14-day notice). Until then the previous pricing applies: $0.05 for every row in the results (as low as $0.035 on higher plans), including extractive briefs and rows for sites that could not be analyzed, and AI briefs through Apify's AI gateway are not available yet (an AI brief with your own OpenAI key works). The Pricing tab always shows what applies today.

AI and your data

• AI is off by default. When it's on, the content of the analyzed pages (title, description, headings, prices and up to 3,000 characters of text per page) goes either to Apify's AI gateway (Apify's OpenRouter Actor, then OpenRouter, then the model provider) or, with your own key, straight to OpenAI. Email addresses and phone numbers are masked first unless you switch on Include personal data, and pages that reserve text-and-data-mining rights are never sent.
• Apify may bill the AI model's usage to your Apify account separately through its OpenRouter Actor, at OpenRouter's rates; free-plan accounts pay more and may be blocked. That model usage isn't part of the prices above.
• On the gateway, AI model accepts openai/... and anthropic/... models only, without :free or stealth variants; any other ID falls back to openai/gpt-4o-mini. Requests ask OpenRouter to use only providers that don't collect data.
• If you choose a model, you also accept that model provider's terms and acceptable-use policy.
• Own OpenAI key: Use a dedicated OpenAI project key with a spending limit. It's used only inside your run, sent only to OpenAI, and never stored or logged. You're responsible for following OpenAI's terms. A brief written with your key still costs $0.05.
• EU AI Act: AIDevs is the provider of this AI feature; you're the deployer. AI fields are marked with aiGenerated, aiModel, aiGeneratedAt, aiDigitalSourceType and aiNotice. Keep the label if you publish AI text. The brief itself also ends with a visible "AI-generated by ..." line.

robots.txt and text-and-data-mining opt-outs

robots.txt is respected by default (Respect robots.txt, respectRobotsTxt). Before the homepage, each subpage and any redirect, the Actor reads the site's robots.txt and follows the AIDevsBot group if there is one, otherwise the * group. A disallowed homepage stops the run with a free ROBOTS_DISALLOWED record; disallowed subpages are skipped and listed in skippedSubpages. If robots.txt answers 401 or 403, the whole site counts as disallowed; if it can't be read (429, server error, network error or timeout), the site is skipped for this run (ROBOTS_UNREACHABLE); a missing robots.txt (404) allows everything. Turn the switch off only for sites you own or have permission to analyze.

Requests identify as AIDevsBot; site owners can opt out with User-agent: AIDevsBot / Disallow: / in robots.txt.

Text-and-data-mining reservations (TDMRep). Each entry in pages has tdmReserved and tdmPolicy. A page reserves its rights with the HTTP header tdm-reservation: 1 or the tag <meta name="tdm-reservation" content="1"> (the tag wins over the header), and may name a policy in tdm-policy. Reserved pages still get the free extractive signals, but their content is never sent to the AI. If no page is left for the AI, you get the free extractive brief, aiError says why, and no AI brief is charged.

FAQ

Is it legal? It depends on the site and on how you use the results; this isn't legal advice. The Actor reads only public pages, never logs in, respects robots.txt by default, and stops at CAPTCHAs, bot challenges and login walls without trying to get around them. Competitor sites' own terms apply: some SaaS vendors forbid automated access to their websites in their terms, so check a site's terms before you analyze it. Personal data is off by default: links to people's profiles are removed and email addresses and phone numbers are masked. If you switch on Include personal data, you are the data controller and need a lawful basis under the GDPR for processing it.

What happens to my data? AIDevs processes your input and results only to run the Actor. Nothing is kept or shared beyond your own Apify storages and the AI service you switch on.

Do I need an OpenAI key? No. The AI brief uses Apify's built-in AI gateway, which may need a paid Apify plan. You can add your own OpenAI key if you prefer your own OpenAI account; the brief still costs $0.05.

What happens when a site blocks bots? The Actor stops at CAPTCHAs, bot challenges and login walls, says so in the run's status message, and charges nothing. It never tries to get around them. Pages that robots.txt doesn't allow are treated the same way.

Wrong charge? Report it in the Issues tab; we reply within 14 days.

What's the difference between the three brief types? Same input, different lens: competitive covers positioning and differentiators, seo covers keywords, content gaps and meta quality, sales covers pain points, buying signals, roles to contact and an outreach angle.

How many pages does it read? The homepage plus up to maxSubpages (default 3) key internal pages such as pricing, about and product, except those robots.txt disallows.

Not affiliated with, endorsed by or sponsored by OpenAI, Anthropic, OpenRouter, Google, LinkedIn, Make, Zapier or n8n, or by the companies and tools named in its results.

Limitations

• One company website per run.
• Public pages only: no logins, and sites that block automated access are skipped, never bypassed.
• Pages that robots.txt disallows for AIDevsBot are not read, and a site whose robots.txt can't be read is skipped for that run.
• Heavily client-rendered sites (JavaScript-only) yield fewer signals.
• AI briefs can contain mistakes; they are labelled aiGenerated: true with an aiNotice and end with a visible AI line.
• AI through Apify's gateway may need a paid Apify plan and uses openai/ and anthropic/ models only.
• Extractive signals are heuristics (e.g. tech stack detection from page markup).
• Uses Apify's standard datacenter proxies only; residential and custom proxies aren't supported.

Works with AI agents through Apify's MCP server.

More tools from AIDevs

• PDF to Markdown Converter: PDF, DOCX and HTML documents to clean Markdown for LLMs and RAG.
• Web Page to Markdown: any URL to clean Markdown, text and metadata, with an optional AI summary.
• Website to Markdown Crawler: whole websites to Markdown for RAG pipelines.
• SaaS Competitor Battlecards: sales battlecards from public pricing and feature pages.
• PricePulse AI: cross-marketplace product and price intelligence.
• Google Maps Lead Engine: local business leads with AI scoring.
• Nordic Jobs Scraper: Swedish and Norwegian job postings from official open-data APIs.
• LinkedIn Jobs Scraper Pro: public LinkedIn job listings, no login or cookies.

---

How it works

1. Check robots.txt (on by default), then fetch the homepage at url.
2. Follow key subpages (optional): pricing, plans, about, product, features, customers, solutions on the same site, unless robots.txt disallows them.
3. Extract signals: title and positioning, messaging (headings), pricing mentions, detected tech stack, calls to action, company social profiles, and each page's TDM reservation.
4. Choose the lens: competitive, seo or sales shapes the AI brief.
5. AI brief (optional): with Generate AI-written brief on, an AI model writes a Markdown brief grounded only in the content of pages without a TDM reservation.
6. Output: one record per run in the dataset.

Input reference

Field	Type	Default	Description
url	string	(required)	The website to analyze.
briefType	enum	competitive	competitive, seo or sales.
includeSubpages	boolean	true	Also analyze key internal pages.
maxSubpages	integer	3	How many internal pages to add (0-8).
respectRobotsTxt	boolean	true	Check robots.txt first (AIDevsBot group, else *). Turn off only for sites you own or may analyze.
useAi	boolean	false	Write an AI brief ($0.05 per brief, also with your own key).
aiModel	string	(empty)	Advanced: gateway model ID, openai/... or anthropic/... only.
openaiApiKey	string (secret)	(empty)	Optional own OpenAI key; switches AI on, as in earlier versions.
model	enum	gpt-4o-mini	Model for the AI brief.
includePersonalData	boolean	false	Keep people's profile links, emails and phone numbers (GDPR applies).
proxyConfiguration	object	Apify Proxy	Proxy settings.

Output reference

One dataset record per run:

Field	Description
inputUrl	The website analyzed.
briefType	The lens used: competitive, seo or sales.
mode	llm (AI brief written) or extractive (structured signals only).
brief	Markdown AI brief ending with a visible AI line, or null in extractive mode.
aiGenerated, aiModel, aiNotice, aiGeneratedAt, aiDigitalSourceType	AI transparency labels for brief.
aiError	Why no AI brief was written when you asked for one (otherwise null).
structured	Company, tagline, key messaging, pricing signals, detected tech stack, social presence, CTAs, pages analyzed.
pages	Per-page signals for each analyzed URL, including tdmReserved and tdmPolicy.
skippedSubpages	Subpages robots.txt disallowed: url, errorCode, reason.
generatedAt	ISO timestamp.

If the site cannot be read, the record has ok: false, error, errorCode (e.g. HTTP_404, BLOCKED, LOGIN_REQUIRED, ROBOTS_DISALLOWED, ROBOTS_UNREACHABLE) and is free.

API example

curl "https://api.apify.com/v2/acts/entranced_gelato~ai-competitive-brief-generator/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://www.example.com", "briefType": "competitive", "useAi": true }'

from apify_client import ApifyClient

client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("entranced_gelato/ai-competitive-brief-generator").call(
    run_input={"url": "https://www.example.com", "briefType": "sales"}
)
brief = next(client.dataset(run["defaultDatasetId"]).iterate_items())
print(brief["structured"]["company"], "-", brief["structured"]["tagline"])
print(brief.get("brief") or "(extractive mode - no AI brief)")

Use it from Make, Zapier or n8n through the Apify integration, and drop briefs straight into CRM deal notes or prospect records.