Pricing

from $4.00 / 1,000 reports

Go to Apify Store

Website Intelligence API

Try for free

Analyze website technologies, comments, structured data, robots.txt, and sitemaps in one run.

Pricing

from $4.00 / 1,000 reports

Rating

0.0

(0)

Developer

Iqbal R

Actor stats

Bookmarked

Total users

Monthly active users

4 hours ago

Last modified

Categories

Start URLs

startUrls

Required

One or more HTTP(S) pages to analyze. Duplicate pages are crawled once while output cardinality is preserved.

Type:array

Min. items:1

Default:

[
  {
    "url": "https://apify.com"
  }
]

Maximum browser concurrency

maxConcurrency

Optional

Maximum number of pages analyzed concurrently.

Type:integer

Minimum:1

Maximum:20

Default:3

Request retries

maxRequestRetries

Optional

Retries after a failed browser navigation.

Type:integer

Minimum:0

Maximum:5

Default:2

Request timeout

timeoutSecs

Optional

Timeout in seconds for browser navigation and site-resource HTTP requests.

Type:integer

Minimum:5

Maximum:120

Default:45

Comment discovery depth

commentsMaxDepth

Optional

Internal-link levels searched for comment systems. Use 0 to check only supplied pages.

Type:integer

Minimum:0

Maximum:10

Default:1

Schema.org discovery depth

schemaMaxDepth

Optional

Internal-link levels searched for Schema.org extraction and validation. Use 0 for supplied pages only.

Type:integer

Minimum:0

Maximum:10

Default:0

Broken structured-data discovery depth

brokenDataMaxDepth

Optional

Internal-link levels scanned for broken structured-data findings. Use 0 for supplied pages only.

Type:integer

Minimum:0

Maximum:10

Default:0

Maximum discovered pages

maxDiscoveryPages

Optional

Maximum additional pages followed across the run, excluding supplied Start URLs.

Type:integer

Minimum:0

Maximum:10000

Default:100

Maximum links per page

maxLinksPerPage

Optional

Maximum eligible same-hostname links considered from each analyzed page.

Type:integer

Minimum:1

Maximum:500

Default:50

Analyze comment systems

analyzeComments

Optional

Detect embedded and native comment systems, including lazy-loaded widgets.

Type:boolean

Default:true

Detect technologies

analyzeTechnologies

Optional

Detect CMS, analytics, commerce, frontend, and infrastructure technologies.

Type:boolean

Default:true

Detect technology versions

detectVersions

Optional

Attempt to extract version numbers from available technology evidence.

Type:boolean

Default:true

Include detection evidence

includeEvidence

Optional

Include bounded evidence explaining each technology detection.

Type:boolean

Default:true

Analyze structured data

analyzeStructuredData

Optional

Extract and validate JSON-LD, Microdata, and RDFa from each page.

Type:boolean

Default:true

Include bounded raw structured data

includeRawStructuredData

Optional

Includes bounded source fragments, never the full page HTML.

Type:boolean

Default:false

Analyze robots.txt

analyzeRobots

Optional

Fetch and evaluate robots.txt once for each unique website origin.

Type:boolean

Default:true

Analyze sitemaps

analyzeSitemap

Optional

Discover and inspect sitemap files once for each unique website origin.

Type:boolean

Default:true

Robots user agent

userAgent

Optional

User-agent token used when evaluating robots.txt rules.

Type:string

Default:*

Maximum sitemap files

maxSitemaps

Optional

Maximum number of sitemap and sitemap-index files processed per origin.

Type:integer

Minimum:1

Maximum:500

Default:50

Maximum sitemap URLs

maxSitemapUrls

Optional

Maximum number of unique URLs collected from sitemaps per origin.

Type:integer

Minimum:1

Maximum:100000

Default:10000

Follow sitemap indexes

followSitemapIndex

Optional

Follow child sitemap references found in sitemap index files.

Type:boolean

Default:true

Proxy configuration

proxyConfiguration

Optional

Optional Apify Proxy or custom proxy settings used by browser and site-resource requests.

Type:object

Default:

{
  "useApifyProxy": false
}

Website Intelligence API

ladra/Website-Intelligence-API

Crawl any public website and turn it into AI-ready intelligence. Extract screenshots, Markdown, HTML, metadata, links, PDFs, compliance evidence, RAG chunks, and structured JSON for sales research, audits, website snapshots, and automation.

Ladra Software

Website Change Intelligence API | 💵$1/1K

webdata_labs/website-change-intelligence-api

[💵 $1.00 / 1K] Monitor public pricing, product, legal, and documentation pages. Detect meaningful changes with severity, price deltas, and before/after evidence.

WebData Labs

Website Tech Detector & SEO Analyzer API

george.the.developer/website-intelligence-api

Detect any website tech stack: CMS, frameworks, analytics, hosting, ecommerce. Get SEO scores with actionable issues. Extract URL metadata and OG tags.

George Kioko

Domain WHOIS Lookup - Age, Expiry, Registrar, DNS

george.the.developer/domain-whois-lookup

Look up WHOIS data, domain age, expiration date, registrar, nameservers, and DNS records for any domain. Bulk lookup supported.

George Kioko

AI Content Detector - GPT, Claude, Gemini Text Check

george.the.developer/ai-content-detector

Detect AI-generated text using statistical analysis. Analyzes perplexity, burstiness, sentence patterns, vocabulary diversity, and more. No external API key needed.

George Kioko

Threads Scraper - Posts, Profiles & Search

webdata_labs/threads-scraper

[$3.00 / 1K] Scrape public Meta Threads posts, profiles, keyword search results, engagement metrics, media, hashtags, mentions, and timestamps without a Threads login.

WebData Labs

5.0

(1)