# Technical SEO Auditor — Screaming Frog Alternative (`khadinakbar/screamingfrog-alternative`) Actor Audit technical SEO across public websites. Crawl raw HTML and return one prioritized report per start URL covering titles, descriptions, canonicals, headings, status codes, image alt text, and duplicate titles. - **URL**: https://apify.com/khadinakbar/screamingfrog-alternative.md - **Developed by:** [Khadin Akbar](https://apify.com/khadinakbar) (community) - **Categories:** SEO tools, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $250.00 / 1,000 seo audit reports This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Technical SEO Auditor — Screaming Frog Alternative Audit technical SEO across public websites. Crawl raw HTML and return one prioritized report per start URL covering titles, descriptions, canonicals, headings, status codes, image alt text, and duplicate titles. For technical SEO teams, each dataset item is one site audit report per accepted start URL, with page-level evidence inside the report. ### Workflow: put the results to work Set the website, crawl scope, and page cap for a repair sprint. Review the prioritized findings and their page evidence, then group fixes by issue type. Each dataset item is a site audit report, so use the separate page-level crawler when your next step needs one row per page. ### Best fit for this Actor Start here when you have a public site URL and want a concise technical SEO report that explains which finding types affect which sampled URLs. - Audit public, raw-HTML pages on one hostname with a defined page and depth budget. - Feed a source URL, severity, issue count, and sample affected pages into a repair triage or reporting workflow. - Run the same bounded request again when a fresh snapshot is needed. - Keep collection read-only: the Actor does not log in, alter the target, submit forms, or access a private network. For a row-per-page cloud export rather than one prioritized report, use the public Website SEO Spider. It returns per-URL on-page fields for spreadsheet-style or downstream data processing. ### When a desktop SEO suite is the better fit This is a deliberately narrow workflow alternative, not a replacement for every feature in a desktop crawler suite. Choose a broader product when the work centrally needs browser-rendered JavaScript, authenticated or staging environments, persisted crawl history and comparison, visualisations, custom extraction, broad desktop integrations, or a large unbounded crawl. ### How this Screaming Frog alternative compares This Actor is a focused alternative for creating an on-demand, source-linked technical SEO audit report from a public raw-HTML URL. Screaming Frog remains the broader choice for its desktop crawler environment and related suite workflows. | Decision | This Actor | Screaming Frog SEO Spider | Best fit | | -------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | Scope | One bounded public raw-HTML audit report per start URL | Broader desktop crawler suite | Choose this Actor for a compact cloud report; choose the desktop product for wider crawler work. | | Billing model | Pay per persisted report plus Apify platform usage | Annual per-user desktop licence | Compare the live Pricing tab and the licence terms against the actual crawl volume. | | Same-job cost | Depends on selected page budget, retrieval route, and platform usage | Depends on licence allocation and the buyer's desktop workflow | Neither model is universally lower cost; evaluate the same workload. | | Input | Public HTTP(S) start URL and documented crawl settings | Desktop product workflow and configuration | This Actor suits a JSON/API handoff; the desktop interface suits interactive crawling. | | Output | One Dataset report with issue counts, source-linked samples, timestamps, and terminal records | Desktop crawl reports and exports | Use this Actor when one structured report is the downstream entity. | | Automation | Repeatable Actor input with API-accessible Dataset and terminal records | Desktop scheduling and integration features | Use the route that fits the existing operator workflow. | | Reliability evidence | Terminal outcomes, warnings, Dataset, OUTPUT, and RUN\_SUMMARY are explicit | No comparative reliability test is asserted here | Interpret each product using its own current evidence and workload. | Screaming Frog is a trademark of its owner. This independent Actor is not affiliated, associated, or endorsed by Screaming Frog. ### From a public URL to an audit queue An SEO lead can start with a production marketing-site URL, choose a 25-page budget, and receive one report that ranks the observed issue groups by severity and affected-page count. The lead can assign priorityFindings and their sampleUrls to an editor or developer, retain startUrl and scannedAt as provenance, and rerun the same bounded input after a release. The Actor keeps change decisions with the operator. It reports observed raw-HTML signals and labels coverage as bounded, partial, robots-governed, or incomplete when that context matters. ### Quick start and input | Field | Required | Meaning | | ------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------- | | startUrls | Yes | One to five authorized public HTTP(S) URLs. Each accepted URL produces at most one report. | | crawlScope | No | same-hostname follows only exact-hostname HTML links; page-only audits only the supplied URLs. | | maxPages | No | Per-report hard cap from 1 to 100. Reaching it is marked as bounded coverage. | | maxDepth | No | Same-hostname link hops from the seed. 0 means the seed only. | | maxConcurrency | No | Parallel requests per start URL. Lower it for a rate-limited target you are authorized to audit. | | respectRobotsTxt | No | Reads a public robots policy before enqueueing matching paths. Keep it enabled unless you have authority to do otherwise. | | ignoreUrlParameters | No | Removes query strings for deduplication so tracking variants stay outside the page budget. | | fetchMode | No | direct is the lower-cost default; residential uses Apify Proxy and can add platform usage. | | proxyConfiguration | No | Optional advanced Apify Proxy configuration that overrides fetchMode. It does not add login or browser-rendering support. | #### Quick start ```json { "startUrls": [ { "url": "https://example.com" } ], "crawlScope": "same-hostname", "maxPages": 25, "maxDepth": 3, "maxConcurrency": 3, "respectRobotsTxt": true, "ignoreUrlParameters": true, "fetchMode": "direct" } ``` Use only targets you are authorized to inspect. The Actor rejects local, private, credentialed, and non-HTTP(S) starts, then verifies that accepted start hostnames resolve publicly. It follows only same-hostname links and does not follow external or unsafe redirects. ### Output data | Field | Meaning | | --------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | reportId | Identifier for the audit snapshot. | | outcome | COMPLETE, PARTIAL, VALID\_EMPTY, or UPSTREAM\_FAILED. | | startUrl and scannedAt | Source URL and collection timestamp for freshness and provenance. | | pagesCrawled, pagesFailed, maxPages, coverageStatus | Bounded coverage context for interpreting the report. | | issueCounts | Totals by severity and finding code. | | priorityFindings | Up to ten issue groups ranked by severity and affected-page count, with sample source URLs. | | pageSamples | A small set of observed page URLs, statuses, titles, link depth, and finding codes. | | failureSamples and warnings | Retrieval, robots, timeout, proxy, or coverage context that should not be mistaken for findings. | Representative result shape: ```json { "reportId": "seo-audit-example", "reportType": "technical-seo-audit", "outcome": "COMPLETE", "startUrl": "https://example.com/", "crawlScope": "page-only", "fetchMode": "direct", "pagesCrawled": 1, "pagesFailed": 0, "maxPages": 5, "hitMaxPages": false, "coverageStatus": "complete", "issueCounts": { "total": 4, "critical": 0, "error": 0, "warning": 3, "notice": 1, "byCode": { "title_too_short": 1, "meta_description_missing": 1, "canonical_missing": 1, "thin_content": 1 } }, "priorityFindings": [ { "code": "canonical_missing", "severity": "warning", "message": "Page has no resolvable canonical URL.", "affectedPages": 1, "sampleUrls": ["https://example.com/"] } ], "pageSamples": [ { "url": "https://example.com/", "statusCode": 200, "title": "Example Domain", "issueCodes": ["title_too_short", "meta_description_missing", "canonical_missing", "thin_content"], "crawlDepth": 0 } ], "failureSamples": [], "warnings": [], "scannedAt": "1999-12-31T23:59:59.000Z" } ``` The Actor also writes two control records to the default key-value store: - OUTPUT is a concise terminal summary for a workflow branch. - RUN\_SUMMARY contains accepted inputs, report summaries, page totals, event counts, and warnings. ### Finding semantics The current raw-HTML checks include missing, short, and long titles; missing, short, and long meta descriptions; missing canonicals; noindex; missing or multiple H1s; short visible body text; images without non-empty alt text; HTTP errors and redirects; and duplicate titles across the crawled set. These are audit signals, not universal prescriptions. For example, noindex, a short page, or a missing canonical can be intentional. Use the source-linked sample URLs and the target's business context before making a change. ### Builder's note I designed this Actor around one report per start URL after building and testing the raw-HTML crawler path. That report-first shape keeps the Dataset useful for a triage queue: it preserves source URLs, timestamps, coverage context, and the highest-priority issue groups without making a downstream system infer meaning from a large raw crawl export. ### API and automation Start a run through the Apify API: ```bash curl -X POST "https://api.apify.com/v2/acts/khadinakbar~screamingfrog-alternative/runs?token=$APIFY_TOKEN" \ -H "content-type: application/json" \ -d '{ "startUrls": [{ "url": "https://example.com" }], "crawlScope": "page-only", "maxPages": 1, "maxDepth": 0, "fetchMode": "direct" }' ``` Read the completed Dataset with the returned defaultDatasetId. For a workflow branch, read the default key-value-store records named OUTPUT and RUN\_SUMMARY after the run has finished. JavaScript example: ```javascript import { ApifyClient } from 'apify-client'; const client = new ApifyClient({ token: process.env.APIFY_TOKEN }); const run = await client.actor('khadinakbar/screamingfrog-alternative').call({ startUrls: [{ url: 'https://example.com' }], crawlScope: 'page-only', maxPages: 1, maxDepth: 0, fetchMode: 'direct', }); const { items } = await client.dataset(run.defaultDatasetId).listItems(); console.log(items[0]); ``` #### Prompt card for an AI agent > Audit this authorized public website as a raw-HTML technical SEO report. Start at https://example.com, stay on the same hostname, inspect at most 25 pages, respect robots.txt, and return the report outcome, issue counts, priority findings, sample URLs, coverage status, and collection time. If the site requires a logged-in or JavaScript-rendered crawl, explain that this workflow does not support it. The required starting information is a public URL. The action is read-only. An agent should retrieve the Dataset report and OUTPUT/RUN\_SUMMARY after completion rather than infer a result from a terminal status alone. ### Limits, reliability, and troubleshooting - **Raw HTML only:** JavaScript rendering is intentionally out of scope. A client-rendered page can have fewer visible signals than a browser-rendered audit. - **Public access only:** No login, cookies, forms-based authentication, private networks, local hosts, or staging environments. - **Robots policy:** With respectRobotsTxt enabled, disallowed paths are skipped. An unreadable public robots file is reported in warnings. - **Same-hostname boundary:** External links and external redirects are not crawled. - **Bounded coverage:** Hitting maxPages makes coverageStatus bounded and the report outcome PARTIAL, even when retrieved pages are valid. - **Partial retrieval:** Exhausted request failures are retained in failureSamples; successful page observations remain available in the report. - **No output is distinct from success:** SUCCEEDED alone is not the audit result. Read the Dataset and terminal records to distinguish complete, partial, empty, and upstream-unavailable runs. - **No third-party SEO data API:** The core uses direct public-page retrieval. The only optional recovery route is an Apify Proxy configuration supplied by the caller; it is not a data provider. ### Freshness and provenance Every report includes its public startUrl and scannedAt timestamp. The report is a current observation of the pages reachable during that run, subject to the page budget, raw-HTML behavior, target availability, robots policy, and selected retrieval route. It is not a historical crawl archive, a search-engine ranking guarantee, or proof of indexation. ### Legal and responsible use Use this Actor only for public pages you are authorized to audit and in compliance with applicable law, target terms, and robots policies. Its intended boundary is publicly reachable content, with access-control and private-network protections preserved. The Actor is read-only and does not provide legal advice. ### FAQ #### Does this replace the complete Screaming Frog application? No. It replaces a narrow workflow: producing a current, programmatic technical SEO audit report from an authorized public raw-HTML site. Use the broader desktop suite when its rendering, history, visualisation, integration, or interactive features are required. #### Why is the report PARTIAL? PARTIAL means some usable observations exist, but the requested coverage was bounded or a retrieval warning occurred. Check coverageStatus, pagesFailed, failureSamples, and warnings before treating finding counts as site-wide. #### Can I retrieve raw per-page rows? This Actor keeps its Dataset homogeneous: one report per start URL. For an export of per-URL SEO fields, continue with the Website SEO Spider workflow instead. #### Does the Actor use an external SEO API? No. The core retrieves public raw HTML directly. Apify Proxy is an optional network route, not an SEO data provider. ### Your feedback If a supported public raw-HTML audit produces an unclear report, include the input shape, expected report field, observed OUTPUT outcome, and any relevant warning. Please never include credentials or private URLs in public support messages. ### Pricing and run costs This Actor uses **Pay per event plus Apify platform usage**. The [Pricing tab](https://apify.com/khadinakbar/screamingfrog-alternative/pricing) lists the current event rates and billing terms. | Event | Billing unit | When it applies | |---|---|---| | `apify-actor-start` | Actor Start | Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event). | | `seo-audit-report` | SEO audit report | Charged once for each validated source-linked technical SEO report persisted to the Dataset. | Run cost combines the charged events and Apify platform usage. Review the run charge limit and requested result count before starting. ### Connect an AI agent Use the [Apify MCP configurator](https://mcp.apify.com) to choose an available client connection. Inspect this Actor’s current input schema and required credentials before running it. # Actor input Schema ## `startUrls` (type: `array`): One to five authorized public website URLs to audit. Each accepted URL creates one source-linked report. Local, private, credentialed, and non-HTTP URLs are rejected. ## `crawlScope` (type: `string`): same-hostname follows public HTML links only on the exact starting hostname; page-only audits each supplied URL without following links. ## `maxPages` (type: `integer`): Hard per-start cap for fetched public pages. Default 25, maximum 100. Reaching the cap is reported as bounded coverage. ## `maxDepth` (type: `integer`): Maximum same-hostname link hops from the supplied start URL. 0 audits the start URL only. Default 3, maximum 10. ## `maxConcurrency` (type: `integer`): Parallel requests per start URL. Default 3, maximum 10. Lower this for a rate-limited site you are authorized to audit. ## `respectRobotsTxt` (type: `boolean`): When enabled, skip paths disallowed for this crawler user agent. Disable only when you are authorized to audit beyond the public robots policy. ## `ignoreUrlParameters` (type: `boolean`): When enabled, strip query strings during URL deduplication so tracking variants do not consume the page budget. Canonical tags are still reported as observed. ## `fetchMode` (type: `string`): Direct HTTP is the lower-cost default. Use Residential Apify Proxy only for authorized public targets after repeatable direct rate limits; additional platform usage can apply. ## `proxyConfiguration` (type: `object`): Optional Apify Proxy configuration that overrides Fetch mode. It does not add browser rendering, login support, or access to private networks. ## Actor input object example ```json { "startUrls": [ { "url": "https://example.com" } ], "crawlScope": "same-hostname", "maxPages": 5, "maxDepth": 2, "maxConcurrency": 3, "respectRobotsTxt": true, "ignoreUrlParameters": true, "fetchMode": "direct", "proxyConfiguration": { "useApifyProxy": true } } ``` # Actor output Schema ## `reports` (type: `string`): Dataset rows with prioritized findings, page samples, coverage, source URL, and collection time. ## `output` (type: `string`): Concise outcome, report count, page count, event count, and warnings. ## `runSummary` (type: `string`): Detailed terminal state, accepted inputs, report summaries, and billing count. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ { "url": "https://example.com" } ], "crawlScope": "same-hostname", "maxPages": 5, "maxDepth": 2, "maxConcurrency": 3, "respectRobotsTxt": true, "ignoreUrlParameters": true, "fetchMode": "direct" }; // Run the Actor and wait for it to finish const run = await client.actor("khadinakbar/screamingfrog-alternative").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": [{ "url": "https://example.com" }], "crawlScope": "same-hostname", "maxPages": 5, "maxDepth": 2, "maxConcurrency": 3, "respectRobotsTxt": True, "ignoreUrlParameters": True, "fetchMode": "direct", } # Run the Actor and wait for it to finish run = client.actor("khadinakbar/screamingfrog-alternative").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ { "url": "https://example.com" } ], "crawlScope": "same-hostname", "maxPages": 5, "maxDepth": 2, "maxConcurrency": 3, "respectRobotsTxt": true, "ignoreUrlParameters": true, "fetchMode": "direct" }' | apify call khadinakbar/screamingfrog-alternative --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,khadinakbar/screamingfrog-alternative" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/UpwZOAzPehGWAB8Af/builds/sjfUUaahhD0MbtJEm/openapi.json