Ai SEO Content Curator

AI SEO Content Curator crawls a website with a real headless Chrome browser and returns, for every page, an on-page SEO audit, SEO metadata, HTTP details and the main content as clean Markdown. One run gives you both an SEO checklist per URL and LLM-ready text, so you can audit a site and feed its content to an AI model from the same dataset.

It follows internal links on the same domain. It does not log in, solve CAPTCHAs, use proxies, measure Core Web Vitals, or give a 0–100 SEO score.

What does AI SEO Content Curator do?

For each page it scrapes, the Actor returns:

• SEO metadata: title, meta description, first H1, lang, canonical, word count, internal and external link counts.
• On-page SEO audit: title and description present and length, H1 and H2 counts, heading structure (exactly one H1), canonical, lang, viewport, robots meta, Open Graph, Twitter tags, microdata (itemtype) structured data. JSON-LD is not checked.
• HTTP details: status code, content type, page size and load time, from a separate plain HTTP request.
• Main content as Markdown: the <main> element, else a single <article>, else <body>, without navigation, headers, footers, scripts or styles.

Counts are measured on the page after navigation, header, footer, scripts and iframes are removed; relative links are not counted as internal.

Start URLs that redirect to another domain (for example a .fr that lands on a .com) are followed, and the crawl stays on the landing domain.

Who is AI SEO Content Curator for?

• SEO consultants and agencies running quick on-page audits for clients.
• Content and marketing teams checking titles, descriptions and headings across a site.
• AI developers who want page content as Markdown plus metadata for RAG or content analysis.
• Web developers checking status codes and missing tags after a release.

Use cases for an SEO audit with Markdown content

• Find pages with a missing meta description, several H1s or no canonical.
• Export a site's titles and descriptions to a spreadsheet for rewriting.
• Feed page content and its SEO metadata to an LLM to suggest better titles and descriptions.
• Re-run on a schedule to catch SEO regressions.

What data can AI SEO Content Curator extract?

Field	Type	Description
url	string	Page that was scraped
content	string	Main page content as Markdown
info.title	string	Page title
info.description	string	Meta description
info.htmlLang	string	lang attribute of the page
info.wordCount	integer	Words in the main page area (header, nav and footer removed)
network.HttpResponseCode	integer	HTTP status code
network.loadTime	number	Request time in seconds
seoAudit.descriptionPresent	string	ok or absent
seoAudit.headingStructureOk	string	ok when the page has exactly one H1
seoAudit.canonicalPresent	string	ok when a canonical link exists
seoAudit.ogTagsPresent	string	ok when Open Graph tags exist
timestamp	string	UTC time of the scrape

The full field list is in the Output tab.

How to run an SEO audit and scrape website content

1. Open the Actor and go to the Input tab.
2. Add one or more Start URLs, for example https://docs.apify.com/.
3. Set Maximum Depth (0 = only the start URLs, 1 = also the pages they link to).
4. Set Maximum URLs to cap the number of pages.
5. Click Start and download the audit from the Output tab as JSON, CSV or Excel.

How much will it cost to audit a website?

You pay only per result, no subscription and no rental. One result is one scraped page. Pages that fail to load are not stored and not charged. Pages that load with an error status (for example 404) are stored with their real status code, because that is a valid audit result. A small standard Actor start event also applies.

Current prices, including discounts on higher Apify plans, are on the Pricing tab. Maximum URLs caps the page count, and the run stops when your maximum cost per run is reached.

Input

See the Input tab for all options. This is the input of the run used for the output below:

{
    "start_urls": [{ "url": "https://docs.apify.com/" }],
    "max_depth": 1,
    "max_urls": 5
}

• start_urls: URLs as {"url": "..."} objects or plain strings.
• search_engine: an informational label copied into each result; it does not change scraping.

Output

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. First item from a real run on docs.apify.com, trimmed:

{
    "url": "https://docs.apify.com/",
    "info": {
        "status": "complete",
        "title": "Apify Documentation",
        "description": "",
        "htmlLang": "en",
        "wordCount": 480,
        "linksCount": 54,
        "linksInternalCount": 25,
        "linksExternalCount": 27
    },
    "network": {
        "HttpResponseCode": 200,
        "HttpContentType": "text/html; charset=utf-8",
        "loadTime": 0.041,
        "pageSizeCompressed": 75777
    },
    "seoAudit": {
        "titlePresent": "ok",
        "titleLength": 19,
        "descriptionPresent": "absent",
        "h1Count": 0,
        "h2Count": 8,
        "headingStructureOk": "problematic",
        "canonicalPresent": "ok",
        "ogTagsPresent": "ok",
        "twitterTagsPresent": "absent",
        "structuredDataPresent": "missing"
    },
    "content": "## Platform The full reference of the Apify platform. ### Apify API Interact with the Apify platform from your applications. …",
    "timestamp": "2026-10-16T15:30:49Z",
    "search_engine": "Google"
}

In this example the audit reports no meta description, no H1 and no microdata in the main page area.

Tips for SEO audits

• Start with max_depth 1 and a small Maximum URLs to check the output, then scale up.
• The browser starts once per run, so auditing several pages per run is more efficient than many single-page runs.
• Use a start URL on the final domain to skip a redirect.

Integrations: API, AI agents (MCP), Make, Zapier and n8n

• Apify API: run audits and fetch results from Python, JavaScript or cURL. See the API tab.
• AI agents (MCP): add quaking_pail/ai-seo-content-markdown-scraper through the Apify MCP server so your AI assistant can audit a page and read its content.
• No-code: send audits to Google Sheets, Looker Studio or Slack with Make, Zapier, n8n or webhooks.
• Scheduling: re-audit key pages weekly and compare results.

FAQ, disclaimers and support

Is it legal to scrape websites for an SEO audit?

You should be aware that your results could contain personal data. Personal data is protected by the GDPR in the European Union and by other regulations around the world. You should not scrape personal data unless you have a legitimate reason to do so. If you're unsure whether your reason is legitimate, consult your lawyers.

Why did I get only one page?

max_depth is 0, the page has no links on the same domain, or Maximum URLs is 1. The log shows how many internal links were found on each page.

Does it give an SEO score?

No. It returns the individual checks (ok, absent, problematic, missing) so you can filter and prioritise them yourself.

How do I report a problem?

Open an issue in the Issues tab with the run ID. If the Actor saved you time, a rating on the Reviews tab helps other users find it.

Related actors by quaking_pail

• AI Website Content Markdown Scraper: Markdown content only, for LLM and RAG pipelines.
• Google Condensed SERP Scraper: see what Google shows for a query, as one LLM-ready text.
• Website Contacts Crawler: emails, phones and team names from company websites.
• Linkedin Job Scraping: LinkedIn job listings with full descriptions.