Who this is for: Technical SEO and indexing QA teams.

What you actually get: Inspect robots and Googlebot meta directives, including declared noindex.

Try one public URL first

Input example:

{"urls":["https://scrapeme.live/shop/Pikachu/"]}

What to do next: Test a public page intended to be indexed, then review indexing directives.

Pricing: Pay per event; view this Actor’s live Pricing tab before larger batches. A successfully processed page can be billable even when no issue is detected. Results can be exported as JSON or CSV.

Important limits: Does not show Google index status or replace robots.txt and crawl audits. Supports applicable static public HTTPS metadata only, without site logins or JavaScript rendering.

Related PAL tool: https://apify.com/practical_ai_lab_jp/my-actor-13. Independently usable; no bundle is required.

---

PAL Robots Meta Audit

Review robots and Googlebot meta directives that affect search indexing.

Ideal users and practical workflow

Best for: SEO specialists, store owners and content teams.

What it checks: Review robots and Googlebot meta directives that affect search indexing. Upload a short list of public HTTPS URLs to receive an individual evidence-based record and issue flags per page. Review the flagged URLs first, then export the Dataset as JSON or CSV for your team's workflow.

When it is useful: Before releasing a new page, after a redesign, or during a monthly search-indexing review.

What it does not do: It does not crawl an entire website automatically, render JavaScript, guarantee a search ranking, or change the target site. Check the exact limitations below before purchasing runs.

Why use this PAL Actor?

Get a structured, repeatable report for this specific issue without opening dozens of webpages by hand. Each input URL is analyzed independently. The output identifies data found on the page and flags likely omissions for manual review. Save output in Apify Dataset and export to JSON, CSV or Excel.

Example input

{"urls": ["https://scrapeme.live/shop/Pikachu/"]}

Results

Each successful fetch produces a Dataset row with sourceUrl, finalUrl, mode, issueCount, issues and details. Issue flags indicate markup detected in static HTML; they are not a guarantee of legal compliance, accessibility certification, search rank, or a live inventory position. Missing data can be published through JavaScript or other channels not read here.

Billing

Pay per event: pal-robots is triggered for each successfully fetched and audited page at the price displayed by Apify Store. URL errors generate a free error row. Maximum 18 URLs per run. No paid API keys, logins, or third-party services required.

Technical limits

Public HTTPS only; no JavaScript rendering, captcha bypass or private pages. Each response is at most 1.5 MB, 11-second network timeout. The site may block bots. Output reflects visible HTML at retrieval time, not a complete website audit. Running the Actor does not itself fix issues.

More Practical AI Lab (PAL) tools

Use independent PAL tools in a natural sequence: first capture price and stock, then audit the corresponding product markup; or locate policy links and inspect returns/shipping markup. Each Actor is separately usable, with no mandatory bundle.

• PAL Product Price & Stock Snapshot
• PAL Return Policy Schema Auditor
• PAL Shipping Promise Schema Audit
• PAL Product GTIN & SKU SEO Auditor
• PAL Store Policy Link Mapper

Browse the PAL developer profile for additional focused audit tools.

---

Practical AI Lab | Catalog product 04 | SEO