Robots-Noindex Conflict Checker

Store Positioning

Store title: Robots-Noindex Conflict Checker

Short description: Detect conflicts between robots.txt blocking and page-level meta/X-Robots noindex signals.

SEO title: Robots-Noindex Conflict Checker — technical SEO, web, and domain audit

SEO description: Detect conflicts between robots.txt blocking and page-level meta/X-Robots noindex signals. Use it to find crawlability, indexability, security, metadata, and page-quality issues with evidence-backed rows and audit reports.

Categories: SEO_TOOLS

Keywords: robots, noindex, conflict, checker, robots.txt, web/domain audit

Fixed-Inclusive PPE Pricing

This actor uses pay-per-event pricing. Event prices include Apify platform usage; users are not expected to pay a separate platform-usage pass-through charge for the configured pricing model.

• Tier: W1 — Web/domain audit
• Primary event: page-audited at $0.00490 base
• Default max charge: $10.00
• Store discounts: FREE/BRONZE base, SILVER discounted, GOLD deepest approved discount

Event set:

• actor-start: base $0.00500, GOLD $0.00400. Robots Noindex Conflict Checker: charged when actor start is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
• page-audited: base $0.00490, GOLD $0.00392. Robots Noindex Conflict Checker: charged when page audited is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
• rule-parsed: base $0.00372, GOLD $0.00298. Robots Noindex Conflict Checker: charged when rule parsed is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
• finding-emitted: base $0.00372, GOLD $0.00298. Robots Noindex Conflict Checker: charged when finding emitted is completed. The price includes Apify platform usage; no separate usage pass-through is intended.
• audit-report-generated: base $0.05000, GOLD $0.04000. Robots Noindex Conflict Checker: charged when audit report generated is completed. The price includes Apify platform usage; no separate usage pass-through is intended.

Public Task Concepts

• Audit public URLs for Robots-Noindex Conflict Checker issues
• Find Robots-Noindex Conflict Checker problems across a website
• Generate a technical Robots-Noindex Conflict Checker report
• Audit robots issues across supplied public inputs
• Extract evidence-backed robots rows for client QA

What It Does

Detect conflicts between robots.txt blocking and page-level meta/X-Robots noindex signals.

This local package is part of the Junipr Apify actors 171-200 premium build for ChatGPT review.

What It Does Not Do

• It does not call live Apify APIs.
• It does not publish, upload Store assets, create public tasks, or configure live PPE.
• It does not claim publish readiness.
• It does not make unbounded network calls in the supplied examples path.

Input Fields

Seed shape: domains or URLs, maxDomains, timeoutMs, strictMode, includeSummary

The tiny fixture is available at examples/input.tiny.json and mirrored at fixtures/input.tiny.json for the local runner.

Output Fields

• url
• robotsUrl
• statusCode
• userAgent
• directiveType
• directiveValue
• issueType
• severity
• evidence
• recommendation

Public Task Examples

Find robots.txt and noindex conflicts

• Search intent: find robots.txt and noindex conflicts
• Specific input: Fixture input shaped for find robots.txt and noindex conflicts.
• Expected output: Dataset rows with url, robotsUrl, statusCode, userAgent plus local KVS report evidence.
• Why run it: SEO teams, agencies, site auditors would run this to get a bounded local proof before any live Apify review or production use.

Check private pages blocked from noindex discovery

• Search intent: check private pages blocked from noindex discovery
• Specific input: Fixture input shaped for check private pages blocked from noindex discovery.
• Expected output: Dataset rows with url, robotsUrl, statusCode, userAgent plus local KVS report evidence.
• Why run it: SEO teams, agencies, site auditors would run this to get a bounded local proof before any live Apify review or production use.

Review crawler directives against page headers

• Search intent: review crawler directives against page headers
• Specific input: Fixture input shaped for review crawler directives against page headers.
• Expected output: Dataset rows with url, robotsUrl, statusCode, userAgent plus local KVS report evidence.
• Why run it: SEO teams, agencies, site auditors would run this to get a bounded local proof before any live Apify review or production use.

Prepare SEO conflict rows for a launch checklist

• Search intent: prepare seo conflict rows for a launch checklist
• Specific input: Fixture input shaped for prepare seo conflict rows for a launch checklist.
• Expected output: Dataset rows with url, robotsUrl, statusCode, userAgent plus local KVS report evidence.
• Why run it: SEO teams, agencies, site auditors would run this to get a bounded local proof before any live Apify review or production use.

Audit redundant robots and meta-robots rules

• Search intent: audit redundant robots and meta-robots rules
• Specific input: Fixture input shaped for audit redundant robots and meta-robots rules.
• Expected output: Dataset rows with url, robotsUrl, statusCode, userAgent plus local KVS report evidence.
• Why run it: SEO teams, agencies, site auditors would run this to get a bounded local proof before any live Apify review or production use.

Tests

• Actor-specific fixture tests live in test/robots-noindex-conflict-checker.test.ts.
• Node-based local suites validate dataset schema conformance, output samples, local smoke runs, and PPE guard behavior.

FAQ

Is this live on Apify?

No. This is a local package for ChatGPT review only.

Is it publish-ready?

No. It needs later live tiny-run validation and console configuration before any publication decision.