Seo Robots Txt Checker
Pricing
$0.01 / actor start
Go to Apify Store
Seo Robots Txt Checker
Pricing
$0.01 / actor start
robotsUrl
Optional
The robots url.
userAgent
Optional
The user agent.
groupFound
Optional
The group found.
allowPaths
Optional
The allow paths.
[]disallowPaths
Optional
The disallow paths.
[]effectiveAllowPaths
Optional
The effective allow paths.
[]effectiveDisallowPaths
Optional
The effective disallow paths.
[]effectiveRuleSource
Optional
The effective rule source.
sitemaps
Optional
The sitemaps.
[]httpStatus
Optional
The http status.
status
Optional
The status.
message
Optional
The message.
error
Optional
The error.
agents
Optional
The agents.
[]rules
Optional
The rules.
[]