Glob patterns matching URLs of pages that will be included in crawling.
Setting this option will disable the default Start URLs based scoping and will allow you to customize the crawling scope yourself. Note that this affects only links found on pages, but not Start URLs - if you want to crawl a page, make sure to specify its URL in the Start URLs field.
For example https://{store,docs}.example.com/**
lets the crawler to access all URLs starting with https://store.example.com/
or https://docs.example.com/
, and https://example.com/**/*\?*foo=*
allows the crawler to access all URLs that contain foo
query parameter with any value.
Learn more about globs and test them here.