Can be main page URL or any category URLs. Article pages are found and enqueued from these.
Maximum number of articles
Maximum number of valid articles scraped. The crawler will stop automatically after reaching this number.
Only articles from this day to present will be scraped. If empty, all articles will be scraped. Format is YYYY-MM-DD, e.g. 2019-12-31, or Number type e.g. 1 week or 20 days
Only new articles
If true, will scrape only new articles each time you run it. All URLs you scraped are compared with saved ones. Scraped articles are saved in one dataset per each domain, datasets are named 'ARTICLES-SCRAPED-domain'
Default value of this property is