Website Content Vector Retriever
Pricing
Pay per usage
Go to Apify Store
Pricing
Pay per usage
A URL of a website where to fetch the web pages from. The URL can be a top-level domain like https://example.com, a subdirectory https://example.com/some-directory/, or a specific page https://example.com/some-directory/page.html.
The maximum number pages to crawl. It includes the start URLs, pagination pages, pages with no content, etc. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway.
The maximum number of resulting web pages to store. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway. If both Max page and Max results are defined, then the crawler will finish when the first limit is reached. Note that the crawler skips pages with the canonical URL of a page that has already been crawled, hence it might crawl more pages than there are results.
The maximum size of each chunk in characters or tokens.
The number of overlapping characters or tokens between consecutive chunks.
The maximum age of cached records, in days, to consider them useful. By default, set to 30.
Name of the key-value store where the actor will store its cache.
When set, it invokes a special mode in which we index the crawl to the database, rather than answer user queries. This is invoked using webhook from Website Content Crawler, scheduled by this actor. Note that the URL and Query need to also be passed, but the URL can be dummy, as they it's ignored.
