Website Content Vector Retriever
Pricing
Pay per usage
Go to Apify Store
Website Content Vector Retriever
0.0 (0)
Pricing
Pay per usage
5
20
1
Last modified
2 years ago
Pricing
Pay per usage
0.0 (0)
Pricing
Pay per usage
5
20
1
Last modified
2 years ago
urlstringRequired
A URL of a website where to fetch the web pages from. The URL can be a top-level domain like https://example.com, a subdirectory https://example.com/some-directory/, or a specific page https://example.com/some-directory/page.html.
maxCrawlPagesintegerOptional
The maximum number pages to crawl. It includes the start URLs, pagination pages, pages with no content, etc. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway.
Default value of this property is 1000
maxResultsintegerOptional
The maximum number of resulting web pages to store. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway. If both Max page and Max results are defined, then the crawler will finish when the first limit is reached. Note that the crawler skips pages with the canonical URL of a page that has already been crawled, hence it might crawl more pages than there are results.
Default value of this property is 9999999
chunkSizeintegerOptional
The maximum size of each chunk in characters or tokens.
Default value of this property is 1000
chunkOverlapintegerOptional
The number of overlapping characters or tokens between consecutive chunks.
Default value of this property is 200
crawlerTypeEnumOptional
"playwright:firefox": string"cheerio": stringDefault value of this property is "cheerio"
maxCacheAgeintegerOptional
The maximum age of cached records, in days, to consider them useful. By default, set to 30.
Default value of this property is 30
cacheKeyValueStoreNamestringOptional
Name of the key-value store where the actor will store its cache.
Default value of this property is "website-content-vector-retriever-cache"
indexWebsiteContentCrawledRunIdstringOptional
When set, it invokes a special mode in which we index the crawl to the database, rather than answer user queries. This is invoked using webhook from Website Content Crawler, scheduled by this actor. Note that the URL and Query need to also be passed, but the URL can be dummy, as they it's ignored.