Ai Powered Scraper
Pricing
$10.00 / 1,000 results
Go to Apify Store
Ai Powered Scraper
AI Powered Scraper using LangChain and OpenAI.
0.0 (0)
Pricing
$10.00 / 1,000 results
1
1
1
Last modified
3 days ago
Pricing
$10.00 / 1,000 results
AI Powered Scraper using LangChain and OpenAI.
0.0 (0)
Pricing
$10.00 / 1,000 results
1
1
1
Last modified
3 days ago
startUrlsarrayOptional
One or more URLs of pages where the crawler will start. Note that the Actor will additionally only crawl sub-pages of these URLs. For example, for start URL https://www.example.com/blog, it will crawl pages like https://example.com/blog/article-1, but will skip https://example.com/docs/something-else.
maxCrawlPagesintegerOptional
The maximum number pages to crawl. It includes the start URLs, pagination pages, pages with no content, etc. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway.
Default value of this property is 9999999
openAIApiKeystringOptional
Enter your OpenAI account and an API key. This is needed for vectorizing the data and also to be able to prompt the OpenAI model.
forceRecrawlbooleanOptional
If enabled, the data will be re-crawled even if cached vector index is available.
Default value of this property is false
loadUrlsFromSitemapsbooleanOptional
If enabled, the scraper will automatically find and load URLs from sitemap.xml files.
Default value of this property is false
respectRobotsTxtbooleanOptional
If enabled, the scraper will respect the robots.txt file and avoid crawling disallowed pages.
Default value of this property is true
crawlerTypeEnumOptional
Select the crawler type based on your needs
"adaptive": string"cheerio": string"playwright": string"jsdom": stringDefault value of this property is "adaptive"
userAgentstringOptional
Custom User-Agent string to use for requests
Default value of this property is "Mozilla/5.0 (compatible; AI-Powered-Scraper/1.0)"