Pricing

Pay per usage

Go to Store

Algolia Webcrawler

Try for free

jancurn/algolia-webcrawler

Developed by

Jan Čurn

Crawls a website using one or more sitemaps and imports the data to Algolia search index. The text content is identified using simple CSS selectors.

0.0 (0)

Pricing

Pay per usage

Monthly users

Last modified

4 years ago

Other Open source

Crawls a website using one or more sitemaps and imports the data to Algolia search index. The text content is identified using simple CSS selectors.

The actor simply runs the algolia-webcrawler NPM package on the Apify cloud, so that you don't need to deploy it yourself. You can run it easily using API or scheduler.

On input, the actor accepts a JSON configuration required by algolia-webcrawler. For details, see https://www.npmjs.com/package/algolia-webcrawler#configuration-options

Pricing

Pricing model

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

6.8k

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

Apify

78.9k

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

5.8k

Google Ads Scraper

silva95gustavo/google-ads-scraper

Extract text, image and video ads from Google Ads, scraped from the ad library provided by Google Ads Transparency Center. Gain access to ad details, ad copy, locations, and more. Dive deeper into the Google Ads Transparency Center for a competitive edge.

Gustavo Silva

1.2k

Playwright Scraper

apify/playwright-scraper

Crawls websites with the headless Chromium, Chrome, or Firefox browser and Playwright library using a provided server-side Node.js code. Supports both recursive crawling and a list of URLs. Supports login to a website.

Apify

1.1k

Extended GPT Scraper

drobnikj/extended-gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

1.4k

Similarweb scraper

curious_coder/similarweb-scraper

Scrape similarweb to get insights for websites in bulk including website traffic, pages per visit, time spent on website, bounce rate, rank, etc

Curious Coder

757

LinkedIn Ads Scraper

silva95gustavo/linkedin-ad-library-scraper

Extract ads of any format from LinkedIn Ad Library. Gain access to ad details, ad copy, media URL, call-to-action buttons and more. Dive deeper into the LinkedIn Ad Library for a competitive edge.

Gustavo Silva

1.3k

Legacy PhantomJS Crawler

apify/legacy-phantomjs-crawler

Replacement for the legacy Apify Crawler product with a backward-compatible interface. The actor uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of front-end JavaScript code.

Apify

1.6k

Website Screenshot Generator

apify/screenshot-url

Create a screenshot of a website based on a specified URL. The screenshot is stored as the output in a key-value store. It can be used to monitor web changes regularly after setting up the scheduler.

Apify

3.3k