No credit card required

Milvus Integration

apify/milvus-integration

No credit card required

This integration transfers data from Apify Actors to a Milvus (running at docker or at Zilliz cloud) and is a good starting point for a question-answering, search, or RAG use case.

Do you want to learn more about this Actor?

Get a demo

Change Log

0.1.3 (2024-09-11)

Set performChunking to true by default as some users were not aware of the setting their runs were failing.

0.1.2 (2024-09-04)

Pinecone - add support to update Pinecone index namespace (Optional arg: PineconeIndexNamespace)

0.1.1 (2024-07-24)

Fixed issue with Weaviate database when collection was not created before inserting data.

0.1.0 (2024-07-24)

Introduced the deleteExpiredObjects setting to enable or disable the automatic deletion feature.
Previously, outdated data deletion was tied to the enableDeltaUpdates setting. Now, the deletion feature can be controlled independently.

0.0.4 (2024-07-12)

Add integration to Milvus database (Zilliz)

0.0.3 (2024-06-26)

The key in the metadata dictionary has been refactored from 'id' to 'chunk_id'. This change was necessitated by the requirements of the Weaviate database.
A new function, 'get_by_item_id', has been introduced. This function fetches objects from the database, including their ids. With this change, it's no longer necessary to store the 'chunk_id' in the metadata. However, storing the 'chunk_id' remains beneficial for referencing chunks within the metadata."

0.0.2 (2024-06-24)

Add PostgreSQL database integration (PGVector integration)

0.0.1 (2024-06-20)

Improved README.md and error messages

0.0.0 (2024-05-24)

Initial import

Developer

Apify

Actor metrics

1 monthly user
0 stars
60.0% runs succeeded
Created in Jul 2024
Modified 5 days ago

Categories

For creators

Integrations

Website Content Crawler

apify/website-content-crawler

Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗LangChain, LlamaIndex, and the wider LLM ecosystem.

Apify

20.9k

451

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

5.2k

Web Scraper

apify/web-scraper

Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Apify

68.4k

163

Google Search Results Scraper

apify/google-search-scraper

Scrape Google Search Engine Results Pages (SERPs). Select the country or language and extract organic and paid results, ads, queries, People Also Ask, prices, reviews, like a Google SERP API. Export scraped data, run the scraper via API, schedule and monitor runs, or integrate with other tools.

Apify

47.3k

169

Puppeteer Scraper

apify/puppeteer-scraper

Crawls websites with the headless Chrome and Puppeteer library using a provided server-side Node.js code. This crawler is an alternative to apify/web-scraper that gives you finer control over the process. Supports both recursive crawling and list of URLs. Supports login to website.

Apify

4.1k

Facebook Reel Scraper

jeremy_frost/facebook-reel-scraper

This scraper will get Reel details from Facebook pages of your choice.

Jeremy Frost

134

Smart Article Extractor

lukaskrivka/article-extractor-smart

📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.

Lukáš Křivka

3.8k

Legacy PhantomJS Crawler

apify/legacy-phantomjs-crawler

Replacement for the legacy Apify Crawler product with a backward-compatible interface. The actor uses PhantomJS headless browser to recursively crawl websites and extract data from them using a piece of front-end JavaScript code.

Apify

1.6k

Reddit Scraper Lite

trudax/reddit-scraper-lite

Pay Per Result, unlimited Reddit web scraper to crawl posts, comments, communities, and users without login. Limit web scraping by number of posts or items and extract all data in a dataset in multiple formats.

Gustavo Rudiger

3.4k

GPT Scraper

drobnikj/gpt-scraper

Extract data from any website and feed it into GPT via the OpenAI API. Use ChatGPT to proofread content, analyze sentiment, summarize reviews, extract contact details, and much more.

Jakub Drobník

Python on Docker: how to dockerize your app

What is retrieval-augmented generation (and why use it for LLMs)?

How to run Puppeteer in a Docker container

Build new tools

Are you a developer? Build your own Actors and run them on Apify.

Learn more

Get a custom solution

Get a custom web scraping or RPA solution.

Book a demo