embeddingBatchSize: (only Pinecone) batch size for embedding texts. Default: 1000, Minimum: 1.
usePineconeIdPrefix: (only Pinecone) Optimizes delta updates with a Pinecone ID prefix (item_id#chunk_id) when enableDeltaUpdates is true. Works only when database is empty.
New parameter dataUpdatesStrategy:
- Replaces enableDeltaUpdates.
- Automatically set to deltaUpdates if enableDeltaUpdates = true.
- Options: deltaUpdates, add, or upsert
Renamed deltaUpdatesPrimaryDatasetFields:
- Now dataUpdatesPrimaryDatasetFields.
- Automatically migrated if the old field is present.
Backward Compatibility:
- Legacy enableDeltaUpdates mappings and deltaUpdatesPrimaryDatasetFields are supported.

0.1.9 (2025-01-25)

Convert all fields used for delta updates to strings to avoid issues with different types of data.

0.1.8 (2024-12-20)

Do not create string from metadata fields
Increase backoff time to 900 seconds (15 minutes) (was 300)
Add Apify's badge to the README

0.1.7 (2024-10-25)

Increase backoff time to 300 seconds (was 120)

0.1.6 (2024-10-04)

Add retry logic with exponential backoff for Qdrant errors.
Use Langchain Fake Embeddings for testing.
Parallelize comparison of crawled data with the database.
Implement a count() method for Pinecone and Qdrant.

0.1.5 (2024-09-30)

Add retry logic with exponential backoff for Pinecone database to handle rate limiting.

0.1.4 (2024-09-24)

Change milvusUrl to milvusUri.
Remove milvusUsername and milvusPassword as they were moved to the milvusUri as part of the URI.

0.1.3 (2024-09-11)

Set performChunking to true by default as some users were not aware of the setting their runs were failing.

0.1.2 (2024-09-04)

Pinecone - add support to update Pinecone index namespace (Optional arg: PineconeIndexNamespace)

0.1.1 (2024-07-24)

Fixed issue with Weaviate database when collection was not created before inserting data.

0.1.0 (2024-07-24)

Introduced the deleteExpiredObjects setting to enable or disable the automatic deletion feature.
Previously, outdated data deletion was tied to the enableDeltaUpdates setting. Now, the deletion feature can be controlled independently.

0.0.4 (2024-07-12)

Add integration to Milvus database (Zilliz)

0.0.3 (2024-06-26)

The key in the metadata dictionary has been refactored from 'id' to 'chunk_id'. This change was necessitated by the requirements of the Weaviate database.
A new function, 'get_by_item_id', has been introduced. This function fetches objects from the database, including their ids. With this change, it's no longer necessary to store the 'chunk_id' in the metadata. However, storing the 'chunk_id' remains beneficial for referencing chunks within the metadata."

0.0.2 (2024-06-24)

Add PostgreSQL database integration (PGVector integration)

0.0.1 (2024-06-20)

Improved README.md and error messages

0.0.0 (2024-05-24)

Initial import

OpenAI Vector Store Integration

jiri.spilka/openai-vector-store-integration

This integration uploads data from Apify Actors to the OpenAI Vector Store linked to OpenAI Assistant.

Jiří Spilka

185

4.8

PGVector Integration

apify/pgvector-integration

This integration transfers data from Apify Actors to a Postgres SQL database (with PGVector extension).

Apify

4.6

Pinecone Integration

apify/pinecone-integration

This integration transfers data from Apify Actors to a Pinecone and is a good starting point for a question-answering, search, or RAG use case.

Apify

457

3.2

Magento E-Commerce Scraper 🚧

jupri/magento-scraper

Scrape data about product price, description and other information from Magento E-Commerce websites.

cat

365

Google Direction Scraper

dev_bodex/google-direction-scraper

This actor automates extracting route directions from Google Maps. Using Node.js and Puppeteer, it retrieves travel routes, distances, and estimated times for various transport modes, offering structured data for navigation, logistics, or travel planning applications.

Festus Befgrp

Google Maps Routes

mcdowell/Google-Maps-Route-Scraper

A flexible scraper that returns the quickest route between two locations. Can filter paths by avoiding `Ferries`, `Highways` and `Tolls`

Victor McDowell

BeautifulSoup Scraper

apify/beautifulsoup-scraper

Crawls websites using raw HTTP requests. It parses the HTML with the BeautifulSoup library and extracts data from the pages using Python code. Supports both recursive crawling and lists of URLs. This Actor is a Python alternative to Cheerio Scraper.

Apify

884

4.2

Example Selenium

apify/example-selenium

Example of loading a web page in headless Chrome using Selenium Webdriver.

Apify

288

5.0

Apify Actors

prog-party/apify-actors

This Apify Actors Actor retrieves data from Apify, allowing to filter, and returns a list of actors as a Dataset.

Prog Party

Facebook Ads Scraper

ibraheemalani2/my-fb-ads-scraper

Scrape the Facebook Ads Library for any keyword. This actor extracts ad details and then visits each advertiser's page to capture valuable company info: websites, social links, and categories. Perfect for building high-quality lead lists from active advertisers automatically.