Pricing
Pay per usage
Transfer data from Apify Actors to Amazon OpenSearch Service. This Actor is a good starting point for building question-answering systems, search functionality, or Retrieval-Augmented Generation (RAG) use cases.
Pricing
Pay per usage
Rating
4.4
(5)
Developer
Apify
Actor stats
4
Bookmarked
3
Total users
1
Monthly active users
7 months ago
Last modified
Categories
Share
openSearchUrlstringRequired
The URL of the Amazon OpenSearch Service instance to connect to
awsAccessKeyIdstringRequired
The AWS access key ID for the Amazon OpenSearch Service
awsSecretAccessKeystringRequired
The AWS secret access key for the Amazon OpenSearch Service
openSearchIndexNamestringRequired
The name of the index in the Amazon OpenSearch Service where the data will be stored
autoCreateIndexbooleanOptional
When set to true, the integration will automatically create the index if it does not exist in the Amazon OpenSearch Service instance
Default value of this property is true
awsRegionstringOptional
The AWS region where the Amazon OpenSearch Service instance is located
Default value of this property is "us-east-1"
awsServiceNameEnumOptional
The AWS service name for the Amazon OpenSearch Service
"aoss": string"es": stringDefault value of this property is "aoss"
useSslbooleanOptional
When set to true, the integration will use SSL to connect to the Amazon OpenSearch Service instance
Default value of this property is true
verifyCertsbooleanOptional
When set to true, the integration will verify SSL certificates when connecting to the Amazon OpenSearch Service instance
Default value of this property is true
useAWS4AuthbooleanOptional
When enabled, the integration will use AWS4 authentication to connect to the Amazon OpenSearch Service instance.
Note: If you are connecting to an OpenSearch Service instance that is not hosted on AWS, set this to false. In this case, AWS credentials are not required and will be ignored. You can provide dummy values for awsAccessKeyId and awsSecretAccessKey.
Default value of this property is true
embeddingsProviderEnumRequired
Choose the embeddings provider to use for generating embeddings
"OpenAI": string"Cohere": stringDefault value of this property is "OpenAI"
embeddingsConfigobjectOptional
Configure the parameters for the LangChain embedding class. Key points to consider:
Typically, you only need to specify the model name. For example, for OpenAI, set the model name as {"model": "text-embedding-3-small"}.
It's required to ensure that the vector size of your embeddings matches the size of embeddings in the database.
Here are examples of embedding models:
For more details about other parameters, refer to the LangChain documentation.
embeddingsApiKeystringRequired
Value of the API KEY for the embeddings provider (if required).
For example for OpenAI it is OPENAI_API_KEY, for Cohere it is COHERE_API_KEY)
datasetFieldsarrayRequired
This array specifies the dataset fields to be selected and stored in the vector store. Only the fields listed here will be included in the vector store.
For instance, when using the Website Content Crawler, you might choose to include fields such as text, url, and metadata.title in the vector store.
Default value of this property is ["text"]
metadataDatasetFieldsobjectOptional
A list of dataset fields which should be selected from the dataset and stored as metadata in the vector stores.
For example, when using the Website Content Crawler, you might want to store url in metadata. In this case, use metadataDatasetFields parameter as follows {"url": "url"}
metadataObjectobjectOptional
This object allows you to store custom metadata for every item in the vector store.
For example, if you want to store the domain as metadata, use the metadataObject like this: {"domain": "apify.com"}.
dataUpdatesStrategyEnumOptional
Choose the update strategy for the integration. The update strategy determines how the integration updates the data in the database.
The available options are:
Add data (add):
Upsert data (upsert):
Update changed data based on deltas (deltaUpdates):
Select the strategy that best fits your use case.
"add": string"upsert": string"deltaUpdates": stringDefault value of this property is "deltaUpdates"
dataUpdatesPrimaryDatasetFieldsarrayOptional
This array contains fields that are used to uniquely identify dataset items, which helps to handle content changes across different runs.
For instance, in a web content crawling scenario, the url field could serve as a unique identifier for each item.
Default value of this property is ["url"]
enableDeltaUpdatesbooleanOptional
When set to true, this setting enables incremental updates for objects in the database by comparing the changes (deltas) between the crawled dataset items and the existing objects, uniquely identified by the datasetKeysToItemId field.
The integration will only add new objects and update those that have changed, reducing unnecessary updates. The datasetFields, metadataDatasetFields, and metadataObject fields are used to determine the changes.
Default value of this property is true
deltaUpdatesPrimaryDatasetFieldsarrayOptional
This array contains fields that are used to uniquely identify dataset items, which helps to handle content changes across different runs.
For instance, in a web content crawling scenario, the url field could serve as a unique identifier for each item.
Default value of this property is ["url"]
deleteExpiredObjectsbooleanOptional
When set to true, delete objects from the database that have not been crawled for a specified period.
Default value of this property is true
expiredObjectDeletionPeriodDaysintegerOptional
This setting allows the integration to manage the deletion of objects from the database that have not been crawled for a specified period. It is typically used in subsequent runs after the initial crawl.
When the value is greater than 0, the integration checks if objects have been seen within the last X days (determined by the expiration period). If the objects are expired, they are deleted from the database. The specific value for deletedExpiredObjectsDays depends on your use case and how frequently you crawl data.
For example, if you crawl data daily, you can set deletedExpiredObjectsDays to 7 days. If you crawl data weekly, you can set deletedExpiredObjectsDays to 30 days.
Default value of this property is 30
performChunkingbooleanOptional
When set to true, the text will be divided into smaller chunks based on the settings provided below. Proper chunking helps optimize retrieval and ensures accurate and efficient responses.
Default value of this property is true