Pinecone Integration
No credit card required
Pinecone Integration
No credit card required
This integration transfers data from Apify Actors to a Pinecone and is a good starting point for a question-answering, search, or RAG use case.
It has staring failing, saying chunk size is too big but it is not
It is just one actor but it keeps happening. I think the api is trying to chunk all 500 chunks in one go.
Hi, thank you for using the Pinecone integration.
The batching logic is handled by the official Pinecone implementation, which computes text embeddings in batches of 1,000.
I'm not sure why your chunk_size is set to 40,000 characters. In most cases, embeddings won't effectively capture the meaning of such long text. If you reduce it to 20,000 (which is still quite long), it should work.
I can also adjust the batch size (if you want), but I suspect the chunk_size might have been an unintentional copy-paste mistake.
Hope this helps!
Jiri
ApifyActor Metrics
45 monthly users
-
25 bookmarks
95% runs succeeded
16 days response time
Created in Jun 2024
Modified a month ago
