Pinecone Integration
No credit card required
Pinecone Integration
No credit card required
This integration transfers data from Apify Actors to a Pinecone and is a good starting point for a question-answering, search, or RAG use case.
Do you want to learn more about this Actor?
Get a demoPinecone upserts my json format as a string.
Open
So, in my scraper i scrape a list in json format. And it looks correct and everything. But when i upsert it to pinecone i can see it adds a "" on both sides of my json list so it becomes a string. This is a problem because you can not use metadata filter in one long string. Is there any way to make your metadata not make json to a string?I get the output of my scraper as an array. Here is an example: "metadataavailable": [ "2446", "2447", "2448", "2449", "2450", "2451", "2452", "2501", "2502", "2508", "2509", "2510", "2511", "2512", "2513", "2514", "2515", "2516", "2517", "2518", "2520", "2521", "2522", "2523", "2524", "2525", "2526", "2527", "2529", "2530", "2531", "2532", "2533", "2534", "2535", "2536", "2537", "2538", "2539", "2540", "2542", "2543", "2544", "2545", "2551", "2552", "2601" ] },
But when I push it as metadata to pinecone or a variable to pinecone it gets changed to a string. Example: "metadataavailable": "['2446', '2447', '2448', '2449', '2450', '2451', '2452', '2501', '2523', '2524', '2525', '2526', '2527', '2528', '2529', '2530', '2531', '2532', '2533', '2534', '2535', '2536', '2537', '2538', '2539', '2540', '2541', '2542', '2543', '2544', '2545', '2546', '2547', '2548', '2549', '2550', '2551', '2552', '2601']",
Best regards Nicholas
Hi Nicholas, thank you for using the Pinecone integration.
I’m not sure I fully understand the issue. Could you please provide the run ID so I can check it?
I attempted to reproduce the issue by storing an array in the metadata object, then checked the resulting object in the Pinecone database:
1"metadataObject": { 2 "metadataavailable": [ 3 "2446", 4 "2447" 5 ] 6 }
The array was stored correctly, as shown in the attached screenshot.
responsible_box
The run of the scraper: Gf7qzUpVU5dMavdQN
Hmm, maybe i am making a mistake, this is more info:
The run of the pinecone: vsPjM8Za6qimldPlw
You can see in picture one the scraper output. Picture two the output of pinecone.
Oh, I see it now. Thanks!
The fix is straightforward, but I need to test it further to ensure it doesn’t break anything. I don’t think it will, but I want to be sure. It should be ready sometime tomorrow.
responsible_box
Sounds very good :) I just tried the build 49, did not seem to fix the issue.
I’m really sorry—I didn’t have time to look at it today. Hopefully, I can get to it tomorrow. I’ll let you know.
Hi, I’ve created a new build: 0.0.52. I’ve tested it locally, and it works. However, I don’t have a dataset on the platform at hand to test it.
Could you please give it a try? 🙏 (since the data has changed, it will upsert them)
By the way, there’s still an issue with "Too many requests" to the Pinecone database. I’ve raised a question on the community forum, but no response so far. I’ve also just submitted a support ticket.
ApifyActor Metrics
42 monthly users
-
19 stars
87% runs succeeded
5.8 days response time
Created in Jun 2024
Modified 3 days ago
