# Needle (`needle/needle`) Actor

Production-ready RAG service to search and retrieve information from your documents.

- **URL**: https://apify.com/needle/needle.md
- **Developed by:** [Needle Team](https://apify.com/needle) (community)
- **Categories:** Integrations, Agents, AI
- **Stats:** 3 total users, 3 monthly users, 0.0% runs succeeded, NaN bookmarks
- **User rating**: No ratings yet

## Pricing

$0.50 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):

```bash
npm install apify-client
```

In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):

```bash
pip install apify-client
```

In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):

````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash

In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).

If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).


# README

## Needle AI Platform Integration

A powerful Apify Actor that integrates with the [Needle AI platform](https://needle-ai.com) to manage documents and perform semantic searches using AI capabilities. This Actor allows you to interact with your Needle collections programmatically through the Apify platform.

### Overview

This Actor provides seamless integration with the Needle AI platform, enabling you to:

- **List Collections**: Retrieve all collections from your Needle account
- **Search Documents**: Perform semantic searches within specific collections
- **Create Collections**: Set up new document collections
- **Add Documents**: Add documents to existing collections via URLs

### Features

- **🔍 Semantic Search**: Leverage AI to find relevant documents using natural language queries
- **📚 Document Management**: Organize and manage your document collections
- **🤖 AI-Powered**: Uses advanced language models for intelligent document retrieval
- **🔗 Easy Integration**: Works seamlessly with the Apify platform and Needle ecosystem
- **⚡ Scalable**: Built on Apify's cloud infrastructure for reliable performance

### How it works

1. **Authentication**: Get your API key from [Needle Settings](https://needle-ai.com/dashboard/settings)
2. **Action Selection**: Choose the operation you want to perform (list, search, create, add)
3. **Parameter Configuration**: Set the required parameters for your chosen action
4. **Execution**: The Actor communicates with the Needle AI platform and returns results
5. **Results**: Get structured data back with your search results or operation status

### Quick Start

1. **Get your Needle API key**: Visit [Needle Settings](https://needle-ai.com/dashboard/settings) to generate your API key
2. **Run the Actor**: Choose from the available actions (list collections, search, create, add documents)
3. **View results**: Get structured data back with your collections, search results, or operation status

### Supported Operations

#### List Collections
Retrieve all collections in your Needle account:
```json
{
  "action": "list_collections",
  "needle_api_key": "your-api-key"
}
````

#### Search Collection

Perform semantic search within a specific collection:

```json
{
  "action": "search_collection",
  "needle_api_key": "your-api-key",
  "collection_id": "collection-id",
  "query": "information about AI and machine learning"
}
```

#### Create Collection

Create a new collection for organizing documents:

```json
{
  "action": "create_collection",
  "needle_api_key": "your-api-key",
  "collection_name": "My Research Collection"
}
```

#### Add Documents

Add documents to an existing collection:

```json
{
  "action": "add_documents",
  "needle_api_key": "your-api-key",
  "collection_id": "collection-id",
  "document_urls": "[\"https://example.com/doc1.pdf\", \"https://example.com/doc2.pdf\"]"
}
```

### Use Cases

- **📚 Knowledge Management**: Build searchable knowledge bases from web content
- **🔍 Research Automation**: Automatically collect and search through research materials
- **📊 Competitive Intelligence**: Monitor competitor content and extract insights
- **📝 Content Analysis**: Analyze large volumes of documents for key information
- **🤖 RAG Pipelines**: Build Retrieval-Augmented Generation workflows

### Output

The Actor returns structured data in JSON format containing:

- Collection information and metadata
- Search results with relevance scores
- Operation status and confirmation messages
- Error details if operations fail

### Configuration

#### Required Fields

- **needle\_api\_key**: Your Needle API key (can also be set via `NEEDLE_API_KEY` environment variable)
- **action**: The operation to perform

#### Optional Fields (depending on action)

- **collection\_id**: Required for search and add operations
- **query**: Search query text (for search operations)
- **collection\_name**: Name for new collections
- **document\_urls**: JSON array of document URLs to add

### Support

- **Documentation**: [Needle API Documentation](https://docs.needle-ai.com)
- **Community**: Join our [Discord community](https://discord.gg/XSHaP5pPHT) for support and discussions
- **Issues**: Report issues on [GitHub](https://github.com/needle-ai)

### About Needle

Needle AI is a platform that simplifies building RAG (Retrieval-Augmented Generation) pipelines. It provides semantic search capabilities and document management tools to help you build intelligent applications with AI.

Visit [needle-ai.com](https://needle-ai.com) to learn more.

# Actor input Schema

## `needle_api_key` (type: `string`):

Your Needle API key from https://needle-ai.com/dashboard/settings. Can also be set via NEEDLE\_API\_KEY environment variable.

## `action` (type: `string`):

The action you want to perform with the Needle AI platform

## `collection_name` (type: `string`):

Name for the new collection (required for create\_collection action)

## `collection_id` (type: `string`):

ID of the collection to interact with (required for get\_collection\_details, search\_collection, add\_documents actions)

## `search_query` (type: `string`):

The search query to execute (required for search\_collection action)

## `search_limit` (type: `integer`):

Maximum number of search results to return (default: 10)

## `document_urls` (type: `string`):

JSON array of URLs to add as documents to the collection, e.g. \["https://example.com/doc1.pdf", "https://example.com/doc2.pdf"] (required for add\_documents action)

## Actor input object example

```json
{
  "action": "list_collections",
  "search_limit": 10
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("needle/needle").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("needle/needle").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call needle/needle --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=needle/needle",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

```json
{
    "openapi": "3.0.1",
    "info": {
        "title": "Needle",
        "description": "Production-ready RAG service to search and retrieve information from your documents.",
        "version": "0.1",
        "x-build-id": "1TXtOlnnBemUJBdVS"
    },
    "servers": [
        {
            "url": "https://api.apify.com/v2"
        }
    ],
    "paths": {
        "/acts/needle~needle/run-sync-get-dataset-items": {
            "post": {
                "operationId": "run-sync-get-dataset-items-needle-needle",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        },
        "/acts/needle~needle/runs": {
            "post": {
                "operationId": "runs-sync-needle-needle",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor and returns information about the initiated run in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK",
                        "content": {
                            "application/json": {
                                "schema": {
                                    "$ref": "#/components/schemas/runsResponseSchema"
                                }
                            }
                        }
                    }
                }
            }
        },
        "/acts/needle~needle/run-sync": {
            "post": {
                "operationId": "run-sync-needle-needle",
                "x-openai-isConsequential": false,
                "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
                "tags": [
                    "Run Actor"
                ],
                "requestBody": {
                    "required": true,
                    "content": {
                        "application/json": {
                            "schema": {
                                "$ref": "#/components/schemas/inputSchema"
                            }
                        }
                    }
                },
                "parameters": [
                    {
                        "name": "token",
                        "in": "query",
                        "required": true,
                        "schema": {
                            "type": "string"
                        },
                        "description": "Enter your Apify token here"
                    }
                ],
                "responses": {
                    "200": {
                        "description": "OK"
                    }
                }
            }
        }
    },
    "components": {
        "schemas": {
            "inputSchema": {
                "type": "object",
                "required": [
                    "action"
                ],
                "properties": {
                    "needle_api_key": {
                        "title": "Needle API Key",
                        "type": "string",
                        "description": "Your Needle API key from https://needle-ai.com/dashboard/settings. Can also be set via NEEDLE_API_KEY environment variable."
                    },
                    "action": {
                        "title": "Action to perform",
                        "enum": [
                            "list_collections",
                            "create_collection",
                            "get_collection_details",
                            "search_collection",
                            "add_documents"
                        ],
                        "type": "string",
                        "description": "The action you want to perform with the Needle AI platform",
                        "default": "list_collections"
                    },
                    "collection_name": {
                        "title": "Collection Name",
                        "type": "string",
                        "description": "Name for the new collection (required for create_collection action)"
                    },
                    "collection_id": {
                        "title": "Collection ID",
                        "type": "string",
                        "description": "ID of the collection to interact with (required for get_collection_details, search_collection, add_documents actions)"
                    },
                    "search_query": {
                        "title": "Search Query",
                        "type": "string",
                        "description": "The search query to execute (required for search_collection action)"
                    },
                    "search_limit": {
                        "title": "Search Limit",
                        "minimum": 1,
                        "maximum": 100,
                        "type": "integer",
                        "description": "Maximum number of search results to return (default: 10)",
                        "default": 10
                    },
                    "document_urls": {
                        "title": "Document URLs (JSON array)",
                        "type": "string",
                        "description": "JSON array of URLs to add as documents to the collection, e.g. [\"https://example.com/doc1.pdf\", \"https://example.com/doc2.pdf\"] (required for add_documents action)"
                    }
                }
            },
            "runsResponseSchema": {
                "type": "object",
                "properties": {
                    "data": {
                        "type": "object",
                        "properties": {
                            "id": {
                                "type": "string"
                            },
                            "actId": {
                                "type": "string"
                            },
                            "userId": {
                                "type": "string"
                            },
                            "startedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "finishedAt": {
                                "type": "string",
                                "format": "date-time",
                                "example": "2025-01-08T00:00:00.000Z"
                            },
                            "status": {
                                "type": "string",
                                "example": "READY"
                            },
                            "meta": {
                                "type": "object",
                                "properties": {
                                    "origin": {
                                        "type": "string",
                                        "example": "API"
                                    },
                                    "userAgent": {
                                        "type": "string"
                                    }
                                }
                            },
                            "stats": {
                                "type": "object",
                                "properties": {
                                    "inputBodyLen": {
                                        "type": "integer",
                                        "example": 2000
                                    },
                                    "rebootCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "restartCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "resurrectCount": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "computeUnits": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "options": {
                                "type": "object",
                                "properties": {
                                    "build": {
                                        "type": "string",
                                        "example": "latest"
                                    },
                                    "timeoutSecs": {
                                        "type": "integer",
                                        "example": 300
                                    },
                                    "memoryMbytes": {
                                        "type": "integer",
                                        "example": 1024
                                    },
                                    "diskMbytes": {
                                        "type": "integer",
                                        "example": 2048
                                    }
                                }
                            },
                            "buildId": {
                                "type": "string"
                            },
                            "defaultKeyValueStoreId": {
                                "type": "string"
                            },
                            "defaultDatasetId": {
                                "type": "string"
                            },
                            "defaultRequestQueueId": {
                                "type": "string"
                            },
                            "buildNumber": {
                                "type": "string",
                                "example": "1.0.0"
                            },
                            "containerUrl": {
                                "type": "string"
                            },
                            "usage": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "integer",
                                        "example": 1
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            },
                            "usageTotalUsd": {
                                "type": "number",
                                "example": 0.00005
                            },
                            "usageUsd": {
                                "type": "object",
                                "properties": {
                                    "ACTOR_COMPUTE_UNITS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATASET_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "KEY_VALUE_STORE_WRITES": {
                                        "type": "number",
                                        "example": 0.00005
                                    },
                                    "KEY_VALUE_STORE_LISTS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_READS": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "REQUEST_QUEUE_WRITES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_INTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "DATA_TRANSFER_EXTERNAL_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
                                        "type": "integer",
                                        "example": 0
                                    },
                                    "PROXY_SERPS": {
                                        "type": "integer",
                                        "example": 0
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}
```