# Similarity Graph From Embeddings (`mhamas/similarity-graph-from-embeddings`) Actor Builds a similarity graph from vector embeddings. Fetches vectors from URLs, computes pairwise cosine similarities using optimized linear algebra, and connects each point to its K nearest neighbors - revealing hidden clusters and relationships in your high-dimensional data. - **URL**: https://apify.com/mhamas/similarity-graph-from-embeddings.md - **Developed by:** [Matej Hamas](https://apify.com/mhamas) (community) - **Categories:** Automation, Developer tools - **Stats:** 1 total users, 0 monthly users, 0.0% runs succeeded, NaN bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Similarity Graph From Embeddings Apify Actor that builds a similarity graph from vector embeddings using cosine similarity. It fetches vectors from provided URLs, computes pairwise cosine similarities, filters edges by configurable outgoing and incoming limits per node, and outputs a graph JSON. ### How it works 1. **Fetch vectors** — Downloads JSON data from each provided URL. Each URL must return a JSON object mapping IDs to float arrays: `{ "id1": [0.1, 0.2, ...], "id2": [0.3, 0.4, ...] }`. 2. **Validate** — All vectors must have the same dimensionality. Duplicate IDs across URLs are not allowed. 3. **Compute similarities** — Builds a full cosine similarity matrix using vectorized numpy operations (L2-normalize + single matrix multiply via BLAS). 4. **Filter edges** — Applies global top-percentage threshold, then limits outgoing edges per node, then limits incoming edges per node. Each filter keeps only the highest-similarity edges using `argpartition` for O(n) performance. 5. **Build graph** — Each vector becomes a node. Surviving edges become directed edges with cosine similarity as the weight. 6. **Store output** — The graph is saved as `graph.json` in the default key-value store. A link to the file is pushed to the default dataset and displayed on the output tab. ### Input | Field | Type | Required | Default | Description | |---|---|---|---|---| | `urls` | `string[]` | Yes | — | List of URLs, each returning JSON of form `{ id: [float, float, ...] }`. All vectors must have the same dimensionality. | | `topPercentage` | `number` | No | `100` | Keep only the top X% of all pairwise similarities (globally). Lower values produce sparser graphs. Applied before the per-node edge limits. | | `maxOutgoingEdgesPerNode` | `integer` | No | — | For each node, keep only the top K most similar neighbors as outgoing edges. If not set, all edges surviving the top percentage filter are kept. Applied before the incoming edges limit. | | `maxIncomingEdgesPerNode` | `integer` | No | — | For each node, keep only the top K highest-similarity incoming edges. If not set, incoming edges are not limited. Applied after the outgoing edges limit. | | `keepAtLeastOneEdge` | `boolean` | No | `false` | When enabled, each node always keeps its most similar neighbor regardless of other filtering. Prevents isolated nodes in the graph. | #### Example input ```json { "urls": [ "https://example.com/embeddings-part1.json", "https://example.com/embeddings-part2.json" ], "maxOutgoingEdgesPerNode": 10, "maxIncomingEdgesPerNode": 20 } ```` #### Expected URL response format Each URL must return a JSON object where keys are string IDs and values are arrays of floats (all the same length): ```json { "apple": [0.12, 0.85, 0.33, 0.67], "banana": [0.11, 0.82, 0.30, 0.71], "car": [0.90, 0.05, 0.88, 0.12] } ``` ### Output #### Key-value store The Actor stores a single file `graph.json` in the default key-value store. Example: ```json { "version": "1", "nodes": [ { "id": "apple" }, { "id": "banana" }, { "id": "car" } ], "edges": [ { "source": "apple", "target": "banana", "weight": 0.987 }, { "source": "banana", "target": "apple", "weight": 0.987 } ] } ``` - **Nodes** — One per vector ID from the input data. - **Edges** — Directed. Outgoing edges per node are limited by `maxOutgoingEdgesPerNode`, incoming edges by `maxIncomingEdgesPerNode`. Edge weight is the cosine similarity (0 to 1). #### Graph JSON schema The output `graph.json` conforms to the following JSON schema: ```json { "$schema": "http://json-schema.org/draft-07/schema#", "title": "Similarity Graph", "type": "object", "required": ["version", "nodes", "edges"], "properties": { "version": { "type": "string", "const": "1" }, "nodes": { "type": "array", "items": { "type": "object", "required": ["id"], "properties": { "id": { "type": "string", "description": "Vector ID from the input data." } } } }, "edges": { "type": "array", "items": { "type": "object", "required": ["source", "target", "weight"], "properties": { "source": { "type": "string", "description": "ID of the source node." }, "target": { "type": "string", "description": "ID of the target node." }, "weight": { "type": "number", "description": "Cosine similarity between source and target vectors." } } } } } } ``` #### Dataset The default dataset contains a single record with the public URL of the graph JSON file: ```json { "graphUrl": "https://api.apify.com/v2/key-value-stores//records/graph.json" } ``` # Actor input Schema ## `urls` (type: `array`): List of URLs, each returning JSON of form { id: \[float, float, ...] }. All vectors must have the same dimensionality. ## `topPercentage` (type: `number`): Keep only the top X% of all pairwise similarities (globally). Lower values produce sparser graphs. Applied before the per-node neighbor limit. ## `maxOutgoingEdgesPerNode` (type: `integer`): For each node, keep only the top K most similar neighbors as outgoing edges. If not set, all edges surviving the top percentage filter are kept. Applied before the incoming edges limit. ## `maxIncomingEdgesPerNode` (type: `integer`): For each node, keep only the top K highest-similarity incoming edges. If not set, incoming edges are not limited. Applied after the outgoing edges limit. ## `keepAtLeastOneEdge` (type: `boolean`): When enabled, each node will always keep its most similar neighbor regardless of other filtering (top percentage, K neighbors). Prevents isolated nodes in the graph. ## Actor input object example ```json { "urls": [], "topPercentage": 100, "keepAtLeastOneEdge": false } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [] }; // Run the Actor and wait for it to finish const run = await client.actor("mhamas/similarity-graph-from-embeddings").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [] } # Run the Actor and wait for it to finish run = client.actor("mhamas/similarity-graph-from-embeddings").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [] }' | apify call mhamas/similarity-graph-from-embeddings --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=mhamas/similarity-graph-from-embeddings", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Similarity Graph From Embeddings", "description": "Builds a similarity graph from vector embeddings. Fetches vectors from URLs, computes pairwise cosine similarities using optimized linear algebra, and connects each point to its K nearest neighbors - revealing hidden clusters and relationships in your high-dimensional data.", "version": "0.0", "x-build-id": "uZltAxFBTkxbXfdth" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/mhamas~similarity-graph-from-embeddings/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-mhamas-similarity-graph-from-embeddings", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/mhamas~similarity-graph-from-embeddings/runs": { "post": { "operationId": "runs-sync-mhamas-similarity-graph-from-embeddings", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/mhamas~similarity-graph-from-embeddings/run-sync": { "post": { "operationId": "run-sync-mhamas-similarity-graph-from-embeddings", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "urls" ], "properties": { "urls": { "title": "Vector Data URLs", "type": "array", "description": "List of URLs, each returning JSON of form { id: [float, float, ...] }. All vectors must have the same dimensionality.", "items": { "type": "string" } }, "topPercentage": { "title": "Top Similarity Percentage", "minimum": 0.1, "maximum": 100, "type": "number", "description": "Keep only the top X% of all pairwise similarities (globally). Lower values produce sparser graphs. Applied before the per-node neighbor limit.", "default": 100 }, "maxOutgoingEdgesPerNode": { "title": "Max Outgoing Edges Per Node", "minimum": 1, "type": "integer", "description": "For each node, keep only the top K most similar neighbors as outgoing edges. If not set, all edges surviving the top percentage filter are kept. Applied before the incoming edges limit." }, "maxIncomingEdgesPerNode": { "title": "Max Incoming Edges Per Node", "minimum": 1, "type": "integer", "description": "For each node, keep only the top K highest-similarity incoming edges. If not set, incoming edges are not limited. Applied after the outgoing edges limit." }, "keepAtLeastOneEdge": { "title": "Always Keep At Least One Edge Per Node", "type": "boolean", "description": "When enabled, each node will always keep its most similar neighbor regardless of other filtering (top percentage, K neighbors). Prevents isolated nodes in the graph.", "default": false } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```