# 🕸️ Knowledge Graph Extractor - Entities & Typed Relationships (`that_red_bird/knowledge-graph-extractor`) Actor

🕸️ Crawl a site or corpus and get back a real graph: typed ENTITIES and typed RELATIONSHIPS, no LLM. ✅ JSON-LD first, then headings/tables/breadcrumbs/pricing tables. ✅ Cross-page entity resolution merges aliases into one node. ✅ Edges built from structure, every one with an evidenceUrl.

- **URL**: https://apify.com/that\_red\_bird/knowledge-graph-extractor.md
- **Developed by:** [mohamed alaya](https://apify.com/that_red_bird) (community)
- **Categories:** AI
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Knowledge Graph Extractor

Crawl a site (or hand over a corpus directly) and get back a real graph: typed
**entities** and typed **relationships**, deterministically, with no LLM and no
NER model anywhere in the pipeline.

### What it actually does

**1. Entity extraction without an NER model.** JSON-LD/schema.org blocks are
read first (highest precision, reusing the same `jsonLdBlocks` parser
structured-data-generator and llms-txt-generator rely on). Wherever a page has
no JSON-LD, structural signals fill the gap instead of free-text NLP: heading
hierarchy, breadcrumbs, tables (including pricing tables), definition lists,
and — as a last resort — capitalised multiword phrases in body text, filtered
by a stopword guard so sentence starts don't get scooped up as entities.

**2. Eight entity types.** Organization, Product, Feature, Person, Location,
PricingPlan, Technology, Document. schema.org `@type`s map onto these
directly; structural signals get a lightweight heuristic classifier (legal
suffixes -> Organization, API/SDK/Platform-style words -> Technology, country
names and street/city words -> Location, two-to-three-word Title Case names ->
Person, plan-ish words like Free/Pro/Enterprise -> PricingPlan).

**3. Relationships from structure, never from parsing free text.** Breadcrumb
trail -> `parentOf` chain. A table's own layout -> row/column edges, or, when
the table looks like a pricing grid, `PricingPlan -[includesFeature]->
Feature`. `H2` under `H1` -> `partOf`. A JSON-LD `Product`'s
`additionalProperty` list -> `Product -[hasFeature]-> Feature`. Every entity
pair that shares a page also gets a weak `coOccursWith` edge (capped per page,
can be turned off). **Every single edge carries the `evidenceUrl`** of the
page it was observed on plus a `confidence` score.

**4. Entity resolution across pages.** "Acme Corp" on one page and "ACME,
Inc." on another collapse into one node — reusing entity-resolver's own
`normalizeCompany` (legal-suffix stripping) and `normalizePerson` (nickname
expansion) for the exact-match pass, then a Jaro-Winkler fuzzy pass
(`resolutionThreshold`, default 0.90) inside the same type for spelling drift
that survives normalisation. The merged node keeps every distinct spelling it
was seen with as an `aliases` list, plus every page it was mentioned on as
`sourceUrls`.

### Input

Either crawl a live site:

```json
{ "siteUrl": "https://example.com", "maxPages": 50 }
```

or hand over the corpus directly (also how the offline test suite runs it):

```json
{ "pages": [{ "url": "https://example.com/pricing", "html": "<html>...</html>" }] }
```

Both can be combined — inline `pages` are extracted alongside anything
crawled from `siteUrl`.

### Output

`node` rows: `id, label, nodeType, aliases, sourceUrls, mentionCount`.
`edge` rows (the flat edge-list, ready to import into any graph tool):
`from, edgeType, to, evidenceUrl, confidence, value`. One `summary` row with
counts by type, counts by edge type, and the most-connected nodes.
`outputMode` can restrict the run to `nodesOnly` or `edgesOnly`.

### Honest limitations

- This is **structural extraction, not NLP or NER**. There is no language
  model reading sentences for meaning — every entity comes from a schema.org
  tag, a heading, a table cell, a breadcrumb, or a capitalised phrase pattern.
  A page that is one giant unstructured wall of prose with no markup, no
  headings and no JSON-LD will yield a thin or empty graph. Recall tracks how
  well-structured the source site is, not how "important" the entities are.
- The capitalised-phrase heuristic is the noisiest source by design and is
  gated by `minPhraseMentions` (default: a phrase must appear at least twice
  across the whole corpus to survive) — raise it further on prose-heavy sites,
  or disable the heuristic entirely with `extractCapitalizedPhrases: false`.
- Person detection from free text is a coarse two/three-Title-Case-word
  pattern, not a real name model — it will occasionally misfire on brand names
  that happen to look like "Firstname Lastname".
- Entity resolution merges within a type only. A `Product` named "Acme" and an
  `Organization` named "Acme" are never merged into one node, even though a
  reader would recognise them as related — cross-type identity is out of
  scope here.
- Nav/header/footer/sidebar/cookie-banner/social-share regions are stripped
  before any extraction runs, so site chrome should never surface as an
  entity — but a site that puts real content inside a `<nav>` or `<footer>`
  tag will lose that content along with the boilerplate.

# Actor input Schema

## `siteUrl` (type: `string`):

The site to crawl, e.g. https://docs.example.com. Pages are discovered from its sitemap, or by following same-origin links if there is no sitemap. Leave empty if you pass "pages" directly.

## `pages` (type: `array`):

Optional: supply the corpus directly as an array of {"url":"...","html":"..."} objects instead of crawling. Combine freely with siteUrl - both sets of pages are extracted together.

## `maxPages` (type: `integer`):

Upper bound on pages to crawl from siteUrl.

## `useSitemap` (type: `boolean`):

Prefer the site's sitemap (including sitemap indexes) for discovery. Falls back to following same-origin links automatically if none is found.

## `includePatterns` (type: `array`):

Keep only URLs containing one of these substrings, e.g. /docs/, /pricing.

## `excludePatterns` (type: `array`):

Drop URLs containing any of these, e.g. /tag/, /author/, ?page=.

## `concurrency` (type: `integer`):

How many pages to fetch in parallel while crawling.

## `extractCapitalizedPhrases` (type: `boolean`):

Scan body paragraphs and list items for capitalised multiword phrases (e.g. proper nouns, brand names) as a last-resort entity source, guarded by a stopword filter. Turn off on prose-heavy sites to cut noise.

## `minPhraseMentions` (type: `integer`):

A capitalised-phrase-derived entity must be mentioned at least this many times across the whole corpus to survive into the graph. Does not apply to JSON-LD, breadcrumb, table or heading entities - only to the free-text phrase heuristic, which is the noisiest source.

## `resolutionThreshold` (type: `integer`):

After exact normalisation (legal-suffix stripping for companies, nickname expansion for people), remaining same-type entities whose names score at or above this Jaro-Winkler similarity are still merged into one node. Expressed 0-100; 90 means 0.90. Lower catches more spelling drift at the risk of false merges.

## `includeCoOccurrenceEdges` (type: `boolean`):

Add a coOccursWith edge between every pair of distinct entities found on the same page. The weakest relationship type here (no structural claim beyond "appeared together") - turn off for a cleaner, purely-structural graph.

## `maxCoOccurrenceEdgesPerPage` (type: `integer`):

Caps how many coOccursWith edges one page can contribute, so a page with dozens of entities cannot blow up the edge count combinatorially.

## `outputMode` (type: `string`):

full = nodes + edges + a summary row · nodesOnly = just the resolved entity nodes · edgesOnly = just the typed relationship edges (the flat edge-list for graph import).

## `proxyConfiguration` (type: `object`):

Optional Apify proxy configuration used while crawling siteUrl.

## Actor input object example

```json
{
  "siteUrl": "https://docs.apify.com",
  "maxPages": 50,
  "useSitemap": true,
  "concurrency": 5,
  "extractCapitalizedPhrases": true,
  "minPhraseMentions": 2,
  "resolutionThreshold": 90,
  "includeCoOccurrenceEdges": true,
  "maxCoOccurrenceEdgesPerPage": 30,
  "outputMode": "full"
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `downloadCsv` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `count` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "siteUrl": "https://docs.apify.com"
};

// Run the Actor and wait for it to finish
const run = await client.actor("that_red_bird/knowledge-graph-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "siteUrl": "https://docs.apify.com" }

# Run the Actor and wait for it to finish
run = client.actor("that_red_bird/knowledge-graph-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "siteUrl": "https://docs.apify.com"
}' |
apify call that_red_bird/knowledge-graph-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,that_red_bird/knowledge-graph-extractor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/j2EbEDbjunU8u3nxS/builds/30oYZOywi2Std9npu/openapi.json