Wikidata Scraper: Structured Entity Facts & Claims

Wikidata Entities Scraper pulls entities out of Wikidata, labels, descriptions, aliases and every property claim with its qualifiers, and flattens the deeply nested claim structures into rows you can actually use.

Wikidata is the machine-readable knowledge graph behind Wikipedia: 100M+ entities, CC0-licensed, with a public API built for programmatic access. This Actor reads that API directly: no proxy setup, no browser, no credentials to manage. Entities are fetched 50 at a time, and identifiers inside claims are resolved to human-readable labels for you.

Agent skill: SKILL.md

https://api.apify.com/v2/key-value-stores/t7YoTxpZEJOWvw4Ug/records/wikidata-entities-scraper.md

What you get

Output field	Meaning
entityId	Wikidata identifier, e.g. Q42
label, description	Preferred label and short description in your requested language
aliases	Alternative names in the same language
instanceOf	The P31 values as { id, label }, the entity's type, pulled out for convenience
claims	Every property, keyed by P-id, with its label, datatype and flattened values
qualifiers	Statement-level detail for the claims that carry qualifiers (start/end dates, roles, sources)
sitelinks	Map of every linked Wikimedia project → URL
wikipediaUrl	Direct Wikipedia link in your first requested language
modified	Last edit timestamp of the entity
scrapedAt	Run timestamp

A RUN_SUMMARY record in the key-value store holds per-run counts, the filters used, and any entity that failed.

Common use cases

• Entity resolution. Turn free-text names into stable Q-ids and canonical identifiers.
• Knowledge graphs. Extract typed facts (birth dates, coordinates, parent companies) as clean rows.
• Record enrichment. Attach VIAF, ISNI, GRID, IMDb, MusicBrainz and 6,000 other external IDs to your own data.
• AI agents and RAG. A fast, licence-clean fact layer that does not hallucinate.
• Media and entertainment analysis. Cast, crew, awards, franchises and release data, already structured.

Quick start

Three entities, everything they have:

{
 "entityIds": ["Q42", "Q64", "Q95"]
}

Resolve names, then keep only the facts you care about:

{
 "searchTerms": ["Douglas Adams", "Ada Lovelace"],
 "properties": ["P31", "P569", "P570", "P106"],
 "languages": ["en"],
 "includeSitelinks": false
}

German labels, raw identifiers, minimal payload:

{
 "entityIds": ["Q64"],
 "languages": ["de", "en"],
 "resolveLabels": false,
 "includeSitelinks": false
}

Input

Field	Type	Default	Notes
entityIds	array	[]	Q-identifiers (Q42), full wikidata.org/wiki/… URLs or entity URIs. Anything that is not an identifier is treated as a search term.
searchTerms	array	[]	Names resolved to entities via wbsearchentities before fetching.
properties	array	[]	Keep only these P-properties. Empty = every claim on the entity.
languages	array	["en"]	Fallback chain for labels, descriptions and aliases, first language with a value wins.
resolveLabels	boolean	true	Resolve referenced P-ids and Q-ids to labels (P31 → instance of, Q5 → human).
includeSitelinks	boolean	true	Include the site → URL map. A popular entity carries 300+ sitelinks.
resultsPerSearchTerm	integer	1	Entities kept per search term. Raise it when names are ambiguous.

Provide entityIds, searchTerms, or both, at least one entity must be resolvable or the run errors out immediately.

Output example

{
 "entityId": "Q42",
 "label": "Douglas Adams",
 "description": "British science fiction writer and humorist (1952-2001)",
 "aliases": ["Douglas Noël Adams", "Douglas N. Adams"],
 "instanceOf": [{ "id": "Q5", "label": "human" }],
 "claims": {
 "P31": { "label": "instance of", "datatype": "wikibase-item", "values": [{ "id": "Q5", "label": "human" }] },
 "P569": { "label": "date of birth", "datatype": "time", "values": [{ "time": "1952-03-11", "precision": "day", "calendar": "Gregorian", "raw": "+1952-03-11T00:00:00Z" }] },
 "P2048": { "label": "height", "datatype": "quantity", "values": [{ "amount": 1.96, "unit": { "id": "Q11573", "label": "metre" }, "lowerBound": null, "upperBound": null }] },
 "P1813": { "label": "short name", "datatype": "monolingualtext", "values": [{ "text": "Douglas Adams", "language": "en" }] },
 "P856": { "label": "official website", "datatype": "url", "values": ["https://douglasadams.com"] },
 "P18": { "label": "image", "datatype": "commonsMedia", "values": [{ "file": "Douglas adams portrait.jpg", "url": "https://commons.wikimedia.org/wiki/File:Douglas_adams_portrait.jpg" }] },
 "P214": { "label": "VIAF cluster ID", "datatype": "external-id", "values": ["113230702"] }
 },
 "qualifiers": {
 "P26": [{
 "value": { "id": "Q14623681", "label": "Jane Belson" },
 "rank": "normal",
 "qualifiers": {
 "P580": { "label": "start time", "values": [{ "time": "1991-11-25", "precision": "day", "calendar": "Gregorian" }] },
 "P582": { "label": "end time", "values": [{ "time": "2001-05-11", "precision": "day", "calendar": "Gregorian" }] }
 }
 }]
 },
 "sitelinks": { "enwiki": "https://en.wikipedia.org/wiki/Douglas_Adams", "dewiki": "https://de.wikipedia.org/wiki/Douglas_Adams" },
 "wikipediaUrl": "https://en.wikipedia.org/wiki/Douglas_Adams",
 "modified": "2026-08-03T13:13:51Z",
 "scrapedAt": "2026-08-06T11:40:00.000Z"
}

How claims are flattened

Raw Wikidata claims are five levels deep and every datatype nests differently. Each one is unwrapped explicitly:

Datatype	Flattened to
wikibase-item, wikibase-property	{ id, label }
time	{ time, precision, calendar, raw }, trimmed to the stated precision, so a year-precision date stays "1952" rather than pretending to be 1 January
quantity	{ amount, unit: { id, label }, lowerBound, upperBound }, amount is a real number, not "+1.96"
globe-coordinate	{ latitude, longitude, altitude, precision, globe }
monolingualtext	{ text, language }
string, url, external-id, math, geo-shape	plain string
commonsMedia	{ file, url }, the resolvable Commons URL, not just a file name
no value / unknown value	{ "special": "novalue" } / { "special": "somevalue" }

claims is the flat digest of every statement. qualifiers carries the statements that have qualifiers, with rank and the qualifier snaks flattened the same way, so nothing is lost, and the common case stays readable.

API example

curl -X POST "https://api.apify.com/v2/acts/arman-bd~wikidata-entities-scraper/run-sync-get-dataset-items?token=YOUR_TOKEN" \
 -H "Content-Type: application/json" \
 -d '{
 "entityIds": ["Q42"],
 "properties": ["P31", "P569", "P106"]
 }'

JavaScript example

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const run = await client.actor('arman-bd/wikidata-entities-scraper').call({
 searchTerms: ['Douglas Adams', 'Ada Lovelace'],
 properties: ['P31', 'P569', 'P570'],
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const e of items) {
 console.log(`${e.entityId} ${e.label}, born ${e.claims.P569?.values[0]?.time ?? 'unknown'}`);
}

Limits and behaviour

• Bad ids don't kill the run. wbgetentities rejects an entire 50-id batch when one id does not exist. The Actor reads the offending id out of the error, drops it into RUN_SUMMARY.failures and retries the remaining 49. You never lose good entities to one typo.
• Batched by design. Entities are fetched 50 per request, and label lookups are batched and cached across the whole run.
• mul labels are handled. Wikidata has moved Latin-script labels and aliases to the shared mul language, so asking for en alone returns an empty label on a large share of entities. This Actor sends languagefallback=1 and adds mul to the chain, which is why Q42 still comes back as "Douglas Adams".
• Transient errors are retried. 429 and 5xx get four attempts with exponential backoff and jitter.
• Descriptive user agent. Wikimedia throttles generic clients within a handful of requests; every request identifies this Actor.
• Public data only. Wikidata is CC0. No authentication, no personal data, no access-control bypass.

FAQ

Do I need a proxy? No. Proxy configuration is not required to run this Actor.

Do I need an account on Wikidata? No. You supply no credentials.

What happens if an entity is unavailable? It is recorded in RUN_SUMMARY.failures and the run continues. The Actor only errors out if nothing at all could be fetched.

Can I schedule it? Yes, it is designed for scheduled runs. Diff on modified to detect changed entities cheaply.

How do I find a Q-id? Put the name in searchTerms and let the Actor resolve it, or read it from the Wikipedia sidebar under Wikidata item.

Why is a claim value { "special": "novalue" }? Wikidata distinguishes "we know there is no value" (novalue) from "we know there is one but not which" (somevalue). Both are preserved rather than silently becoming null.

Can I get lexemes or properties? Yes, L and P identifiers are accepted in entityIds alongside Q items.

Can I integrate it with something else? Yes, Apify API, client libraries, webhooks, scheduled runs, dataset exports (JSON/CSV/Excel) or MCP. Output is structured JSON.