# Original Source Finder - Syndicated Article & Canonical Checker (`neverempty/original-source-finder`) Actor

For AI agents citing news: give article URLs (Yahoo, MSN, dev.to, AP/Reuters reprints) and get the original source, the original URL or news agency, with every clue, a confidence and a same-article check. On 37 pages with a known answer, 35 were right and none named a wrong source.

- **URL**: https://apify.com/neverempty/original-source-finder.md
- **Developed by:** [NeverEmpty](https://apify.com/neverempty) (community)
- **Categories:** AI, News, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $2.80 / 1,000 url checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Original Source Finder - Syndicated Article & Canonical Checker

**For AI agents and researchers who must cite the original source:** give article URLs and get, for each one, where the story was first published - the original article URL, or the news agency or publisher it came from - with every clue as evidence, a confidence, and a check that the original page really is the same article.

The same story often sits on many sites at once: a Yahoo News or MSN copy of a newspaper story, an AP or Reuters wire story on a local TV site, a dev.to or Medium cross-post of a blog, a Conversation article republished under Creative Commons. Citing the copy instead of the original is a common mistake. This Actor reads the page the way a careful editor would and tells you which one to cite - and says so plainly when it cannot tell.

### Input and output in one example

Input (the default when you give nothing):

```json
{ "urls": ["https://dev.to/bradtraversy/traditional-coding-vs-agentic-coding-the-flow-state-problem-57p5"] }
```

Output (one row per URL, shortened):

```json
{
  "status": "ok",
  "url": "https://dev.to/bradtraversy/traditional-coding-vs-agentic-coding-the-flow-state-problem-57p5",
  "originStatus": "original-found",
  "originUrl": "https://www.traversymedia.com/blog/traditional-coding-vs-agentic-coding-the-flow-state-problem",
  "originName": "traversymedia.com",
  "confidence": "high",
  "originVerified": true,
  "summary": "This page is a republished copy: it names traversymedia.com (https://www.traversymedia.com/blog/...) as the original (the canonical link, an \"originally published\" note). Confirmed by reading that page: the opening text of this page is on the original page (exact). Cite the original.",
  "citeUrl": "https://www.traversymedia.com/blog/traditional-coding-vs-agentic-coding-the-flow-state-problem",
  "titleSimilarity": 1,
  "openingTextMatch": "exact",
  "firstPublished": "same-day",
  "signals": [
    { "signal": "canonical-other-site", "strength": "strong", "pointsTo": "https://www.traversymedia.com/blog/...", "evidence": "<link rel=\"canonical\" href=\"...\"> names a page on another site as the main version of this page" },
    { "signal": "text-attribution", "strength": "strong", "pointsTo": "https://www.traversymedia.com/blog/...", "evidence": "the page says \"…Originally published at traversymedia.com…\" and links to ..." }
  ]
}
```

A wire story comes back like this (a WTOP copy of an AP story):

```json
{ "originStatus": "original-named", "originName": "The Associated Press", "originType": "news-agency", "confidence": "high", "originUrl": null,
  "citeAs": "The Associated Press, via WTOP News",
  "signals": [{ "signal": "news-agency-dateline", "strength": "strong", "evidence": "the story opens with the agency dateline \"BANGKOK (AP) —\"" }] }
```

### What it looks at

| Clue (`signal`) | Example |
|---|---|
| `canonical-other-site` | `<link rel="canonical">` pointing to another site (dev.to, Medium, Yahoo Finance and many publishers use this for syndicated copies) |
| `meta-original-source` | `<meta name="original-source">`, `<meta name="syndication-source">`, `<link rel="original-source">` |
| `jsonld-is-based-on`, `jsonld-url-other-site` | JSON-LD `isBasedOn`, or the article's `url` / `mainEntityOfPage` on another site |
| `text-attribution` | "Originally published at …", "This article first appeared in …", "republished from The Conversation", "The post … appeared first on …", 「転載」「初出」「元記事」 - with the link next to it |
| `news-agency-dateline` | "(AP) —", "(Reuters) -", "(Bloomberg) --", "(AFP)", 「（共同）」「【共同】」「［東京 ロイター］」 at the start of the story |
| `news-agency-byline` | "By The Associated Press", JSON-LD author / provider / sourceOrganization, meta author, a title ending in 「（共同通信）」 |
| `provider-named` | JSON-LD provider or author organization other than the site, "Provided by …", 「提供：…」 |
| `msn-source` | MSN articles: MSN's own article data names the provider's article (MSN pages draw their text with JavaScript, so the page HTML has nothing) |
| `url-wrapper` | AMP cache (`*.cdn.ampproject.org/c/s/...`), Google AMP (`google.com/amp/s/...`) and older Google News links that contain the article address |
| `og-url-other-site` | `og:url` on another site (weak: sometimes a copied setting) |

News agencies recognised: The Associated Press, Reuters, AFP, Bloomberg, dpa, EFE, ANSA, UPI, Xinhua, Yonhap, Kyodo News (共同通信), Jiji Press (時事通信).

### It does not over-claim

- **No clue is not proof.** A page with no sign of being a copy gets `no-other-source-found`: "It may be the original, or a copy that does not say so". It never says "this page is the original".
- **Wire stories get the agency, not a made-up URL.** Agency copies are often not published at a public URL, so `originUrl` stays null and `citeAs` says "Agency, via Site".
- **The named original is checked.** With `verifyOriginal` on (default), the original page is read (respecting its robots.txt) and compared: `originVerified` is `true` only when this page's opening text is on the original page or the headlines match, `false` when the named page is clearly a different article (for example a canonical link to a section front page), and `null` when it could not be read or the comparison is not conclusive.
- **Dates do not decide.** `firstPublished` shows which page carries the earlier date (read the way the Page Publish Date Finder reads dates: JSON-LD, meta tags, `<time>`, visible text, URL), but the original is chosen from the clues, not from dates.
- **Pages it cannot fully read are free rows.** A page that draws its article with JavaScript, a check page (CAPTCHA, Cloudflare), a sign-in page or a page whose robots.txt disallows reading comes back as a free row that says why. It does not solve or bypass check pages and does not sign in.

### Measured

Checked on 2026-09-25 against 37 real republished and original pages whose answer is known from a separate source (dev.to's API `canonical_url`, the `<source>` of the Yahoo Finance RSS feed, the per-publisher Yahoo!ニュース RSS feed, AP wire pages, pages carrying The Conversation's republishing notice, MSN syndication):

| Kind | URLs | Right |
|---|---|---|
| dev.to cross-posts (original found at the canonical URL) | 8 | 8 |
| dev.to original posts (no other source) | 3 | 3 |
| Yahoo Finance partner articles (source named or found) | 5 | 5 |
| Yahoo!ニュース (Kyodo, Nikkei, Sankei Sports, FNN) | 4 | 4 |
| AP stories on WTOP and ABC News | 4 | 4 |
| The Conversation republished (phys.org, Yahoo Australia, CounterPunch, PeaceRep) | 5 | 5 |
| MSN syndication | 3 | 3 |
| Publishers' own articles (Jiji Press, Salon) | 2 | 2 |
| AMP cache link | 1 | 1 |
| AP story with no AP mark on the page (The Independent) | 1 | 0 (says no-other-source-found) |
| Medium (Cloudflare check page) | 1 | free row (not read) |

### Input

| Field | What it does |
|---|---|
| `urls` | Article URLs, 1 to 1,000 per run. |
| `url` | One URL without a list (added to `urls`). |
| `verifyOriginal` | Default `true`: open the named original page and compare it (charged as `source-verified`). |
| `onlyChanges` | Monitor: return only URLs whose answer changed since the last run of the same watch. |
| `watchName` | Monitor: the name of the list being watched. |
| `resetMonitoringState` | Monitor: forget what the watch remembered. |
| `maxConcurrency` | URLs handled at the same time (1 to 8, default 4). |
| `requestTimeoutSecs` | Time limit per request (5 to 60 seconds, default 20). |

### Pricing

Pay per event: a small start fee once per run that returns at least one checked URL, plus one charge per checked URL - `source-checked` when the page was read and answered, or `source-verified` (a little more) when the original page was also opened and compared. Free rows (blocked, sign-in, robots.txt, JavaScript-only, not found, bad input) cost nothing. A run whose maximum total charge has no room for the start fee plus one URL requests nothing and costs nothing.

### Good to know

- robots.txt is followed for every page, including the original page it opens (a robots.txt that disallows reading means the page is not requested). Crawl-delay is respected.
- The reader identifies itself as `NeverEmptyOriginalSource/1.0`; it does not run JavaScript.
- Google News links: news.google.com does not allow automated reading in its robots.txt, so only links that contain the article address in themselves can be answered.
- This Actor is not affiliated with any of the sites or agencies named here. It reads only public pages.

### Support

Questions and requests: open an issue in the Issues tab.

# Actor input Schema

## `urls` (type: `array`):

Article pages to check (1 to 1,000 per run), for example a Yahoo News, Yahoo Finance, MSN, dev.to or Medium copy, a news site running an AP or Reuters story, or an AMP / Google News link. Each URL gets one row: the original article URL when the page names one, or the original publisher or news agency when only a name is given, with every clue as evidence. If this and the single URL field are both empty, an example dev.to cross-post is checked.

## `url` (type: `string`):

For checking one page without a list. Added to Article URLs if both are given.

## `verifyOriginal` (type: `boolean`):

When the page names an original article URL, read that page too (respecting its robots.txt) and compare the headline and the opening text, so the row says originVerified true (same article), false (the named page is a different article) or null (could not be read or not conclusive). Rows where the original page was read and compared are charged as source-verified; turn this off to only read the page you give (source-checked).

## `onlyChanges` (type: `boolean`):

Return a row only when the original-source answer for a URL differs from the last run of the same watch (for example a page that later adds a canonical link or an attribution). The first run of a watch returns every URL. A changed answer is read a second time before it is reported.

## `watchName` (type: `string`):

Name of the watch whose results are remembered between runs (letters, digits, dot, dash, underscore; up to 40). Setting it turns on monitor mode; use different names for different lists.

## `resetMonitoringState` (type: `boolean`):

Start the watch over: this run treats every URL as a first check.

## `maxConcurrency` (type: `integer`):

How many URLs are handled in parallel (1 to 8). Requests to the same site still follow its robots.txt Crawl-delay.

## `requestTimeoutSecs` (type: `integer`):

How long to wait for one page to answer (5 to 60). A page that does not answer is asked again up to two more times.

## Actor input object example

```json
{
  "urls": [
    "https://dev.to/bradtraversy/traditional-coding-vs-agentic-coding-the-flow-state-problem-57p5",
    "https://www.msn.com/en-za/news/other/latest-news-bulletin-september-22nd-2026-morning/ar-AA2cIkpb"
  ],
  "verifyOriginal": true,
  "onlyChanges": false,
  "resetMonitoringState": false,
  "maxConcurrency": 4,
  "requestTimeoutSecs": 20
}
```

# Actor output Schema

## `results` (type: `string`):

One row per article URL: originStatus (original-found, original-named or no-other-source-found), the original URL or the news agency / publisher name, confidence, originVerified (the original page was read and compared: same article or not), which URL to cite, publication dates of both pages, and every clue as evidence. A page that robots.txt does not allow, that needs a sign-in, that shows a check page or that draws its text with JavaScript comes back as a free row that says why nothing is claimed.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://dev.to/bradtraversy/traditional-coding-vs-agentic-coding-the-flow-state-problem-57p5",
        "https://www.msn.com/en-za/news/other/latest-news-bulletin-september-22nd-2026-morning/ar-AA2cIkpb"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("neverempty/original-source-finder").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://dev.to/bradtraversy/traditional-coding-vs-agentic-coding-the-flow-state-problem-57p5",
        "https://www.msn.com/en-za/news/other/latest-news-bulletin-september-22nd-2026-morning/ar-AA2cIkpb",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("neverempty/original-source-finder").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://dev.to/bradtraversy/traditional-coding-vs-agentic-coding-the-flow-state-problem-57p5",
    "https://www.msn.com/en-za/news/other/latest-news-bulletin-september-22nd-2026-morning/ar-AA2cIkpb"
  ]
}' |
apify call neverempty/original-source-finder --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,neverempty/original-source-finder"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/BIJRf0GgnmXO6T4AG/builds/sU5xyhldVcqbcDPtx/openapi.json