Pricing

from $5.00 / 1,000 results

Try for free

Go to Apify Store

Duplicate & Near-Duplicate Content Checker

Try for free

Compares texts or URLs pairwise for exact similarity percentage plus overlapping phrases as evidence. Catches near-duplicate pages, AI-generated content that reads too similar, and plagiarism.

Pricing

from $5.00 / 1,000 results

Rating

0.0

(0)

Developer

victor

Actor stats

Bookmarked

Total users

Monthly active users

13 hours ago

Last modified

What it does

Give it a list of texts (or URLs — it fetches and extracts the visible text for you) and it compares every pair, returning:

A similarity percentage based on shared 8-word phrases
The actual shared phrases, so you can see exactly what's duplicated, not just trust a number

Input

{
  "items": [
    { "id": "page-a", "url": "https://example.com/product-1" },
    { "id": "page-b", "text": "Or paste raw text directly instead of a URL." }
  ],
  "minSimilarityToReport": 20
}

Who this is for

SEO teams checking if AI-generated pages read too similar to each other
Content managers finding redundant blog posts or landing pages
Agencies verifying content is unique after a migration or rewrite
E-commerce teams checking if product descriptions are too close to a competitor's or to each other

Pricing

Pay per use — no subscription:

$0.01 per run
$0.005 per comparison returned

RAG Dataset Quality Auditor

gifted_wagon/rag-dataset-quality-auditor

Find duplicate, stale, incomplete, repetitive, and poorly chunked documents before they weaken RAG retrieval.

Michael Olmos

Image Similarity API | Compare Images, Find Duplicates

johnvc/image-similarity-api

Compare images by visual similarity with a CLIP vision model plus perceptual hashing. Score a source image against up to 500 targets, catch exact and near-duplicates, and get one clean row per comparison. URLs, base64, or file upload. Pay per result, MCP ready for AI agents.

John

5.0

(1)

Shopify SEO & Performance Auditor

trovevault/shopify-seo-performance-auditor

Audits public Shopify stores for prioritized technical SEO, canonical, duplicate-content, structured-data, image, Lighthouse, and third-party bloat findings.

Trove Vault

Chinese AI Training Corpus Engine

zhorex/chinese-corpus-engine

Turn China's public web into AI-training-ready text. Pulls Weibo, Bilibili, Xueqiu, Douban & RedNote, then deduplicates, quality-scores, PII-scrubs and provenance-stamps every document. From $0.025/doc, pay-as-you-go. For LLM training-data teams, data vendors & academic NLP researchers.

Sami

Domain Health Checker

zinin/domain-health-checker

Audit DNS resolution, MX, SPF, DMARC, name servers, and TLS for a list of domains. Complete evidence cards include a 0–100 score, source states, issues, confidence, data gaps, and a next action. Incomplete or unsafe destinations remain unscored and are not billed as results.

Tim Zinin

SEO Service Area Visibility Checker

solutionssmart/seo-service-area-visibility

Checks how visible a business is in Google across multiple service areas and generates an SEO visibility report.

Solutions Smart

SEO Page Audit — Sitemap Crawl & On-Page Checker

insight.solutions/seo-page-audit

Point it at a domain. It reads robots.txt, walks the sitemap (gzip and nested indexes included) or crawls the site, then audits every page: title, description, H1s, canonical, hreflang, schema, images, links, speed. 23 issue codes, a transparent 0-100 score, optional broken-link check. No API key.

Insight Solutions

AI Crawler Checker – robots.txt Rules for GPTBot & ClaudeBot

locomint/ai-crawler-checker

AI crawler checker: read each website's robots.txt and see whether GPTBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot and 13 other AI crawlers are allowed or blocked, with the line that decided each verdict, plus Content-Signal and llms.txt.

Locomint

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

Locomint

AI Crawler & Content-Signal Permission Delta Monitor

stefano_seggio/ai-crawler-content-signal-permission-monitor

Watches any domain and reports the instant its AI-crawler permissions change: a robots.txt Allow/Disallow flip for GPTBot, ClaudeBot and 15 other tracked tokens, a Cloudflare Content-Signal flip, or llms.txt appearing/changing. Pay-per-event: billed only for what changed, with a free baseline audit.

Stefano Seggio

RSS Feed Reader - Bulk RSS & Atom Feed Parser

logiover/bulk-rss-feed-reader

Read and parse RSS, Atom and RDF feeds in bulk, or auto-discover feeds from any website. Extract thousands of articles with full metadata for news monitoring, content aggregation and AI/RAG pipelines. No API key, export to CSV or JSON.