Pricing

from $0.20 / 1,000 normalized urls

Try for free

Go to Apify Store

URL Normalizer & Deduplicator

Try for free

Cleans a list of URLs and marks the ones that point at the same page. Strips tracking parameters and unifies scheme, host case, default ports, query order and trailing slashes.

Pricing

from $0.20 / 1,000 normalized urls

Rating

0.0

(0)

Developer

Ryusuke Asai

Actor stats

Bookmarked

Total users

Monthly active users

22 days ago

Last modified

What it does to each URL

	Before	After
Host case	`https://Example.COM/a`	`https://example.com/a`
Default port	`https://example.com:443/a`	`https://example.com/a`
Tracking parameters	`…/a?utm_source=x&id=1`	`…/a?id=1`
Query order	`…/a?b=2&a=1`	`…/a?a=1&b=2`
Trailing slash	`https://example.com/a/`	`https://example.com/a`
Fragment	`…/a#section`	`…/a`
Scheme	`http://example.com/a`	`https://example.com/a`

Non-default ports, percent-encoding and query parameters you actually use are left alone.

Input

Field	Type	Default
`urls`	array	—	required
`stripParams`	array	`utm_*`, `fbclid`, `gclid`, `ref`	the parameters to remove
`forceHttps`	boolean	`true`	rewrite `http://` as `https://`
`keepTrailingSlash`	boolean	`false`	keep the slash at the end of the path

Output

One row per input URL, in the order you sent them.

{ "original": "https://Example.com/a?utm_source=x", "normalized": "https://example.com/a", "isDuplicate": false }
{ "original": "https://example.com/a/",             "normalized": "https://example.com/a", "isDuplicate": true }
{ "original": "not a url",                          "normalized": "",                      "isDuplicate": false, "error": "not a valid URL" }

Nothing is dropped. A URL that cannot be parsed comes back with an error and the run continues, so the output always lines up with the input.

What it does not do

It does not fetch anything. No requests are made to the URLs, so it is fast and cannot be blocked.
It does not follow redirects. Two URLs that redirect to the same page are not detected as duplicates.
It does not know your site. If a query parameter matters to you, keep it out of stripParams.

Project Normalizer

wild_equipment/project-normalizer

Zhang Luxin

CDN Cache and Compression Delivery Contract Gate

kingii98/cdn-cache-and-compression-delivery-contract-gate

Proves that a deploy serves the delivery contract that you agreed. For every URL that you name, the Actor sends header-only transactions: one request for each required compression algorithm, an HTTP-scheme probe and a trailing-slash variant, so four trans

kingii98

YouTube Keyword Rank Tracker

sarangofficial64/youtube-keyword-rank-tracker

Searches YouTube for a list of keywords and returns the ranked organic video results in the same order a real user would see them, for a given country.

Sarang V

Marks & Spencer Scraper

toludare/marks-and-spencer-scraper

Scrape the multinational British retailer, Marks & Spencer. Input direct product URLs or search query. Extract titles, description, product codes, prices, ratings, images, sizes, colours, offers, reviews and much more.

tolu.

Facebook Page URL Resolver — match duplicate page links (free)

mentat-logic/mentat-resolver

The same Facebook page appears in exports under several URLs: /profile.php links, vanity names, numeric IDs, mobile hosts, tracking parameters. This canonicalizes them and tells you which are the same page. Identities it cannot match with certainty are reported with evidence, never merged. Free.

Mentat Logic

Dataset Deduplicator v2

zentrafoundry/dataset-deduplicator-v2

Transform dataset deduplicator v2 inputs into structured rows, clear errors, confidence signals, and automation-ready output.

Zentra

Tennis Point-by-Point Scraper — Rally & Match Stats

sian.agency/tennis-point-by-point-scraper

Point-by-point tennis data no other scraper sells — every serve, rally, and break point of a match, in play order, plus the complete match- and set-level statistics table. Find players by name, then export full match records as JSON, CSV, or Excel. No API key.

SIÁN OÜ

165

Port Scanner

onescales/port-scanner

Scan hundreds of hosts for open TCP ports in one run. Get open ports, detected services, and versions as CSV, JSON, or Excel.

One Scales

5.0

Domain name or website url to Linkedin Company page url

sbzh/domain-name-or-website-url-to-linkedin-company-page-url

Use this tool to retrieve the LinkedIn URL of the company page on the website of the company you are searching for. Simply enter the domain name or URL of the company's website and retrieve the LinkedIn URL of the company page in the format https://www.linkedin.com/company/...

Sambzh

whiskybase-scraper

mrdoe/whiskybase-scraper

the online store run by Whiskybase, one of the largest whisky communities in the world. Point it at a category page, the "new arrivals" list, or an individual product page, and it returns clean, structured data.