Pricing

from $0.20 / 1,000 normalized urls

Try for free

Go to Apify Store

URL Normalizer & Deduplicator

Try for free

Cleans a list of URLs and marks the ones that point at the same page. Strips tracking parameters and unifies scheme, host case, default ports, query order and trailing slashes.

Pricing

from $0.20 / 1,000 normalized urls

Rating

0.0

(0)

Developer

Ryusuke Asai

Actor stats

Bookmarked

Total users

Monthly active users

22 days ago

Last modified

Categories

Developer tools

SEO tools

You can access the URL Normalizer & Deduplicator programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "urls": [
10        "https://Example.com/a?utm_source=x&id=1",
11        "https://example.com/a/",
12        "http://example.com:80/a",
13    ],
14    "stripParams": [
15        "utm_source",
16        "utm_medium",
17        "utm_campaign",
18        "utm_term",
19        "utm_content",
20        "fbclid",
21        "gclid",
22        "ref",
23    ],
24    "forceHttps": True,
25    "keepTrailingSlash": False,
26}
27
28# Run the Actor and wait for it to finish
29run = client.actor("ryua_marines/url-normalizer").call(run_input=run_input)
30
31# Fetch and print Actor results from the run's dataset (if there are any)
32print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
33for item in client.dataset(run.default_dataset_id).iterate_items():
34    print(item)
35
36# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

URL Normalizer: clean URL lists, find duplicates API in Python

The Apify API client for Python is the official library that allows you to use URL Normalizer & Deduplicator API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

URL Normalizer & Deduplicator API in JavaScript

URL Normalizer & Deduplicator API through CLI

URL Normalizer & Deduplicator OpenAPI definition

URL Normalizer & Deduplicator API

Project Normalizer

wild_equipment/project-normalizer

Zhang Luxin

CDN Cache and Compression Delivery Contract Gate

kingii98/cdn-cache-and-compression-delivery-contract-gate

Proves that a deploy serves the delivery contract that you agreed. For every URL that you name, the Actor sends header-only transactions: one request for each required compression algorithm, an HTTP-scheme probe and a trailing-slash variant, so four trans

kingii98

YouTube Keyword Rank Tracker

sarangofficial64/youtube-keyword-rank-tracker

Searches YouTube for a list of keywords and returns the ranked organic video results in the same order a real user would see them, for a given country.

Sarang V

Marks & Spencer Scraper

toludare/marks-and-spencer-scraper

Scrape the multinational British retailer, Marks & Spencer. Input direct product URLs or search query. Extract titles, description, product codes, prices, ratings, images, sizes, colours, offers, reviews and much more.

tolu.

Facebook Page URL Resolver — match duplicate page links (free)

mentat-logic/mentat-resolver

The same Facebook page appears in exports under several URLs: /profile.php links, vanity names, numeric IDs, mobile hosts, tracking parameters. This canonicalizes them and tells you which are the same page. Identities it cannot match with certainty are reported with evidence, never merged. Free.

Mentat Logic

Dataset Deduplicator v2

zentrafoundry/dataset-deduplicator-v2

Transform dataset deduplicator v2 inputs into structured rows, clear errors, confidence signals, and automation-ready output.

Zentra

Tennis Point-by-Point Scraper — Rally & Match Stats

sian.agency/tennis-point-by-point-scraper

Point-by-point tennis data no other scraper sells — every serve, rally, and break point of a match, in play order, plus the complete match- and set-level statistics table. Find players by name, then export full match records as JSON, CSV, or Excel. No API key.

SIÁN OÜ

165

Port Scanner

onescales/port-scanner

Scan hundreds of hosts for open TCP ports in one run. Get open ports, detected services, and versions as CSV, JSON, or Excel.

One Scales

5.0

Domain name or website url to Linkedin Company page url

sbzh/domain-name-or-website-url-to-linkedin-company-page-url

Use this tool to retrieve the LinkedIn URL of the company page on the website of the company you are searching for. Simply enter the domain name or URL of the company's website and retrieve the LinkedIn URL of the company page in the format https://www.linkedin.com/company/...

Sambzh

whiskybase-scraper

mrdoe/whiskybase-scraper

the online store run by Whiskybase, one of the largest whisky communities in the world. Point it at a category page, the "new arrivals" list, or an individual product page, and it returns clean, structured data.