Pricing

from $0.01 / 1,000 results

Try for free

Go to Apify Store

Get URLs from link

Try for free

Extracts URLs from a sitemap or webpage with intuitive path matching. Use comma-separated patterns to include or exclude URL paths with smart matching: '/tags/' for exact paths, '/product' for paths starting with, or simple text for substring matches.

Pricing

from $0.01 / 1,000 results

Rating

5.0

(2)

Developer

Audrius L.

Actor stats

Bookmarked

262

Total users

Monthly active users

6 days ago

Last modified

Categories

Developer tools

You can access the Get URLs from link programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1from apify_client import ApifyClient
2
3# Initialize the ApifyClient with your Apify API token
4# Replace '<YOUR_API_TOKEN>' with your token.
5client = ApifyClient("<YOUR_API_TOKEN>")
6
7# Prepare the Actor input
8run_input = {
9    "link": "https://www.google.com/sitemap.xml",
10    "excludeExtensions": "jpg,jpeg,png,gif,bmp,webp,svg,ico",
11    "customExcludePattern": "/tags/,category",
12}
13
14# Run the Actor and wait for it to finish
15run = client.actor("boring_code/get-urls-from-link").call(run_input=run_input)
16
17# Fetch and print Actor results from the run's dataset (if there are any)
18print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
19for item in client.dataset(run.default_dataset_id).iterate_items():
20    print(item)
21
22# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

Get URLs from link API in Python

The Apify API client for Python is the official library that allows you to use Get URLs from link API in Python, providing convenience functions and automatic retries on errors.

Install the apify-client

$pip install apify-client

Other API clients include:

Get URLs from link API in JavaScript

Get URLs from link API through CLI

Get URLs from link OpenAPI definition

Get URLs from link API

Product Matching API

vivid_astronaut/product-matching

BRAINIALL Team

Sitemap URL Finder

thescrapelab/sitemap-target-url-extractor

Find and export URLs from any website’s robots.txt and sitemaps. Enter a domain or website URL, optionally filter matching URLs by text, and get clean dataset rows with the URL, domain, path, source sitemap, and match details.

Inus Grobler

Sitemap Scraper: Every URL on Any Site

glitchbound/sitemap-scraper

Get every URL a website publishes, from a bare domain. Finds the sitemap through robots.txt or the usual paths, follows nested sitemap index files to any depth, unpacks .xml.gz, and returns each URL with lastmod, changefreq and priority. Filter by path regex or by what changed since a date.

Daniel Meshulam

Public Integration Ecosystem Signal Agent

jacksu/public-integration-ecosystem-signal-agent

Analyze public integrations, app marketplace, partner, connector, and ecosystem pages for tool coverage, category coverage, API/docs paths, partner paths, marketplace signals, risks, and change status.

jack su

Sitemap Sniffer

crawlerbros/sitemap-sniffer

Discover every sitemap file for a website. Reads robots.txt for Sitemap directives, probes common sitemap paths, and recursively unpacks sitemap-index files. HTTP-only, no proxy or cookies needed.

Crawler Bros

Product Matching

moving_beacon-owner1/product-matching

Product Matching — matches products across two datasets using identifiers, names, and brands, then returns confidence scores, matching evidence, product differences, and clear match classifications.

Jamshaid Arif

Sitemap Extractor

johnlenflure/sitemap-extractor

Extract and parse XML sitemaps from any website. Discovers sitemaps via robots.txt and common paths. Outputs structured URL data with metadata.

Sinan Donmez

YouTube Channel Videos & Views Scraper - No Blocked Paths

neverempty/youtube-channel-videos

For channel tracking, competitor research and AI datasets: give it a handle and get that channel's videos with views, length and publish time - read only through paths YouTube's robots.txt allows. The RSS feed most scrapers use is disallowed; the allowed page gave 30 videos where RSS gives 15.

NeverEmpty

Robots.txt & Sitemap Analyzer

automation-lab/robots-sitemap-analyzer

This actor fetches and parses robots.txt and sitemap.xml files for any list of websites. It extracts crawl directives (user-agent rules, allowed/disallowed paths, crawl-delay), discovers sitemap URLs, and counts the number of pages listed in each sitemap. Use it for SEO audits, competitive...

Stas Persiianenko

Sitemap Generator

datawinder/sitemap-generator

Automatically crawl a website and generate an SEO-ready sitemap in XML, HTML, or TXT format. Supports crawl depth limits, URL include/exclude patterns, and optional merging with an existing sitemap.xml. Ideal for SEO audits, site migrations, and automation.