Get Urls Pro

Pricing

$18.00/month + usage

Try for free

Go to Apify Store

Get Urls Pro

Try for free

This Apify actor crawls websites, extracts and creates a hierarchy of links, allowing you to visualize the structure of a website. The crawler can be configured to use either standard HTTP requests with BeautifulSoup (fast HTML parsing) or Selenium (for JavaScript-heavy pages)

Pricing

$18.00/month + usage

Rating

5.0

(1)

Developer

Maged

Maintained by Community

Actor stats

Bookmarked

Total users

Monthly active users

2 months ago

Last modified

Start URL

startUrlstringRequired

The starting URL to crawl (e.g., https://jamesclear.com/five-step-creative-process)

Use Selenium Browser

useSeleniumbooleanOptional

Use Selenium for JavaScript-heavy pages (slower but more accurate for dynamic content)

Default value of this property is false

Allow Duplicate URLs

allowDuplicatesbooleanOptional

Allow duplicate URLs to be included in the output. If false, each URL will appear only once.

Default value of this property is false

Maximum Crawl Depth

maxDepthintegerOptional

Maximum depth of link recursion

Default value of this property is 3

Maximum Children per Link

maxChildrenPerLinkintegerOptional

Maximum number of children links to follow from each parent link

Default value of this property is 3

Same Domain Only

sameDomainOnlybooleanOptional

Only crawl URLs within the same domain as the start URL

Default value of this property is true

Ignored File Extensions

ignoredExtensionsarrayOptional

File extensions to ignore when crawling

Get Urls Rental

maged120/get-urls-rental

scrap links from any website, ppr version: https://apify.com/maged120/get-urls

Maged

5.0

Get URLs from link

boring_code/get-urls-from-link

Extracts URLs from a sitemap or webpage with intuitive path matching. Use comma-separated patterns to include or exclude URL paths with smart matching: '/tags/' for exact paths, '/product' for paths starting with, or simple text for substring matches.

Audrius L.

199

5.0

Link Extractor

juansgaitan/link-extractor

Extract links from an Array of different paths/users parsed with a baseUrl and a pageFunction.

Juan Gaitán Villamizar

177

Find Sitemap from url

eesti/find-sitemap-from-url

A powerful [Apify Actor] that finds sitemap URLs for any website. This Actor helps you discover XML sitemaps by checking common locations, robots.txt files, and analyzing HTML content for sitemap links.

ando

149

Sitemap Scraper

pvillalva/sitemap-scraper

The Sitemap Scraper extracts and outputs all URLs from a given sitemap.

Percival Villalva

152

5.0

Cheerio Scraper

apify/cheerio-scraper

Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.

Apify

11K

4.9

Sitemap Sniffer

vaclavrut/sitemap-sniffer

Sitemap sniffer will check the most used variants of sitemaps and you can use that for crawling. This will just save you time so you don't have to check manually.

Vaclav Rut

683

5.0

Simple HTTP Status Code Checker

onescales/simple-http-status-code-checker

This Apify Actor allows you to check in bulk HTTP status codes and redirects for a list of URL's. It's a useful tool for SEO audits, website maintenance, and link checking.

One Scales

189

5.0

Sitemap URL Extractor

onescales/sitemap-url-extractor

Provide a link to a sitemap.xml and the app will extract and list all URLs in the sitemap as well as additional data in the sitemap (i.e. https://onescales.com/sitemap.xml).

One Scales

235

5.0

URLs List - Extract ALL website urls

lofomachines/urls-extractor

Automatically discovers and extracts ALL URLs from any website. Perfect for SEO analysis, content inventory, and bulk URL extraction from multiple websites. Get complete URL lists with metadata including last modified dates and priority levels.