This Apify actor crawls websites, extracts and creates a hierarchy of links, allowing you to visualize the structure of a website. The crawler can be configured to use either standard HTTP requests with BeautifulSoup (fast HTML parsing) or Selenium (for JavaScript-heavy pages)

Maged

5.0

Download HTML from URLs

mtrunkat/url-list-download-html

This actor takes a list of URLs and downloads HTML of each page.

Marek Trunkát

8.4k

Get URLs from link

boring_code/get-urls-from-link

Extracts URLs from a sitemap or webpage with intuitive path matching. Use comma-separated patterns to include or exclude URL paths with smart matching: '/tags/' for exact paths, '/product' for paths starting with, or simple text for substring matches.

Audrius L.

5.0

Get Urls Pro

maged120/get-urls-pro

Maged

5.0

Link Extractor

juansgaitan/link-extractor

Extract links from an Array of different paths/users parsed with a baseUrl and a pageFunction.

Juan Gaitán Villamizar

172

Internal Links Scraper

mysteriousshadow/internal-links-scraper

When given a sitemap of a website, this scraper will go through every page listed on the sitemap and find all the internal links. Useful for SEO, finding orphaned pages, and visualizing internal linking structure.

Mysterious Shadow

Get Site to Markdown

jhaisley/get-site

Website to Markdown Crawler An asynchronous web crawler that mirrors websites into a single organized markdown file, with handling for images and directory structure preservation. Designed to operate with low cost. This works great to build context for AI agents.

b-w.pro

🔗✨ Link Extractor Pro: URL to HTML List Downloader

dainty_screw/link-extractor-pro-url-to-html-list-downloader

Maximize productivity with HTML URL List Downloader. Quickly extract, manage, and organize URLs from HTML pages. Ideal for SEO professionals and digital marketers. Streamline your workflow today!

codemaster devops

115

Find Sitemap from url

eesti/find-sitemap-from-url

A powerful [Apify Actor] that finds sitemap URLs for any website. This Actor helps you discover XML sitemaps by checking common locations, robots.txt files, and analyzing HTML content for sitemap links.