Url to Hash

Generate cryptographic hashes from any URL - webpages, images, PDFs, documents, and more. Perfect for monitoring content changes, verifying file integrity, and detecting duplicates across any file type accessible via URL.

---

What does URL to Hash do?

URL to Hash fetches content from any URL and generates cryptographic hashes (MD5, SHA-256, or SHA-512). Whether it's a webpage, image, PDF, spreadsheet, or any other file, you'll get a unique hash fingerprint. Even the smallest change in content will produce a completely different hash.

Supported File Types

Webpages - HTML, PHP, ASP pages
Images - JPG, PNG, GIF, WebP, SVG
Documents - PDF, DOCX, TXT, RTF
Spreadsheets - XLSX, XLS, CSV, TSV
Videos - MP4, AVI, MOV
Archives - ZIP, RAR, TAR, GZ
Any file accessible via direct URL

Why use URL to Hash?

Monitor website changes - Track when web content is updated by comparing hashes over time
Verify file integrity - Ensure downloaded files (PDFs, images, documents) match expected values
Detect duplicates - Identify identical files across different URLs or locations
Content auditing - Compare versions of documents, images, or web pages
Security verification - Verify that files haven't been tampered with or altered
Image tracking - Monitor when images or graphics are updated on websites
Document versioning - Track changes to PDFs, Word docs, or spreadsheets over time

---

Quick Start

1. Add URLs - Enter one or more URLs you want to hash
2. Choose algorithm - Select MD5, SHA-256 (default), or SHA-512
3. Run - Get results in seconds with hash values for each URL

---

Input

{
  "startUrls": [
    { "url": "https://example.com" },
    { "url": "https://another-site.com/page" }
  ],
  "hashAlgorithm": "SHA-256",
  "proxyConfiguration": {
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"]
  }
}

Input Parameters

Parameter	Type	Required	Description
URLs	Array	Yes	List of URLs to fetch and hash. Works with any file type: webpages, images, PDFs, documents, videos, etc. Supports bulk upload via text file.
Hash Algorithm	String	Yes	Choose: MD5, SHA-256 (default), or SHA-512
Proxy Configuration	Object	Optional	Proxy settings (defaults to Apify Residential proxies)

---

Output

Each URL returns a row with the following data:

{
  "url": "https://example.com",
  "hash": "6f5635035f36ad500b4fc4bb7816bb72ef5594e1bcae44fa074c5e988fc4c0fe",
  "hashAlgorithm": "SHA-256",
  "status": "success"
}

Output Fields

Field	Description
URL	The original URL that was fetched
Hash	Cryptographic hash of the entire file/content (hexadecimal format)
Hash Algorithm	The algorithm used (MD5, SHA-256, or SHA-512)
Status	"success" or "error" with detailed error message

---

Use Cases

1. Website Change Detection

Monitor competitors' websites, documentation pages, or pricing pages. Run the actor daily and compare hashes - when the hash changes, you know content was updated.

2. File Integrity Verification

Verify that downloaded files (software installers, PDFs, images, documents) haven't been altered or corrupted. Compare the hash of your downloaded file against the official hash to ensure authenticity.

3. Image & Media Monitoring

Track when product images, logos, banners, or other media files change on websites. Perfect for brand monitoring and competitive analysis.

4. Document Version Control

Monitor legal documents, terms of service, privacy policies, contracts, or PDFs for changes over time. Get instant alerts when documents are updated.

5. Duplicate Content Detection

Find duplicate or near-duplicate files across multiple URLs by comparing their hashes. Identical hashes mean identical files, making it easy to identify duplicates.

6. Web Scraping Quality Control

When scraping websites, use hashes to verify you're getting consistent data and detect when target pages change structure.

---

Technical Details

• Concurrency: Processes up to 7 URLs simultaneously for fast results
• Retry Logic: Automatically retries failed requests once
• Supported Content: Any file type accessible via URL (HTML, images, PDFs, documents, videos, archives, etc.)
• Hash Algorithms:
  • MD5 - Fast, 128-bit (not recommended for security purposes)
  • SHA-256 - Secure, 256-bit (recommended default)
  • SHA-512 - Maximum security, 512-bit
• Content Hashed: Entire raw file/content exactly as downloaded (byte-for-byte)
• Proxy Support: Uses Apify Residential proxies by default (configurable)

---

Understanding Hash Algorithms

What is a hash?
A cryptographic hash function converts any amount of data into a fixed-size string of characters. The same input always produces the same hash, but even tiny changes produce completely different hashes.

Which algorithm should I choose?

Algorithm	Hash Length	Speed	Use Case
MD5	32 characters	Fastest	Quick content comparison (not for security)
SHA-256	64 characters	Fast	General purpose, good balance (recommended)
SHA-512	128 characters	Slower	Maximum security, sensitive content

---

Example Results

Input:

• URL 1: https://example.com
• URL 2: https://example.org
• Algorithm: SHA-256

Output:

URL	Hash	Hash Algorithm	Status
https://example.com	6f5635035f36ad500b4fc4bb7816bb72ef5594e1bcae44fa074c5e988fc4c0fe	SHA-256	success
https://example.org	a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447	SHA-256	success

---

Limitations

• Direct URLs only - Must be direct links to files/content (not download pages or login-protected content)
• Dynamic content - For webpages with timestamps, session IDs, or personalized content that will produce different hashes on each request
• No JavaScript rendering - Only the initial HTML response is hashed (JavaScript-rendered content is not executed)
• Single retry - Failed requests are retried only once before being marked as errors
• Authentication - Cannot access password-protected or login-required content

---

Troubleshooting

Q: Why do I get different hashes for the same URL?
A: The file/content likely changed between requests. For webpages, dynamic elements like timestamps or session IDs can also cause different hashes.

Q: Can I hash files that require login or authentication?
A: No, the actor can only access publicly available URLs that don't require authentication.

Q: Does it work with all file types?
A: Yes! As long as the file is accessible via a direct URL, it will be downloaded and hashed. This includes images, PDFs, documents, videos, archives, and any other file type.

Q: Can I hash JavaScript-rendered content?
A: No, for webpages, this actor only hashes the initial HTML response. For JavaScript-heavy sites, consider using a browser-based scraper first.

Q: Which hash algorithm is most secure?
A: SHA-512 is the most secure, but SHA-256 is recommended for most use cases as it provides excellent security with better performance.

Q: How do I monitor a file or page for changes?
A: Run the actor periodically (daily/weekly) using Apify's scheduling feature. Store the hashes and compare new results against previous runs.

---

Support

Need help? Have questions or need additional features?

We're here to support you! Whether you need assistance getting started, have questions about the actor, or want to request new features, we're ready to help.

Contact Support - Fill out this quick form

We'll respond as quickly as possible and work with you to resolve any issues or implement features that make this actor even more useful for your needs.

---

Built with care by One Scales Inc.

---

Tags

content-monitoring hash cryptography web-scraping change-detection duplicate-detection sha256 md5 sha512 data-integrity content-verification file-integrity image-hashing pdf-hash document-verification