Pricing

$20.00 / 1,000 url diagnoseds

Smart Web Scraper — Block Diagnosis & URL Debugger

Fetches URLs with intelligent fallback strategies and honest block detection. Diagnoses Cloudflare challenges, TLS mismatches, 429s, and JS requirements. Tries direct fetch, TLS downgrade, and archive.org, then reports what worked and why.

Pricing

$20.00 / 1,000 url diagnoseds

Rating

0.0

(0)

Developer

Mack

Actor stats

Bookmarked

Total users

Monthly active users

24 days ago

Last modified

Smart Web Scraper — Diagnose blocks, extract content

Why does your scraper break on certain sites? Stop guessing.

This actor fetches a URL and tells you exactly what went wrong — Cloudflare challenge, TLS mismatch, rate limit, JS requirement, or just a slow server. It tries three strategies (direct, TLS downgrade, archive.org fallback) and reports what worked.

What it returns per URL

{
  "url": "https://reddit.com",
  "status_code": 200,
  "blocked": true,
  "blocked_reason": "Cloudflare challenge page (Please wait for verification...)",
  "source": "direct",
  "tls_fallback_used": false,
  "server": "cloudflare",
  "response_time_ms": 423,
  "redirect_chain": ["https://reddit.com", "https://www.reddit.com"],
  "security_headers_found": [],
  "archive_fallback": {"available": false},
  "diagnosis": "Blocked by bot wall: Cloudflare... | Server: cloudflare"
}

Who it's for

Anyone building scrapers who spends time debugging blocks
QA engineers testing whether a site is reachable from different networks
Researchers collecting data from tricky sites
Developers who want a cheap, honest health check for a URL

Input

{"urls": ["https://example.com", "https://reddit.com"]}

Output per URL

Field	Description
`status_code`	HTTP status or 0 if unreachable
`blocked`	True if a bot wall was detected
`blocked_reason`	The detected block pattern
`source`	`direct`, `archive.org`, or TLS fallback
`response_time_ms`	Server response time
`server`	Server header from response
`security_headers_found`	List of detected security headers
`archive_fallback`	Whether archive.org had a copy
`diagnosis`	Plain-English explanation

What it does not do

This is a diagnostic scraper, not a generic crawler. It fetches one URL at a time with detailed metadata.
It does not execute JavaScript. If a site requires JS, it will report the block reason.
It does not maintain sessions, cookies, or proxies.
It is honest about what it finds. If it can't get through, it tells you why.

Pricing

$0.02 per URL diagnosed.

TLS Certificate Monitor

junipr/tls-certificate-monitor

Monitor TLS certificates, expiry, chain health, issuer changes, and HTTPS readiness.

junipr

SSL/TLS Certificate Scraper

taroyamada/ssl-certificate-monitor

Scan thousands of websites for expiring TLS certs, extract fingerprint hashes, and run Google Web Risk malware checks without rendering a browser.

naoki anzai

SSL Certificate Inspector - TLS Audit & Expiry Monitor

santamaria-automations/ssl-certificate-inspector

Inspect SSL/TLS certificates for any domain: chain validation, expiry dates, cipher suites, TLS versions, subject alt names, issuer details, and security scoring. Export data, run via API, schedule and monitor runs, or integrate with other tools.

NanoScrape

Instagram Hashtag Scraper

crawlerbros/instagram-hashtag-scraper

Extract posts from Instagram hashtags with complete metadata including engagement metrics, captions, media info, and author details. Features smart pagination, anti-block strategies, and multiple scraping approaches.

Crawler Bros

5.0

Bulk SSL Certificate Checker — Expiry, Issuer & TLS

logiover/bulk-ssl-certificate-checker

Check SSL/TLS certificates for thousands of domains at once — expiry date, issuer, SANs, days remaining, TLS versions supported and chain validation. No API key. Export to CSV or JSON. Ideal for security audits, certificate monitoring and compliance.

Logiover

SSL Certificate Checker API - TLS Expiry Scanner

pink_comic/ssl-certificate-checker

Bulk SSL/TLS certificate checker API for expiry monitoring, HTTPS audits, issuer/SAN extraction, chain validation, TLS version, and cipher-suite evidence. Scan domains for expiring, expired, or misconfigured certificates before outages or compliance reviews.

Ava Torres

SSL/TLS Certificate Expiry Monitor

gp005/tls-cert-monitor

Bulk-check TLS certificates for any HTTPS hosts: days-to-expiry, expiry alerts, and issuer/SAN/fingerprint change detection. Pure protocol, no scraping.

Geo

ICD-10 Codes Scraper | Medical Diagnosis Lookup

parseforge/icd10-codes-scraper

Extract ICD-10-CM diagnosis codes with code, short and long descriptions, chapter, block, and parent hierarchy. Filter by chapter or keyword. Built for medical billing software, healthcare apps, clinical research, and AI training datasets in the health sector.