Sherlock Scraper

Find public username profiles across hundreds of websites using the open-source Sherlock site catalog.

Sherlock Scraper is an Apify actor for OSINT, brand protection, fraud review, recruiting enrichment, and username availability monitoring. Give it one or more usernames and it checks profile URL patterns from the Sherlock catalog with conservative HTTP rules. The output tells you which checks were found, not found, or uncertain.

What does Sherlock Scraper do?

Sherlock Scraper checks whether a username appears to have a public profile on many social networks, developer communities, forums, creator platforms, and directories.

It returns one dataset row per username and site.

Each row includes the checked URL, HTTP status, classification rule, response time, and timestamp.

The actor is best-effort by design. Some websites block automated checks or return ambiguous redirects. Those rows are marked unknown rather than guessed as found or not found.

Who is it for?

OSINT researchers

Use it to quickly map where a handle might exist before deeper manual investigation.

Fraud and risk teams

Check whether a claimed username has a broad cross-site footprint or appears only on one suspicious source.

Brand protection teams

Monitor usernames that match your brand, executive names, campaign names, or product handles.

Recruiting and sales enrichment teams

Enrich a known handle with public profile URLs from developer and community sites.

Creators and agencies

Check availability patterns before launching a new brand or creator identity.

Why use this actor?

• 🔎 Batch username checks in one Apify run.
• 🧭 Uses the public Sherlock catalog, not a tiny hand-built list.
• 🧾 Returns evidence fields, not just a yes/no answer.
• 🛡️ Marks uncertain checks as unknown to reduce false positives.
• ⚙️ Lets you limit sites, concurrency, and timeouts.
• 📦 Exports cleanly to JSON, CSV, Excel, Google Sheets, and API workflows.

What data can you extract?

Field	Description
username	Normalized queried username
site	Sherlock catalog site name
url	Public profile URL checked
exists	true, false, or null for unknown
status	found, not_found, or unknown
statusCode	HTTP status code when available
matchedBy	Rule used for classification
responseTimeMs	Request duration in milliseconds
checkedAt	ISO timestamp for the check
error	Request error details when a site could not be checked

How much does it cost to check usernames with Sherlock?

This actor uses pay-per-event pricing.

You pay a small start fee per run and a per-result fee for each username/site row saved to the dataset.

Keep the first test cheap by using the default prefill or a small maxSites value.

For broad OSINT scans, increase maxSites after confirming that the output format fits your workflow.

How to use Sherlock Scraper

1. Open the actor on Apify.
2. Add one or more usernames in usernames.
3. Optionally add site filters such as GitHub, Reddit, Medium, or YouTube.
4. Choose maxSites for the breadth of the scan.
5. Keep includeNotFound enabled when you need a full audit trail.
6. Disable it or set onlyFound when you only want likely profile URLs.
7. Run the actor.
8. Export the dataset or consume it through the Apify API.

Input options

usernames

Required list of usernames or handles.

Leading @ characters are removed automatically.

Examples:

• torvalds
• github
• apify

sites

Optional list of Sherlock site names or partial names.

If empty, the actor checks the first matching sites up to maxSites.

Examples:

• GitHub
• Reddit
• Telegram
• YouTube
• Medium

maxSites

Maximum catalog sites checked per username.

Use a low number for testing and a higher number for broader monitoring.

includeNotFound

When enabled, you get every attempted check.

When disabled, only found rows are saved.

onlyFound

Compatibility shortcut for found-only exports.

If this is true, it overrides includeNotFound.

includeNsfw

Include sites marked NSFW by the upstream Sherlock catalog.

Default is false.

concurrency

Number of HTTP checks running at the same time.

The default is conservative to avoid noisy target rate limits.

timeoutSecs

Maximum time to wait for each site before returning unknown.

Example input

{
  "usernames": ["torvalds", "github"],
  "sites": ["GitHub", "Reddit", "Medium", "YouTube", "Telegram"],
  "maxSites": 25,
  "includeNotFound": true,
  "concurrency": 5,
  "timeoutSecs": 12
}

Example output

{
  "username": "torvalds",
  "site": "GitHub",
  "url": "https://www.github.com/torvalds",
  "exists": true,
  "status": "found",
  "statusCode": 200,
  "matchedBy": "http_status",
  "responseTimeMs": 421,
  "checkedAt": "2026-06-25T00:00:00.000Z",
  "error": null
}

Understanding unknown results

unknown does not mean the username is absent.

It means the site did not provide a reliable public signal during this run.

Common reasons include:

• temporary target outage,
• DNS or TLS failure,
• anti-bot response,
• rate limit,
• ambiguous redirect,
• unusual status code.

For investigations, sort by status and manually review important unknown URLs.

Tips for accurate scans

• Start with a small list of high-value sites.
• Keep concurrency moderate for broad scans.
• Re-run important unknown rows later.
• Treat found rows as leads, not legal identity proof.
• Keep not-found rows when you need audit evidence.
• Use sites filters for brand monitoring workflows.

Integrations

Google Sheets

Export the dataset to CSV or use the Apify Google Sheets integration to append found profiles to a monitoring sheet.

Webhooks

Trigger a webhook when a run finishes and send found profile rows into Slack, Make, Zapier, or your case-management tool.

CRM enrichment

Use usernames from CRM records and append public profile URL candidates back to the contact record.

Brand monitoring

Schedule daily or weekly runs for important handles and compare the resulting dataset with a previous run.

API usage

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('automation-lab/sherlock-scraper').call({
  usernames: ['torvalds'],
  sites: ['GitHub', 'Reddit'],
  maxSites: 10
});
console.log(run.defaultDatasetId);

Python

from apify_client import ApifyClient

client = ApifyClient('YOUR_APIFY_TOKEN')
run = client.actor('automation-lab/sherlock-scraper').call(run_input={
    'usernames': ['torvalds'],
    'sites': ['GitHub', 'Reddit'],
    'maxSites': 10,
})
print(run['defaultDatasetId'])

cURL

curl -X POST 'https://api.apify.com/v2/acts/automation-lab~sherlock-scraper/runs?token=YOUR_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"usernames":["torvalds"],"sites":["GitHub","Reddit"],"maxSites":10}'

MCP usage

Use the Apify MCP server with:

https://mcp.apify.com?tools=automation-lab/sherlock-scraper

Claude Code setup:

$claude mcp add apify-sherlock "https://mcp.apify.com?tools=automation-lab/sherlock-scraper"

Claude Desktop JSON config:

{
  "mcpServers": {
    "apify-sherlock": {
      "url": "https://mcp.apify.com?tools=automation-lab/sherlock-scraper"
    }
  }
}

Example prompts:

• "Check whether the username torvalds exists on GitHub, Reddit, and Medium."
• "Run a weekly Sherlock username monitor for our brand handles."
• "Return only found public profile URLs for these candidate usernames."

Scheduling

You can schedule this actor in Apify for recurring monitoring.

Useful schedules include:

• daily checks for brand impersonation,
• weekly OSINT enrichment for a watchlist,
• monthly username availability checks before product launches.

Data quality notes

The actor uses public HTTP signals.

No automated username checker can guarantee identity ownership.

A found profile URL only means that a public page likely exists at that username pattern.

Review high-risk results manually before making decisions.

Legality and ethical use

Only use this actor for lawful workflows and public information.

Do not use it for harassment, doxxing, credential attacks, or violating platform terms.

Respect local privacy laws and your organization's compliance requirements.

Attribution

This actor consumes the public site catalog from the open-source Sherlock project.

Sherlock is an MIT-licensed OSINT project.

This actor is not affiliated with or endorsed by the Sherlock project.

FAQ

Is Sherlock Scraper affiliated with the Sherlock project?

No. This actor uses the public Sherlock site catalog and is not affiliated with or endorsed by the upstream project.

Can this prove who owns a username?

No. It finds public profile URL candidates. Treat results as leads for review, not identity proof.

Troubleshooting

Why are many rows unknown?

Some sites block automated checks, return ambiguous redirects, or time out. Lower concurrency, narrow the sites list, or re-run later.

Why did a known profile return not found?

Sites change profile URLs and anti-bot rules. Check the returned url, statusCode, and matchedBy fields, then retry the specific site.

Why did I get no rows?

If onlyFound is true and no profiles are classified as found, the dataset can be empty. Enable includeNotFound for a full audit trail.

Related scrapers

You may also find these automation-lab actors useful:

• https://apify.com/automation-lab/domain-availability-checker
• https://apify.com/automation-lab/website-uptime-checker
• https://apify.com/automation-lab/bulk-url-status-checker
• https://apify.com/automation-lab/robots-sitemap-analyzer
• https://apify.com/automation-lab/structured-data-extractor

Changelog

Initial version: batch username checking across the Sherlock public catalog with found, not-found, and unknown states.