Comment System Detector

What does Comment System Detector do?

Comment System Detector finds comment systems on websites and reports whether visitors can currently comment. Submit one or more public URLs, including homepages, and the Actor can follow eligible internal links to look for article or post pages within your configured crawl limits.

Each unique supplied site produces one final structured record containing the detected provider, commenting state, form availability, confidence score, and supporting evidence. Supported providers include WordPress native comments, Disqus, Facebook Comments, Hyvor Talk, Commento, native HTML comment forms, and custom comment widgets.

Use the Actor from Apify Console or through the Apify API. Results can be scheduled, monitored, downloaded, or connected to other Apify integrations.

Why use Comment System Detector?

• Publisher discovery: find websites with active reader discussions.
• Technology research: identify which comment provider a site uses.
• Website audits: locate open, closed, unavailable, or login-required comment areas.
• Migration planning: inventory comment systems before moving content platforms.
• Lead enrichment: add community and engagement signals to website datasets.
• Ongoing monitoring: schedule repeated checks to notice changes in commenting availability.

The Actor returns evidence with every positive detection, making results easier to review and filter by confidence.

How to use Comment System Detector

1. Open the Actor in Apify Console.
2. Add one or more pages under Start URLs.
3. Set Maximum crawl depth to 0 for only the supplied pages, or use 1 or more to follow internal links.
4. Set a total page limit with Maximum pages per run.
5. Click Start.
6. Open the Output tab to review or download one final result per unique site.

The Actor is read-only. It may scroll to reveal lazy-loaded comment widgets, but it does not post comments, submit forms, or sign in to accounts.

Supplied URLs sharing the same hostname are grouped into one site target. If a comment system is detected on any page, the Actor immediately finalizes that site, skips navigation for its remaining queued URLs, and continues other sites independently. Intermediate page analyses are not written to the Dataset.

Input

The following input analyzes Apify's homepage and follows one internal-link level:

{
    "startUrls": [
        { "url": "https://apify.com" }
    ],
    "maxCrawlDepth": 1,
    "maxRequestsPerCrawl": 150,
    "maxLinksPerPage": 50,
    "proxyConfiguration": {
        "useApifyProxy": false
    }
}

Field	Type	Default	Description
startUrls	array	https://apify.com/	One or more public HTTP or HTTPS URLs, without a fixed application-level maximum.
maxCrawlDepth	integer	1	Internal-link levels followed from each start URL. 0 analyzes only supplied URLs.
maxRequestsPerCrawl	integer	150	Maximum total pages analyzed across all start URLs; the default automatically expands to cover every unique start URL and must be at least 1.
maxLinksPerPage	integer	50	Maximum eligible internal links queued from each page; must be at least 1.
proxyConfiguration	object	disabled	Optional Apify Proxy or custom rotating proxy configuration.

Internal crawling stays on the start website, respects robots.txt, removes common tracking parameters, skips common account and administration paths, and avoids non-HTML files. Article-like links are prioritized when the crawl budget is limited.

Link discovery for each site ends as soon as a comment system is detected, even when a higher maximum depth was configured.

Using Apify Proxy

Enable Apify Proxy from the input editor, then optionally choose proxy groups and a country available to your Apify account:

{
    "useApifyProxy": true,
    "apifyProxyGroups": ["RESIDENTIAL"],
    "apifyProxyCountry": "US"
}

Apify Proxy availability, groups, geographic options, and usage charges depend on your Apify plan. If you provide a subdivision, also select its country.

Using custom proxies

To rotate your own proxies, keep Apify Proxy disabled and provide one or more complete proxy URLs:

{
    "useApifyProxy": false,
    "proxyUrls": [
        "http://username:password@proxy.example:8000",
        "socks5://proxy.example:1080"
    ]
}

Do not enable Apify Proxy and custom proxy URLs at the same time. Proxy configuration and credentials are not included in Dataset output.

Output

The Actor creates exactly one Dataset item per unique hostname, treating www.apify.com and apify.com as the same site. The first supplied URL for that hostname becomes startUrl. A detected result is written as soon as the comment system is found. An unknown or failed result is written as soon as that site's own request queue is exhausted; only a global crawl limit can defer it until the run ends.

{
    "status": "unknown",
    "startUrl": "https://apify.com/",
    "url": "https://apify.com/",
    "depth": 0,
    "commentSystem": {
        "detected": false,
        "provider": {
            "id": "unknown",
            "name": null,
            "type": "unknown"
        },
        "state": "unknown",
        "form": {
            "available": false,
            "loginRequired": false
        },
        "confidence": 0
    },
    "evidence": [],
    "meta": {
        "rendered": true,
        "pagesAnalyzed": 1,
        "failedRequests": 0,
        "completionReason": "crawl_finished_without_detection"
    }
}

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. Results are also available through the Apify API.

Data table

Field	Description
status	Final outcome: detected, unknown, or failed.
startUrl	First supplied URL representing the unique site.
url	Final analyzed URL after redirects.
depth	Link distance from the start URL; start pages use 0.
commentSystem.detected	Whether a credible comment system was found.
commentSystem.provider	Provider ID, display name, and provider type.
commentSystem.state	open, login_required, closed, unavailable, or unknown.
commentSystem.form	Whether commenting is available and whether login is required.
commentSystem.confidence	Detection confidence from 0 to 1.
evidence	Signals supporting the selected provider and state.
meta.rendered	Whether at least one page was successfully browser-rendered.
meta.pagesAnalyzed	Number of successfully analyzed pages for this site.
meta.failedRequests	Number of requests that exhausted all retries.
meta.completionReason	Why the final result was emitted.

How much does it cost to detect website comment systems?

Cost mainly depends on the number of pages rendered, page load time, and selected Actor memory. A run with depth 0 and a few start URLs is usually cheaper than a homepage crawl that follows dozens of internal links. Use maxRequestsPerCrawl as the primary cost control. Proxy traffic may add separate charges depending on the selected Apify Proxy group or your external proxy provider.

You can test small runs using the usage included with the current Apify free plan. For recurring or larger crawls, review the average cost of representative runs before scheduling at scale.

Tips and advanced options

• Start with depth 1 when submitting a homepage; this often reaches article pages without exploring too broadly.
• Increase depth gradually and keep a firm maxRequestsPerCrawl limit.
• Add article or category URLs directly when you already know where content is located.
• Use confidence and evidence when deciding which detections require manual review.
• A result with state unknown can still identify a provider when its interactive controls are blocked or delayed.
• Submit multiple start URLs in one run when auditing several independent websites.
• Use proxies only when needed; they can improve access reliability but may increase cost and latency.

FAQ, disclaimers, and support

Does the Actor crawl external websites linked from a page?

No. Only eligible internal links associated with each start website are considered.

Is every supplied URL analyzed?

URLs on the same hostname are grouped. They remain seed candidates until one page detects a comment system; any remaining queued URLs for that site are then skipped. URLs can also be skipped when blocked by robots.txt or when a configured global limit is reached.

What happens after a comment system is found?

The Actor immediately writes the positive result, stops following deeper links, and skips navigation for remaining queued URLs on that site. Other sites continue independently.

Are pages without a detection included in the Dataset?

No intermediate page records are stored. The Dataset contains one final item per unique site: the detected page, or an unknown/failed result emitted when that site's work finishes.

Does the Actor submit comments or log in?

No. It only observes rendered pages and never submits comment or authentication forms.

Do I need a proxy?

Not always. Start without a proxy for publicly accessible pages. Enable Apify Proxy or custom proxies when websites block direct data-center traffic, require a specific location, or need IP rotation.

Why was a page not analyzed?

The page may exceed the configured depth or request limits, be blocked by robots.txt, require authentication, fail to load, or point to a skipped non-HTML resource.

Can every custom comment system be identified?

No detector can recognize every private or heavily customized implementation. Unknown or low-confidence results may require manual review.

Automated website access may be governed by website terms, privacy requirements, robots.txt, and applicable law. Only analyze websites you are authorized to access. Use the Actor's Issues tab for bug reports, provider requests, or support. Custom configurations and tailored data extraction solutions are available on request.