Toxic Comment Detector: Content Moderation API

This Actor gives scores to English text for toxicity, insults, profanity, threats, identity attacks and severe toxicity. Each attribute gets a probability from 0 to 1. Each text also gets a flagged value (true or false). Use it for comments, posts, reviews, chat messages, support tickets and LLM output.

• Fast and deterministic. The Actor uses a dedicated classifier, not an LLM. Thus, the same text always gets the same score.
• A maximum of 10,000 texts for each run. The results go into a dataset. You can download the dataset as JSON, CSV or Excel.
• Ready for agents. AI agents can use the Actor through the Apify API or MCP. They pay for each use.
• Long texts. The model divides a long text into chunks that overlap. Thus, it examines the end of the text too.
• Limit for long texts. In a very long, polite text, one insult has a small effect on the score. For example, one insult at the end of a 20 KB text gets 0.47. For documents, divide them into paragraphs, or use a lower threshold (for example, 0.3).
• Your data stays yours. The Actor uses your texts only to calculate the scores that you get.

How to use the Toxic Comment Detector

1. Click Try for free.
2. In Texts to analyze, add your texts. Add one comment, post or review in each entry.
3. Optional: in Attributes, select the scores that you need.
4. Optional: change the Flag threshold. The default is 0.5.
5. Click Start.
6. When the run is complete, open the Output tab.
7. Download the results as JSON, CSV or Excel, or get them through the API.

Input

{
  "texts": ["Thanks for sharing, this was really helpful!", "You are an idiot and nobody wants you here."],
  "attributes": ["TOXICITY", "INSULT"],
  "threshold": "0.5"
}

Field	Necessary	Description
texts	yes	1 to 10,000 English texts. The Actor ignores empty entries
attributes	no	One or more of TOXICITY, SEVERE_TOXICITY, INSULT, PROFANITY, THREAT, IDENTITY_ATTACK. If this field is empty, you get all six
threshold	no	0 to 1. The default is 0.5. If the TOXICITY score is equal to or more than this value, flagged is true

Output

The dataset has one item for each text. The items are in the same sequence as the input:

{"text": "You are an idiot and nobody wants you here.", "flagged": true,
 "scores": {"TOXICITY": 0.987509, "INSULT": 0.947843}, "billed_units": 1}

If a text is possibly not in English, its item has a warning field. The model supports English only. Thus, the scores for that text are not reliable.

Use the Actor from code

from apify_client import ApifyClient
client = ApifyClient("<APIFY_TOKEN>")
run = client.actor("koyourmoon/toxicity-detector").call(run_input={"texts": ["you are pathetic", "nice work!"]})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["flagged"], item["scores"]["TOXICITY"], item["text"])

Prices

Event	Price
Text unit (each started 1,000 characters of a text)	$0.50 for 1,000 units ($0.0005 for each unit)
Actor start	$0.002 for each GB of memory (about $0.004 for each run at the default 2 GB)

Most comments, posts and reviews have less than 1,000 characters. Thus, most texts cost 1 unit ($0.0005). A longer text costs more units. For example, a text with 2,500 characters costs 3 units.

Each result shows billed_units. Thus, you can always see what you paid for. Azure Content Moderator and Google Cloud Natural Language use the same method: they charge for the length of the text.

You do not pay platform usage fees in addition to these prices. There is no subscription and no minimum. You can set a maximum cost for the run. The run stops before the cost goes above this limit.

Speed and limits

• At the default 2 GB of memory, the Actor examines about 10 short comments each second.
• 1,000 short comments take about 2 minutes. This time includes the start of the run.
• The maximum length of each text is 20,480 bytes.
• A long text takes much more time. One 20 KB text takes about the same time as 250 short comments.
• The Actor saves the results in small groups during the run, and charges for each group. If the run stops early (timeout, abort or cost limit), you keep all results that the Actor completed. You pay only for these results.
• The default run timeout is 1 hour. For very large jobs, divide the job into many runs, or increase the timeout.

Applications

• Comment moderation: find toxic comments before you publish them.
• Guardrails for agents: examine the text that an AI agent will post or send.
• Community analytics: measure the toxicity of a subreddit, a YouTube comment section or a set of reviews. Use this Actor together with a comment scraper on Apify.
• Clean datasets: remove abusive samples from training data.

Integrations

• Apify API and clients: start a run and get the dataset from Python, JavaScript or any HTTP client.
• MCP: AI agents can find and run the Actor through the Apify MCP server.
• Make, Zapier and n8n: send new comments to the Actor, then send the flagged items to Slack, email or your database.
• Scrapers on Apify: run a comment scraper or a review scraper first. Then send its texts to this Actor.

Frequently asked questions

Which languages does the Actor support? The Actor supports English only. If a text is possibly not in English, its result has a warning field.

How much does 1,000 comments cost? Most comments have less than 1,000 characters. Thus, 1,000 comments usually cost $0.50, plus about $0.004 for the start of the run.

Is the score the same as a Google Perspective API score? No. The scores are similar, but they are not equal. Test your threshold on a sample of your own texts before you use the Actor in production.

Does the Actor keep my texts? The Actor saves the texts and the scores only in the dataset of your run, in your Apify account. You can delete the dataset at any time. Apify also deletes unnamed datasets after the data retention period of your plan.

What happens if my run stops before the end? You keep all results that the Actor completed. You pay only for these results.

How do I report a problem or request a feature? Open an issue on the Issues tab of this Actor. We reply in 14 days or less.

About the model

The classifier is the open-source Detoxify "original" model (Apache-2.0). Detoxify learned from the Jigsaw Toxic Comment dataset. The data for the Google Perspective API comes from the same family of datasets.

Do you need an HTTP endpoint that is compatible with Perspective? Use Perspective API Alternative.