Check AI chatbot replies for toxicity before posting

Created by

KOYOURMOON

Actor

Toxic Comment Detector - Content Moderation API

Use this task as a guardrail for AI agents and chatbots. Send the replies that your bot will post, and get toxicity, insult, threat and identity attack scores from 0 to 1. The task uses a strict threshold of 0.3, so it flags borderline replies too. Hold flagged replies for a manual check. Fast, deterministic and no LLM. English only.

Try for free

Toxic Comment Detector - Content Moderation APIkoyourmoon/toxicity-detector

Text

Flagged

Toxicity

Severe toxicity

+4 fields

Input

Texts to analyze(required):Thanks for reaching out! Your refund was approved and will arrive in 3 to 5 business days.+4

Attributes:TOXICITY+3

Flag threshold:0.3

Output fields

Text

Flagged

Toxicity

Severe toxicity

Insult

Profanity

Threat

Identity attack

How it works

Sign up on Apify or log in

The Toxic Comment Detector - Content Moderation API Actor is ready to run with the inputs already set for "Check AI chatbot replies for toxicity before posting". Sign up or log in and you'll land in Apify Console with a copy of this exact task.

Start the "Check AI chatbot replies for toxicity before posting" task

The Toxic Comment Detector - Content Moderation API Actor will start running based on the input automatically. You can also change all the input fields, like texts to analyze, attributes, and more, to match your specific use case.

Receive the "Check AI chatbot replies for toxicity before posting" output

Monitor the progress in real time. You will be notified as soon as your dataset is complete and ready for review. Each result includes data such as text, flagged, toxicity, and more.

Integrate into your workflow

The final output is delivered in JSON, CSV, or Excel format, ready to be plugged into your workflow. You can also run the "Check AI chatbot replies for toxicity before posting" task via API.