WorqHat
AI Nodes

Content Moderation

Category
AI
Node Type
Analysis

Overview

The Content Moderation Node automatically detects and flags inappropriate or unsafe content using AI. It can analyze both text and image inputs to determine whether they contain offensive, harmful, or restricted material before allowing the workflow to proceed.

This node is essential for ensuring that user-generated content, uploaded files, or messages meet your platform’s safety and compliance requirements.

Description

Moderate content using AI.

Use the Content Moderation Node to evaluate content for categories such as sexual content, harassment, hate speech, violence, and more. It supports two main types of moderation:

Text Moderation

Analyze written content such as messages, comments, or posts for unsafe language.

Image Moderation

Analyze uploaded or generated images for inappropriate visual content.

The node returns a detailed breakdown of flagged categories and confidence scores, allowing you to take automated actions based on the results.

Input Parameters

The Content Moderation node accepts flat key-value pairs that specify the content to analyze and the type of moderation to perform.

attachmentsstringRequired
Comma-separated list of file IDs or variable references to the content that needs moderation. Required for image moderation.
Example
"file1.jpg,file2.png" or "{{nodeId.output.image}}"
moderationTypestringOptional
Defines the type of moderation to perform. Supported values: 'text-moderation' or 'image-moderation'. If omitted, the node attempts to detect the type.
Example
"text-moderation"
moderationTextstringOptional
The text string to be analyzed for moderation. Required for text moderation.
Example
"This is a test message."
Instructions

Provide all parameters as flat key-value pairs.

For multiple files, use comma-separated values. Access input values dynamically within the workflow using {{nodeId.input.<key>}}.

Output Parameters

After execution, the Content Moderation node returns the AI’s analysis of the submitted content along with moderation details and confidence scores.

flaggedbooleanOptional
Indicates whether the content was flagged for any policy violations. Returns true if one or more categories were triggered.
Example
true
flaggedCategoriesstringOptional
A comma-separated list of the categories that were flagged during moderation.
Example
"violence,hate"
processingTimestringOptional
The total time taken by the AI to analyze the content, returned in ISO timestamp format.
Example
"2025-10-27T10:45:12Z"
processingIdstringOptional
A unique identifier assigned to the moderation request.
Example
"modr-3438"
categories.sexualnumberOptional
Confidence score (0-1) representing the likelihood of sexual or adult content.
Example
0.05
categories.harassmentnumberOptional
Confidence score (0-1) indicating potential harassment or bullying language.
Example
0.95
categories.hatenumberOptional
Confidence score (0-1) for hate speech or discriminatory expressions.
Example
0.10
categories.illicitnumberOptional
Confidence score (0-1) showing the presence of illegal, restricted, or drug-related content.
Example
0.01
categories.self-harmnumberOptional
Confidence score (0-1) for self-harm, suicide, or unsafe behavior mentions.
Example
0.00
categories.violencenumberOptional
Confidence score (0-1) measuring the presence of violent or graphic content.
Example
0.80
Accessing Data

Access output results using variable references:

{{nodeId.output.flagged}}               → true / false
{{nodeId.output.flaggedCategories}}     → "violence, hate"
{{nodeId.output.categories.sexual}}     → 0.05

Output Type

Output Type: text-moderation/image-moderation

This identifying string indicates the node handles both text-based and image-based moderation tasks. Do not modify this value.

Example Usage

Example 1: Text Moderation

Analyze a text message for harassment.

{  "moderationType": "text-moderation",  "moderationText": "I hate you!"}
{  "flagged": true,  "flaggedCategories": "harassment",  "categories": {    "harassment": 0.98,    "sexual": 0.01,    "hate": 0.05,    "illicit": 0.00,    "self-harm": 0.00,    "violence": 0.02  },  "processingTime": "2025-10-27T10:30:45Z",  "processingId": "modr-3438"}

Example 2: Image Moderation

Analyze an uploaded image for violence.

{  "moderationType": "image-moderation",  "attachments": "file123.jpg"}
{  "flagged": false,  "flaggedCategories": "",  "categories": {    "violence": 0.1,    "sexual": 0.05,    "harassment": 0.01,    "hate": 0.00,    "illicit": 0.00,    "self-harm": 0.00  },  "processingTime": "2025-10-27T10:46:55Z",  "processingId": "modr-9921"}

How to Use in a No-Code Workflow

1

Add the Node

Drag and drop the Content Moderation Node into your workflow canvas.

2

Choose Input Type

  • Use moderationText for moderating text messages or comments.
  • Use attachments for moderating images or file uploads.
3

Set Moderation Type

Choose "text-moderation" or "image-moderation" as needed.

4

Connect Inputs

Link the output from a previous node (like file upload or text generation) to the attachments or moderationText fields.

5

Access Outputs

Use variable references to pass results to other nodes, such as conditional checks or notifications.

6

Set Conditions

Create conditional branches (If/Else) in your workflow to stop or flag content automatically if {{nodeId.output.flagged}} is true.

Best Practices

  • Always verify that uploaded files are properly connected before moderation.
  • For text moderation, keep inputs under 5,000 characters for optimal performance.
  • Combine both moderationText and attachments to analyze mixed media submissions.
  • Review flagged outputs manually for high-risk content before taking automated action.
  • Store processingId values for tracking or audit purposes.

Do / Don’t

Do
  • ✔️ Use specific thresholds for category scores (e.g., flag if violence > 0.8).
  • ✔️ Provide clear feedback to users if their content is flagged.
  • ✔️ Use this node to protect your community and brand reputation.
Don’t
  • ❌ Don’t rely solely on the boolean flagged status for nuanced cases; check the scores.
  • ❌ Don’t ignore the processingId when debugging false positives.
  • ❌ Don’t use this node for medical diagnosis or legal judgments.

Common Errors

Missing attachmentsErrorOptional
No file or variable reference was provided for moderation. Add a valid image or file reference.
Missing moderationTextErrorOptional
The text moderation input field was left empty. Provide a valid text string.
Invalid moderationTypeErrorOptional
An incorrect or unsupported moderation type was entered. Use 'text-moderation' or 'image-moderation'.
Empty outputErrorOptional
The AI model returned no response. Retry the workflow or check service availability.
File not accessibleErrorOptional
The referenced image file could not be loaded. Verify file existence and permissions.

Example Workflow Integration

Use Case 1: Chat Safety

Monitor a chat application for harassment.

  1. Trigger Node: Receives a new chat message.
  2. Content Moderation Node: Checks the message text.
  3. Logic Node: If flagged is true, block the message; otherwise, send it.

Workflow Data Flow:

{{chatTrigger.output.message}}      →  {{moderation.input.moderationText}}
{{moderation.output.flagged}}       →  {{logicNode.input.condition}}

Use Case 2: User Uploads

Ensure uploaded profile pictures are appropriate.

  1. File Upload Node: User uploads an image.
  2. Content Moderation Node: Checks the image for sexual or violent content.
  3. Notification Node: Alerts admin if the image is flagged.

Workflow Data Flow:

{{fileUpload.output.fileUrl}}       →  {{moderation.input.attachments}}
{{moderation.output.flagged}}       →  {{notification.input.trigger}}