POST
/
api
/
ai
/
moderation
Text Moderation
curl --request POST \
  --url https://api.worqhat.com/api/ai/moderation \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '{
  "text_content": "This is a sample text to be moderated."
}'
{
  "data": {
    "sexual": "0.016973",
    "hate": "0.001243",
    "harassment": "0.012784",
    "self-harm": "0.000040",
    "sexual/minors": "0.000049",
    "hate/threatening": "0.000002",
    "violence/graphic": "0.000016",
    "self-harm/intent": "0.000059",
    "self-harm/instructions": "0.000007",
    "harassment/threatening": "0.000002",
    "violence": "0.000333"
  },
  "processing_time": 888,
  "processing_id": "4b139bfe-1f71-4345-b539-fa6c1700f166",
  "processing_count": 3
}

​
Text and Content Moderation AI

Content Moderation AI is a powerful tool that can be used to detect and filter out inappropriate content from your website or app.

Introducing the new Moderation AI endpoint by WorqHat, powered by the advanced AiCon V2 model. This powerful tool is designed to support developers in safeguarding their applications by offering fast and accurate content moderation.

With Moderation AI, developers can ensure that their platforms remain free from inappropriate content, such as sexual, violent, hateful, and self-harm promoting material that violates WorqHat’s content policy. Leveraging LLM-based classifiers, the system efficiently identifies and filters out such content, providing a secure environment for users.

By incorporating AI into the moderation process, developers can enhance human supervision, boosting their confidence in the content displayed on their platforms. This proactive approach reduces the risk of inadvertently showcasing inappropriate content and upholds the integrity of your product.

The versatility of Moderation AI allows it to be seamlessly integrated into various applications, including those that handle sensitive topics like education. WorqHat empowers developers to maintain a safe and inclusive digital space, ensuring a positive user experience for all.

​
How does it work?

WorqHat’s Moderation AI, powered by the AiCon V2 model, employs a deep learning algorithm to examine text inputs and classify them based on their content, specifically targeting sexual, hateful, violent, or self-harm promoting material. These categories align with WorqHat’s strict content policy, which prohibits such content from being displayed on their platforms.

When a text is submitted to the Moderation API endpoint, the algorithm meticulously analyzes it, assigning probability scores for each of the four content categories. If the input contains content that is deemed inappropriate or harmful, the API promptly returns an error message to the user, preventing the display of such content.

The AiCon V2 model was trained on an extensive dataset comprising examples of inappropriate content, enabling it to accurately detect problematic content across various contexts. This empowers WorqHat’s Moderation AI to be employed in sensitive applications, including education and social media platforms, where ensuring user safety and well-being is of paramount importance.

By offering access to this endpoint, WorqHat is actively supporting developers in safeguarding their applications against potential misuse, fostering responsible AI usage in sensitive settings.

​
Use Cases

  • Transcription: Speech to text AI can be used to automatically transcribe audio or video recordings into text format. This can be useful for businesses or individuals who need to transcribe interviews, meetings, or other audio content.
  • Closed captioning: Speech to text AI can be used to generate closed captions for videos, making them accessible to people with hearing impairments. This is important for ensuring that content is accessible to everyone, regardless of their abilities.
  • Voice search: Speech to text AI can be used to power voice search functionality in search engines or other applications. This allows users to search for information using their voice, making the process more natural and intuitive.
  • Customer service: Speech to text AI can be used to transcribe customer service calls, allowing businesses to analyze the conversations and identify areas for improvement. It can also be used to generate transcripts of customer service chats or emails.
  • Language learning: Speech to text AI can be used to transcribe and analyze language conversations, helping language learners to improve their listening and comprehension skills. It can also be used to generate transcripts of language lessons or podcasts.

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

Response

200
application/json

Text moderated successfully

The response is of type object.