Handle Moderation Actions

Actions represent any action you can perform on Items. Some common examples include Delete, Ban, Mute, Send to Moderator, Approve, etc.

In order to take action on an Item on your behalf, we need to know what types of actions we can take, and how to take them. For example, if you want to create rules that automatically delete hateful content, or if you want moderators to be able to manually delete hateful content, Cove needs to be able to invoke your Delete action programmatically (i.e. through a POST request).

How to build an Action API Endpoint

For every Action you define in Cove, you have to expose the action through a public-facing API endpoint. Whenever an automated rule (or a moderator) determines that some Item should receive an Action, we will send a POST request to the Action's API endpoint. When your server receives that POST request, your code should actually perform the corresponding action.

Here's an example of the body of a POST request that we'd send to your API when trying to perform an Action:

{
  'item': {
    'id': 'abc123',
    'typeId': 'def456'
  },
  'action': {
    'id': 'mno654'
  },
  'policies': [
    {
      id: 'ghi789',
      name: 'Hate Speech',
      penalty: 'NONE'
    },
    {
      id: 'jkl321',
      name: 'Graphic Violence',
      penalty: 'HIGH'
    }
  ],
  'rules': [
    {
      id: '061ba7f64db',
      name: 'Composite Hate Speech'
    },
    {
      id: '5a0c37041ac',
      name: 'Violence in Text Content'
    }
   ],
  'custom': {
    // ... any custom parameters that you configured in the Actions Dashboard
  }
}

The body of the request will contain the following fields:

Property	Type	Always Present?	Description
`item`	`Item`	Always Present	The Item that should receive this Action
`action`	`Action`	Always Present	We provide some information about the Action being triggered in case it's helpful.
`policies`	`Array<Policy>`	Always Present	Actions can be associated with Policies. For example, a piece of content can be deleted for violating a Spam Policy, or for violating a Violent Extremism & Terrorism Policy. Both Actions are "Delete" Actions, so we'll send a POST request to the same "Delete" endpoint, but you may want to know why the content is being deleted, and which exact Policy was violated that led to this deletion, as it may affect how you handle it. Note: Each Action may be associated with more than one Policy (for example, if multiple Rules decided to delete a piece of content for different reasons). So, `policies` is an array of `Policy` objects.
`rules`	`Array<Rule>`	Not Always Present	Actions can be triggered in a few different ways: by Rules, by Moderators, through our Bulk Actioning tool, etc. If this Action was triggered by a Rule (or multiple Rules), we will include those Rules in this `rules` array. If the Action was triggered by some other means (for example, by a Moderator), the `rules` array will be empty.
`custom`	`Object`	Not Always Present	If you would like us to include any custom parameters in the request we send to your Action endpoints, you can add those custom parameters to each Action in the Cove Action Form. These can be configured in the "Body" section of the form.
`actorEmail`	`String`	Not Always Present	When using Cove's Manual Review Tool to review a report, the email address of the Cove user or moderator who decides to take action on a piece of content will be included in this field. If an action is taken by an automated rule or AI model, this field will be omitted.

Item schema:

Property	Type	Description
`id`	`String`	Your unique identifier for this Item. You can use this ID to determine on which Item to evoke the Action.
`typeId`	`String`	The ID of the Item Type that corresponds to the Item which should receive the Action. This will exactly match the ID of one of the Item Types that you defined in the Item Types Dashboard. The IDs of each Item Type can be found in that dashboard.

Policy schema:

Property	Type	Description
`id`	`String`	This is Cove's unique identifier for this Policy. You can create, update, and view your company's Policies in the Policies Dashboard. If you click the "Edit" button on a particular Policy, you'll be able to see its `id`.
`name`	`String`	This is the name of the Policy that you created in Cove's dashboard. Important Note: We include this in case you'd like to display the Policy name in an internal UI or in your product's UI, but we do not recommend building any logic in your code that relies on these names to stay consistent. We recommend using the `id` field for that. Policy names are mutable, so if another member of your organization changes the name of a Policy in the Cove dashboard, it could break any code that relies on the Policy's previous name.
`penalty`	`String`	This is a measure of how severe the Policy is. The possible values for the `penalty` property are: `NONE`, `LOW`, `MEDIUM`, `HIGH`, `SEVERE`. For example, the Spam Policy might have a `LOW` penalty, whereas the Child Sexual Abuse Material (CSAM) Policy might have a `SEVERE` penalty.

Rule schema

Property	Type	Description
`id`	`String`	This is Cove's unique identifier for a Rule that triggered this Action. You can view your company's Rules in the Rules Dashboard. If you click the "Edit" button on a particular Rule, you'll be able to see its `id` at the end of the URL.
`name`	`String`	This is the name of the Rule that triggered this Action. Important Note: We include this in case you'd like to display the Rule name in an internal UI or in your product's UI, but we do not recommend building any logic in your code that relies on these names to stay consistent. We recommend using the `id` field for that. Rule names are mutable, so if another member of your organization changes the name of a Rule in the Cove dashboard, it could break any code that relies on the Rule's previous name.

Action schema

Property	Type	Description
`id`	`String`	This is Cove's unique identifier for this Action. You can create, update, and view your company's Actions in the Actions Dashboard. If you click the "Edit" button on a particular Action, you'll be able to see its `id` at the end of the URL.

Authentication and Custom Parameters

We strongly recommend that you authenticate every HTTP request that your Action endpoints receive. Without proper authentication, any developer could send arbitrary Action requests and mutate your data.

In order to authenticate that a POST request is indeed coming from Cove, we allow you to add custom HTTP headers that we'll include in every request we send to your Action endpoint. You can add those headers in the "Headers" section of the Action Form.

You may also add any parameters you'd like for us to include in the body of our POST requests. These can be added to the "Body" section of the Action Form. See the custom parameter in the code snippet above for how to use them.