Rules
Once you've created your Item Types and Actions, you're ready to create your first Rule!
You can think of a Rule as a way to encode your moderation logic. For example, you might want to enforce the following moderation logic:
"For any Comment or Post, if it contains the slurs 'example_slur_1', 'example_slur_2', or 'example_slur_3', then delete the content and suspend the user".
You can turn that into a Rule that evaluates each incoming Comment and Post against that logic.
A Rule has a few components, and we'll break down the logic above into the relevant components. All of these components are easily configurable in our UI - there is no coding required, and the UI will walk you through it step by step.
-
Item Type(s): When you create a Rule, you need to decide which Item Types it will run on. In the example above, the Item Types for this Rule are Comment and Post, which you can tell from the phrase "For any Comment or Post". If you wanted to run this Rule on Profiles, DMs, or any other Item Type, you would just specify that you want the Rule to run on those additional Item Types.
-
Condition(s): This is where you define the logic for how you want to moderate content. In the example above, the only Condition in this Rule is "if it contains the slurs 'example_slur_1', 'example_slur_2', or 'example_slur_3'". A condition is just a piece of logic that can either be true or false for any given piece of content (i.e. for any Comment, it either contains one of those slurs, or it doesn't). You can create lots of different types of Conditions - some very simple, like this keyword-matching Condition, and some very complex, like a Condition that uses an AI model. You can read more about the types of Signals you can use to construct these Conditions.
Here is an example of some Conditions you might create in the Cove UI:
-
Action(s): This is the Action (or Actions) that will be triggered if the Conditions above are met. In the example above, the Actions for this rule are Delete Content and Suspend User, which you can tell from the phrase then delete the content and suspend the user".
More generally, the structure of a Rule is:
For any Items of type <Choose some Item Types like Comment, Post, Profile> that get created on my platform,
If the following Conditions are true: <Create some Conditions>,
Then, trigger the following Actions: <Choose some Actions like Delete, Suspend, Ban>
Rule Status: Live, Background, Draft
Rules can have four different "statuses":
- Live: Live Rules are actively running on all incoming pieces of content, and they trigger Actions.
- Background: Background Rules are also actively running on all incoming pieces of content, but they are not triggering Actions. Instead, Cove just records which Items were caught by the Rule so you can inspect those Items later. You can this of a "Background" Rule as a test or dry run - you can see what the Rule would catch without risking over-actioning on Items that shouldn't be actioned.
- Draft: Draft Rules are not running - they are merely partially constructed Rules that you can save and come back to later.
- Archived: Archived Rules are also not running, and they sit in a separate "Archived" folder. You can always un-archive a rule, but it can be useful to save deprecated rules for auditing purposes and posterity.
How to Create a Rule
- Navigate to the Rules tab
- Click "Create Rule" in the top right corner
- Enter a name, and optionally a description
- Select which Item Types your Rule should run on. We'll only ever run this Rule if we receive an Item that fits one of the Item Types you selected.
- Construct your Rule's Conditions. Conditions are where the main logic of the Rule lives. The Rule in the image above has two Conditions, combined with the conjunction 'OR':
-
If the Item's text contains "some bad word" or "other bad phrase"
OR
If any of the Item's images scores greater than 0.9 on Amazon Rekognition's Nudity model
- Conditions allow you to customize what types of harm and abuse you'd like to detect, and how strict your enforcement will be. You can choose which Signals to use, and which thresholds to set for those Signals.
- Select the Action(s) that should get triggered if an Item matches the Conditions. We'll automatically trigger those Actions for you.
You can think of Conditions and Actions as parts of a conditional statement:
"If these Conditions are true, then trigger these Actions". - Optionally assign Policies to your Rule so you can measure how well you're enforcing across each Policy.
Note: for companies with users in the EU, if you must comply with the Digital Services Act, you should select at least one Policy for each rule to meet your transparency reporting obligations. - Choose the status of your Rule - either Live, Background, or Draft.
- Click "Create Rule"
How to Monitor a Rule
After you create a Rule with a status of "Live" or "Background", Cove will start automatically running all your new incoming Items through that Rule. If you want to see what Items your Rule is catching, you can inspect a Rule, either by clicking on the Rule in your Rules table, or by clicking the "See Insights" button when you're editing the Rule.
In the Rule Insights page, you'll be able to see:
- The total number of Items that have matched the Rule every day
- A large sample of real Items that have matched the Rule
- If you click on one of the Items in the sample, you will see a full audit trail of why that Item matched the Rule and the result of each Condition in the Rule.
Updated 4 days ago