Custom AI Models are the easiest way to enforce any of your Policies. For some Policies, you may be able to come up with a list of keywords to ban, or you may use some other Signals to determine whether an Item violates the Policy. But if you can't easily create a simple Rule that adequately enforces your Policy at scale (which is usually pretty hard to do), then Custom AI Models will be a great solution.

The basic idea is that you give Cove a Policy definition, and Cove will create a new AI model specifically designed to help you enforce that Policy. For example, if you have a Policy against Bullying, then all you need to do is provide your definition of Bullying - a few sentences of plain English will suffice - and Cove will walk you through a quick, 10 minute workflow to produce an AI model that automatically detects Bullying on your platform.

How to Create a Custom AI Model

To create a new AI model, you can do the following:

1. From your Policies dashboard, select the Policy you're trying to enforce (or create a new Policy if needed).
2. On your selected Policy, click "Train a Custom Model"
3. Select the modalities that your model needs to support. The options are Text, Image, Video, and Audio. Based on the modalities you select, Cove will make sure the model is trained to process all of those modalities. Then, click "Next: Start Labeling".
4. The next screen might take up to 5 minutes to load. In this screen, Cove will show you one Item at a time, and the Items will be real Items, pulled from your platform. Your task is to label each Item as violating or non-violating for the Policy you selected. Cove needs to get a sense of exactly how you'd enforce this Policy if you were the one making moderation decisions. That way, it can train the AI model to understand the Policy the same way you do and make the same enforcement decisions you'd make. You'll have to label between 200 - 400 Items in this labeling step. At the end, click "Continue".
5. This next screen might also take up to 5 minutes to load. In this screen, Cove will show you a sample of real Items, along with the scores that your new AI model assigned to each Item. We'll show you some Items that scored very high (i.e. the model strongly believes that those Items violate the Policy), some Items that scored very low (i.e. the model strongly believes that those Items do not violate the Policy), and some Items that scored in the middle. This will give you a sense of how the new model would perform if you use it to enforce your Policy on real content at scale.
6. Optional: If you're not satisfied with your new model's accuracy, you can click on any Item that you think received an incorrect score. Once you click on the Item, you'll see two new buttons appear under the Item - one that says "Violates Policy" and another that says "Does Not Violate Policy". Feel free to click whichever button applies best. You can do this on as many Items as you'd like, and once you're done, scroll to the top and click the "Save Mistakes and Retrain" button. This will incorporate your feedback directly into the model, and the page will refresh after a few minutes with a new sample of scored Items. You can repeat this process until you're happy with the model's accuracy.

Once you create a custom AI model, it will appear as a new Signal when you're creating Rules. You can then build Rules that leverage your new model to make automated enforcement decisions.