How to deal with large amounts of intents
Creating a simple chatbot for a small organization is easy. Conversations are usually limited in scope and the amount of intents a user can have is typically quite small.
For large corporations or more advanced chatbots this is quite different. Some of their chatbots cover all different kinds of questions customers have. For this you will require countless intents. But how can you keep oversight of all these intents?
This week Chatlayer launched a major upgrade to its platform. The Chatlayer 3.0 release brings many new features. We now have built-in voice bot functionality, versioning, synonym entities, a code editor, an iFrame module and language independent NLP. But 3.0 is also introducing two new components to help you optimize your NLP: the NLP Dashboard and the NLP Improve.
NLP Dashboard
The NLP dashboard gives you an overall indication how well you are doing for each language. Based on how you have trained your NLP we give you an overall model score ranging from 0% to 100%. The overall model score is computed based on the amount of intents with too few expressions and the amount of intents with misclassified expressions.
On the left of the dashboard a general summary of your model is given. Below you can find your train history which will show you how you have done in the past and whether or not you are improving over time. On the right we list all the intents with know issues. Between brackets you can see how your score has changed since the last training.
Too few expressions
A first metric we measure is the amount of expressions. To optimally train your model we advise to have at least 20 expressions per intent. Ideally you have around 50 expressions per intent.
Of course it is important to make sure these intents are well diversified. For more information on how to do that, read our documentation here.
Misclassified expressions
Just like humans no AI/NLP system is perfect. And because our minds work differently than the Chatlayer NLP it is hard to predict where mistakes might happen.
To guarantee the best possible outcome for the end user we do an advanced analysis on all the expressions. This determines the expressions for which the NLP might have a hard time. For example it might have difficulties differentiating between sentences like: “I see on my bill that I have the wrong subscription” and “My bill seems wrong if I look at my subscription”. So the second metric we take into account is the number of intents that have a risk for these misclassified expressions.
In the intent list on the right of the NLP Dashboard you can click on the wrench icon to improve your set of expressions, and thus to better discriminate between intents.
NLP Improve
NLP Improve helps you with creating better expressions for intents where the system detected issues. On the top of the dashboard you can select the language and the intent you want to improve. The list is automatically sorted to show you the intents with the most known issues.
Next you can select the expression you want to improve. Once you have selected this expression you will see the confidence on the right for the top two or three intents. Your NLP model might be more confident that your expression belongs to another intent than to the correct intent.
Our system automatically analyses the keywords that are most relevant for detecting a specific expression. As you can see in the example above the words how, are and you were key in classifying the expression “how are you doing?” as Greeting instead of How are you.
There are in total 6 different approaches to improve the performance for this expression using the improve dashboard:
- Move this expression to the Greeting or Who are you intent.
- Delete this expression if it doesn’t belong to any of the intents.
- Add more expressions to the How are you intent that contain the words How, Are and/or You.
- Remove expressions from the Greeting or How are you intent that contain the keywords How, Are or You.
- Add more expressions to the Greeting and/or How are you intent that do not contain any of the keywords.
- And as last option you could merge the Who are you intent with the Greeting or How are you intent. If you still want to differentiate between these intents, you can work with entities instead.
In this dashboard you can click on the ‘i’ icon on top to get advice tailored to your specific expression.
Each of the six options will guarantee that your model performs better. It is up to you as a user to pick the option that works best for your case. Once you have tried one of then you can retrain your model. On the NLP dashboard you can see the progress you made, and you can compare the results with what you achieved since the last time you trained.