How to route conversations based on a knowledge search's confidence score
Last updated: 02 May 2023
Setting up a knowledge search is a powerful way to share content with your chatbot users. With a simple question, they can access any content you’ve imported inside your chatbot (PDFs, web pages, and more).
Learn to set up a knowledge search.
Knowledge searches, much like FAQs, work on the basis of a confidence score. When a chatbot user enters a question, the chatbot tries its best to find an answer amongst the knowledge bases you’ve imported.
It gives each possible answer a score out of 100. The higher the score, the more confident the chatbot is that this is the right answer. This is what we call the confidence score.
You can use this confidence score as a filter. For example, you might want to only show a knowledge search result when the chatbot is over 60% confident in its response.
This tutorial will help you set this up.
Step 1: Create a knowledge base
Before you go any further, make sure you have created a knowledge base. There are several ways to create knowledge bases:
Make sure your knowledge base(s) are searchable and full of juicy content.
All good? Let’s move on.
Step 2: Create a knowledge search conversation
Go to Builder and click + Conversation.
Give your conversation a name, a description, and an optional tag.
Set up your conversation trigger. This can be whatever you’d like it to be. For the purpose of this tutorial, we’ll set it up as When a user first uses the bot.
First message
The first message of the conversation should inform users that they can ask any question they’d like.
Click the three dots and Edit response. Write your copy inside the message field.
Hit Save.
Result message
Next, we’ll create the result message. This message will display the answer the chatbot has pulled from your knowledge base.
Open your Block bank. Click, drag, and drop a new step into the canvas.
Give your step a name (Results).
In the response, we will add the variables used to surface knowledge base content.
For now, let’s keep it simple and set up a standard message with both the knowledge article’s name and content inside the copy.
To bring up the variables, type {{. A modal will pop up, use it to search the names above.
Transition
We now have both sides of the conversation: the search and the result.
We need to tell the chatbot, in transition, to go and find the information from the knowledge base. Easy peasy.
Connect both messages by linking their dots. In transition, use the following conditional: Content import is triggered. This conditional informs the chatbot to go and fetch information from the knowledge base.
Don’t hit save just yet!
Filter by confidence
To finish our tutorial, we need to set up our confidence score filter.
For the purpose of this tutorial, we will set up a simple filter: if the chatbot’s confidence score is over 60%, it will return results from the knowledge base. If it isn’t, it will return a ‘Sorry, I don’t know' type of message.
In the transition, add a second conditional like so:
Make sure you set up an if and.
Hit Save.
Step 3: Create the alternative response
We’re almost all set!
Now, we should set up an alternative response in the event your chatbot’s confidence score is lower than 60%.
Click on the white dot of the first step and drag the dotted line somewhere else on your canvas. Click again to create a new step.
Give your step a name (No answer for example).
Set up the same transition conditionals, except this time make sure the confidence score lookup says is lower than 60%.
Click Continue. Write a few words such as ‘Sorry, I don’t have an answer'. Hit Save.
Voila!