Explained: The self-learning capacity of HiJiffy’s conversational AI

Let’s get technical in this article and explore one of the fascinating capabilities of our Aplysia OS: self-learning. What does it mean? How does it work? We asked the Aplysia Lab team these big questions, and here is what we have learned.

Development of a chatbot’s “brain”

Before deep-diving into the topic of self-learning AI, it is worth taking a moment to define what a chatbot is in this context exactly. It can be explained as an artificially intelligent programme capable of having a conversation with a human, thanks to the latest advances in AI and Natural Language Processing (NLP). However, like humans, it does not have all the knowledge about the world from the start. Sometimes it does not know words or certain specific linguistic expressions because it simply had no chance to learn them yet. To use an analogy, the knowledge contained in the conversational AI model is like a chatbot’s brain.

These models develop and grow over time because they have to be continuously trained. Often, these training sessions are run by a human without supervision and without verification of how the model will react. If it is poorly taught or is not instructed on how to do its job correctly, it will make mistakes. 

Without thorough training and testing, the models will likely have the following issues and challenges:

1. Lack of knowledge of words, phrases, and expressions.

There are millions of expressions, words, and phrases with various purposes throughout the world, and this is also established inside each dialect. This can lead to the chatbot occasionally lacking information and giving the wrong answer to the user’s questions.

2. Overlap in topic identification (confused topics).

If the model has not been fed sufficient information necessary for distinguishing between similar topics, it can incorrectly interpret the topics and offer confusing answers.

3. Reduced level of accuracy in the required answers.

The chatbot’s accuracy determines its success rate and directly impacts the chatbot’s automation capabilities.

How does AI self-learning work?

HiJiffy created a self-learning tool for conversational AI as part of the Aplysia OS. In other words, our technology can learn new topics, expressions, phrases, and words to address the previously presented issues. It can identify more than 200 hospitality-specific topics. Thanks to its self-learning capacity, HiJiffy’s AI can use customer feedback to understand incorrectly identified terms and expressions. It can adapt its model to better differentiate among already-existing topics in its model and ultimately improve the accuracy of its answers by reducing the error rate. 

The following illustration summarises the self-learning flow:

As presented, this self-learning system is based on feedback from our clients. HiJiffy allows hotel staff to report sentences from real conversations with their guests when they feel the AI did not provide the best answer.

Taking into consideration HiJiffy’s focus on better connecting hotels and their guests and helping hoteliers automate processes, we continuously improve our conversational AI model to give the most accurate answer possible to our client’s guests. All feedback is welcome and used to make our model even better.

With this in mind, we develop a self-learning tool checking whether the cases reported by the clients have been correctly registered. It will also test and measure the impact of training with these sentences on improving our model knowledge. This tool will provide specific metrics and decide which cases should be changed and which should not, taking into account AI mechanisms and machine learning.

Self-learning in practice

To better understand this complex process, let’s look at some case studies demonstrating how our self-learning tool addresses the three challenges mentioned above.

Lack of knowledge of words, phrases, and expressions.

The conversational AI did not recognise the word seaplane as a mode of transportation. One of our clients provided Aplysia OS with this input. The self-learning tool used this information to verify whether what the customer had written was accurate and determine that a seaplane is a mode of transportation. Aplysia OS approved this case, and the conversational AI learned that seaplane is a mode of transportation.

Overlap in topic identification (confused topics)

The sentence “charging electric car” was assigned to the “Parking” topic rather than “Electric Cars” due to an overlap between the two topics. In reviewing this circumstance, Aplysia OS learned about this through a client’s feedback. It turned out there was an overlap of mislabeled sentences that were ambiguous to the model, making it difficult to distinguish between the two topics. Aplysia OS handled these mislabeled sentences, and the AI learned how to correctly assign the “charging electric cars” sentence to the most relevant topic.

Reduced level of accuracy in the required answers

To illustrate this challenge, our Aplysia Lab selected a dataset with 2554 messages that were correctly labelled with the topic (this annotation was done by our clients over time). There is at least one message from each topic in this dataset. We ran the conversational AI to verify the results before and after the self-learning capability work.

Below is a brief sample from the dataset to illustrate the results. The sample shows the messages (in original spelling) and the correct answers. In the last two columns are the results before and after the self-learning was implemented.

As illustrated, the messages for which the right answers were not assigned correctly initially, after the self-learning have been given the right ones. The sentences presented may have been some that the conversational AI had to learn and adapt its knowledge accordingly.

It should be noted that although this sample shows cases in which the AI learned the answer, there are cases where client hotel staff submitted sentences that may not make sense, and, as explained earlier,  Aplysia OS prevent the AI from learning those cases.

The graph below shows the accuracy and precision of the conversational AI model before and after self-learning.

Considering all the changes proposed by the self-learning tool, we improved the model and have not registered any degradation on the previous answers that the AI has already been giving correctly.

In addition to the conversational AI being able to address and resolve the challenges presented earlier in this article, we were still able to achieve better results by around 4% on this dataset. Other datasets were tested and on average we observed better results by about 5% after a week of self-learning.

In conclusion, this feature within our Aplysia OS allows our conversational AI to learn over time by meeting what our clients want. On top of that, the results improve over time and HiJiffy continuously perfects our model for the hospitality area.