Blog Post: Artificial Intelligence in Chatbots

Artificial Intelligence in Chatbots

In my previous paper about Chatbots I focused on why chatbots fail and how to succeed. In this report I will give you the basic understanding about Artificial Intelligence (AI) in conjunction with chatbots.

The fundamental parts of AI is:

  • Machine learning
  • Text analysis
  • A domain data model
  • Context

Worldwide, we send over 23 billion text messages a day. Texting is the most widely used mode on smartphones and over 90% of the text messages are read in under 3 minutes. About 30 billion messages are sent daily in Whatsapp. Hence in order to build a successful AI driven chatbot, text analysis is a fundamental part, so let us take a look at the subject.
AI Chatbot
Text analysis in AI driven chatbots consist of 3 major parts:
  • NLP (Natural Language Processing)
  • NLU (Natural Language Understanding)
  • NLG (Natural Language Generation)

Natural Language Processing – NLP

NLP is the broad definition concerned with the interactions between computers and human (natural) languages, in particular how to program computers to process and analyse large amounts of natural language data. It describes the computer’s ability to ingest what is “said” to it, break it down, comprehend its meaning, determine an appropriate action and respond back in a language the user will understand. The history of NLP goes back to the 50-ties.

Natural Language Understanding – NLU

NLU is a subset of NLP that deals with the much narrower, but as important facet of how to best handle unstructured inputs and convert them into a structured form that a machine can understand and act upon. Humans are able to effortlessly handle mispronunciations, swapped words, contractions, colloquialisms, and other quirks, machines are less adept at handling unpredictable inputs.

Natural Language Generation – NLG

NLG deals with, when a computer answers, i.e when a computer writes language. You can say that the NLG process turns structured data into a text that human understands and relate to.
Let us look at an example, to point out the difficulty the AI driven chatbot has, compared to a human being.

A friend of yours asks you to buy some wine. Since you do not know her preferences by heart you ask what kind of wine. She answers, red wine, Australian Shiraz and one bottle. You make some calls to different stores, until you find a store that has the wine your friend wants.

How, would this, be processed by an AI driven bot?
Chatbot
1. The wine selling bot would send in to the messaging backend – wine.

2. Using NLP (what happens when computers read language. NLP processes turn text into structured data), the machine converts this plain text request into codified commands for itself.

3. Now the chatbot throw this data into a decision engine, since in the bots mind it has certain criteria to meet to exit the conversational loop, notably, the type of wine you want.

4. Using NLG (what happens when computers write language. NLG processes turn structured data into text), much like you did with your friend the bot asks you what kind of wine.

5. This array of responses goes back into the messaging backend and is presented to you in the form of a question (red, white or rose). You tell the bot you want red wine, Australian Shiraz and one bottle and we go back through NLP into the decision engine. The NLP sorts out the 3 important parameters – red wine, Australian Shiraz and the quantity (1 bottle).

6. The bot now analyses pre-fed wine data domain documents data about the product, stores, their locations and their proximity to your location. It identifies the closest store that has this product in stock and tells you what it costs.

Please note, for the sake of simplicity, that I have heavily simplified step #5, above. A chatbot that would have managed to sort out these 3 parameters in only one interaction, would have been a heavily trained chatbot.

Artificial Intelligence needs to learn. You can dramatically decrease your failure when deploying your chatbot by giving it a good head start. Chatbots need training, feed it with domain data, they learn to do new things by trawling through a huge swath of information. Furthermore users need to be carefully onboarded so that they understand the constraints of the software they are interacting with. But even with these limitations, reducing complicated, confusing processes down to a normal conversation is potentially very powerful.

And remember what I wrote in my last paper a vast majority of successful chatbots are in English. I have still to see an advanced successful chatbot managing a good dialogue in Norwegian, Swedish, Finnish or Danish. Hence, one of your requirements should be your language. Has the tool you are looking at, a language package in the intended language for the chatbot? Do the vendor/consultancy company that offers you a chatbot have any documented experience in NLP, NLU and NLG?