Building natural language AI assistants is a hot topic in the business and developer worlds so let’s look at how Rasa (www.rasa.com), the growing open-source conversational AI framework, compares with two of the most popular publicly available engines, namely Google’s Dialogflow and Amazon Lex. Arguably this can also be extended to a comparison with IBM Watson and Microsoft’s LUIS conversational AI engines.

Let’s start with some background.

The Growth of Conversational AI by Business

Conversational AI is one of the trending topics these days in the area of Artificial Intelligence (AI) as more and more businesses look at how natural language and AI can be used to automate key customer interactions and engagements. This is becoming increasingly true in the current COVID-19 crisis where traditional contact centers and customer service departments have been disrupted and chatbots are being increasingly deployed to help service customers.

Chatbots have been around for a while but have been met with mixed success rates and much market confusion. Early deployments failed to live up to expectations, resulting in the word chatbot earning some poor perceptions. But a lot has changed since the early and disappointing days of lightweight chatbot solutions. As the tools and technology have advanced, today’s breed of AI assistants have greatly expanded capabilities that are key to automating customer interactions and journeys, proving invaluable to business performance across many industries.

Rasa and the Market for Natural Language Understanding (NLU) Engines

There are a number of publicly available conversational interfaces (also called natural language understanding engines) for building natural language solutions including Google’s Dialogflow, Amazon Lex, Microsoft LUIS and one of the most well know, IBM’s Watson (the famous Big Blue AI engine that won jeopardy in 2011).

All of the major technology companies have or support natural language platforms (including Facebook – wit.ai is their natural language interface) and most make them available as publicly accessible APIs for both aspiring data scientists and increasingly, for citizen developers. Many companies looking to deploy conversational AI solutions use one of the public engines.

Billions of dollars have been invested by the big tech companies in AI technologies and companies can leverage this through a simple API. However, some businesses are increasingly concerned about the big technology giants getting access to their data and are seeking alternatives including open-source solutions.

Rasa is the newest of these platforms and is an open-source natural language processing tool for intent classification, response retrieval, and entity extraction in chatbots. It has a number of components including Rasa NLU (which interprets natural language messages) and Rasa Core (which manages the dialog with a user by building a probability model around next actions in order to manage the context of the conversation).

So here are three ways in which Rasa compares more favorably with Dialogflow and Lex:

1. Rasa Open-source Project versus a Ready-to-Use API

Let’s start with the most obvious comparison – open-source projects vs. ready-to-use API.

Rasa is an open-source project where the code is available for anyone (with Github access) to download and install – in other words, it is software that must be installed on a computer and configured to make it useful. Dialogflow and Lex are APIs that are immediately available to use to anyone with Google or AWS login credentials.

On the one hand, Google and Amazon have made it really easy to get started with building natural language solutions without any fuss around how to install it, configure it, or get going. Just create your first project and start adding intents and training your model with utterances. You don’t have to be a data scientist or engineer to get going with these solutions. With an hour or two of training, they can be used by a techno-savvy business analyst to great effect. (To be fair, Google’s Dialogflow is a lot easier to get started than with AWS Lex and shows better accuracy with minimal training data.)

But…

On the other hand, they are black boxes and you can’t see what makes them work or what algorithms they use. Setting aside (for a moment – patience please) the installation and configuration of Rasa, it is much more transparent about what it does and how it does it. It is designed for developers, AI engineers, and data scientists and exposes the entire pipeline of how it operates allowing an experienced user to tweak many parts of the model to great effect. The results from Rasa are also better than either Google or Amazon.

In a simple test shown below (thanks Shane!) we tested all three NLU engines with the same query to see how they dealt with things like capitalization, acronyms, and sudden context switching.

  • Lex has the biggest problem with capitalization, acronyms, and numbers in normal utterances. It also doesn’t deal well with special characters in the utterance – even something like a question mark.
  • Dialogflow does much better with acronyms and capitalizations and in general, is better than Lex with fuzzy logic.
  • But Rasa clearly beats both of them when it comes to matching fuzzy questions (such as the ones in our test about LASIK [surgery]) and dealing with mis-spellings.

{INSERT COMPARISON TABLE}

2. Rasa Context-switching vs. Dialogflow and Lex

Traditional NLU engines have a classic problem that both Dialogflow and Lex suffer from and that is always trying to match an utterance to an intent, even if the probability of a match is very low. This leads to the problem of false positives where the NLU gives a response but the wrong one.

  • Dialogflow, at least, provides the user with the confidence score allowing the user/developer to make a judgement on when to accept a response or not.
  • Lex doesn’t provide access to a confidence score and so it can be problematic.
  • Rasa does a better job at switching context and allowing the user to control the conversation by defining probabilistic models of what to do after each intent i.e. where the conversation should go next. It also provides feedback on how it makes decisions with access to confidence scores and even allows the user/developer to modify the pipelines that are used to process or pre-process the data.

3. Rasa’s Active Community

Rasa is an open-source project and has an active community. One of the benefits of this is the help that a user can get from the community – very similar to the help that developers get from Stack Overflow. Like many open-source community projects, this is way better than the help offered by either Dialogflow or Lex.

The Caveat: Getting Started with Rasa versus Dialogflow and Lex

But for all the positives about Rasa, it is still quite difficult to get started and it’s not for the casual user. You have to either run the code within a container (requiring some familiarity with Docker), or even more difficult still, install the Python package and it’s dependencies natively on your machine. Then the real fun begins – selecting the appropriate pipeline and configuring the various tuning parameters. And that’s only to build and train your AI model. You then have to worry about putting it into production, scaling it, allowing users access to it, and dealing with everything from security to backup and recovery. Dialogflow and Lex, on the other hand, do all of that for you.

Summary of the Pros and Cons of Rasa versus Dialogflow and Lex

In summary, Dialogflow and Lex are easy to get started with and provide acceptable results from a cold start i.e. without much training data. They are easy to use by everyone – from business users to data scientists and AI engineers. Dialogflow does a better job of handling “fuzzy” questions and in dealing with capitalization, acronyms and special characters.

Both of them are, however,  black boxes compared to the power and transparency provided by Rasa and the control that the professional developer has over many aspects of the AI – from pipelines to training models to feedback. Rasa is a powerful tool designed for the professionals but it takes a lot of work to get it up and running and the user/developer has to deal with all the other issues such as production, scaling, security, backups, and user access.

In our next blog, we tackle these drawbacks showing you how RaaS.ai can help take the pain out of getting started with Rasa.

Top