Build a powerful search engine that fits with AI technology.
Watson Assistant, formerly known as Watson Conversation, provides a great technical foundation for building applications that understand and respond to natural language, such as messaging channels, web apps, or virtual assistants. Due to the popularity of such technologies, chatbots are almost ubiquitous.
But not all chatbots are created equal, and user expectations can vary widely. Some chatbots may be like the well-known Watson Jeopardy AI, while others are more like interactive FAQ programs that can only give generic and lengthy responses to specific questions. Depending on the initial interaction, users may be happy or bored, and this experience can influence whether or not they will come back again.
Unfortunately, extensive training is not always feasible for nascent chatbots. If the chatbot is not trained enough, they may only inform the user of what they have been trained on, which may not necessarily help the user solve a specific problem. They may also be relegated to performing a search, but they can't provide the same content as a proper search interface.
For many, search remains an effective way to find answers, while users may face the dilemma of whether to explore a new chatbot or continue to search on their own. There can be overlap in functionality between these two very different solutions, especially when chatbots are forced to frequently downgrade to perform searches. If the chatbot doesn't meet their initial expectations, the user may revert to search and avoid using the chatbot in the future. When it comes to providing options, first impressions are critical to long-term adoption. If there is another way to replace a discarded chatbot, it may not be possible to keep those users coming back even if the bot makes significant improvements in the future.
Another consideration is the role of natural language. Chatbots need a rich vocabulary that includes synonyms, idioms, and technical terms from the corresponding knowledge base. Users may be forced to guess and memorize specific words, and misunderstandings may arise. Natural language support is a valuable tool, and recognizing its limitations can provide a better overall user experience. For example, don't let natural language support hinder the use of keywords. Keyword searches with advanced features such as quotation marks, booleans, exclusions, and inclusions in an appropriate way are incredible. The ideal solution would support the use of natural language and keyword search terms, as appropriate.
When we set out to integrate this technology into existing applications, we explored a new way to overcome many of these shortcomings while taking full advantage of this exciting new technology.
We circumvent these problems by integrating both search and chatbot technologies into a single user interface based on typical search paradigms, avoiding the need to choose between competing systems. The existing search system has a powerful search engine that is keyword-based and uses Apache Solr, and the system has been tuned for stand-alone use. Based on that, we lead to Watson Assistant and the results from Solr. This "hybrid search" approach aligns user expectations with what they expect from traditional search engines. However, when trained around a specific intent, Watson Assistant, which fully understands natural language, responds in a way that is simply pleased, which could be an answer to a question that traditional search engines can't give or a self-service form.
This approach has a number of advantages. By basing on the traditional search paradigm, search results are always returned, but Watson only provides a solution if it is specifically trained. This is extremely useful for gradually scaling watson components with more training. While traditional chatbots are relegated to performing searches, hybrid search approaches become progressively smarter by training on more solutions over time.
Another point of disagreement is restricting the use of natural language to identifying the correct Watson intent, rather than providing a restricted or semi-directed approach to the presentation of conversations. For example, if a user enters the phrase "I want to reset my mobile device" into a traditional chatbot, the conversation might provide a list asking the user which device to reset. At this point, the user might type a valid response or a completely different answer that could cause problems with the conversation flow. At the very least, it can reset the conversation to its initial state and take another path, or cause Watson to re-ask the question.
Our approach is to limit the user to choose from a set of options that Watson has already been trained on. For the same prompt for resetting a device, Hybrid Search responds with a set of predefined alternatives, one for each device that is enrolled and one for the device that is "not listed". This reduces the risk of possible misunderstandings in the case of natural language use and avoids unnecessary guesswork.
By expanding on the idea of providing only valid options, we've also added intelligence on the front end to detect the different types of responses that Watson might return. For example, if we determine that the response is a "support option" (for example, number, create a ticket, or start talking to someone), the information will be displayed in a card type format that the user can interact with. If you return a list of documents, the hybrid UI retrieves information about those documents so that users don't have to open them to make better choices. Watson can still inform the hybrid UI to enter formatted text (for example, a serial number) or free-form text. These types of controls are the foundation of the interactive form and lead to a richer support experience.
When a user enters information in the search bar, our app implements an autocomplete result mode that displays a quick list of the main search results to those users. When we started integrating Watson into search, we ran into some limitations. When a user enters a search string, the app examines each character entered to see if there are results that can be served directly to the user. This call must be exceptionally fast. The app doesn't wait patiently for a response from Watson in IBM Cloud to create a session, perform an intent match, and then respond. Another issue is that the response returned by Watson could be a form, a document, a support definition, or a request for more information. This doesn't work with fast search or autocomplete paradigms. Finally, there is the cost factor. Watson is a billing service, and you have to pay for what you use. The autocomplete keyword search method is used 10-20 times more than the full keyword search.
To enable us to include Watson-based intents in our quick results, we developed a custom service that extracts information from the Watson Assistant corpus used by the app. In this new service, we use the existing Watson API for each corpus to retrieve all intent and conversation information. Conversation information is used to extract top-level intents (i.e., entry points into the user's conversation flow). Subsequently, this collection can be further constrained by removing non-critical intents, such as "conversation start" and "anything else". After narrowing down to the core intent, we extract the intent examples from Watson. Logically, we think of the intent and its examples as a "document". The resulting documents are stored in Solr along with other indexed content. This process runs nightly.
Now, when a user starts typing in a search query, we show results from the original content in Solr, along with key intents that can start a conversation in Watson. For our own purposes, we promoted branding and added a brand mark on top of other content in Watson's intent to showcase this new technology. From the user's perspective, in the autocomplete results area, they will consider the first example intent as a possible outcome. If the user selects this from the drop-down menu, they are taken to a fully blended search result, and the sample intent is passed into Watson as if the user entered it. This creates a Watson session with a guaranteed positive outcome.
As users become more familiar with the Watson integrated in their applications, we expect them to start performing more natural language processing (NLP) searches. Originally, users would type in a keyword-based search, such as "connect to VPN," to find help on how to connect to a corporate VPN. When they recognize that Watson is also looking at what they've typed, smart users may start typing full phrases, such as, "I can't connect to VPN from home."
The use of NLP phrases brings with it the problem that the quality of search results may deteriorate. From the tests we've done, Solr itself doesn't handle nlp statements properly. To help provide accurate results, we've added functionality from the Apache OpenNLP component. This component can look at utterances and extract different parts of speech. We use this feature to extract nouns and verb phrases and effectively construct keyword searches based on the entered statements. Feed the previous NLP example into the processor and the result might look something like:
i_prp can't_md connect_vb to_to vpn_nnp from_in home_nnNote that in the output above, the suffix is an abbreviation for the part of speech of the word.
By extracting only the noun and verb phrases, a set of keywords "connect IBM home" can be obtained. This logic only applies if the input string exceeds a certain threshold for the number of words (for example, 5 or more). This new set of keywords will be used with the original string to build the combined results, which should contain the exact match and keyword. Note that only the raw string is sent to Watson and it will perform NLP intent matching on its own. From a user's point of view, we combine the best of both worlds efficiently and fully.
Here's some background information about this app. It is a microservices architecture that is deployed using docker images. Each service is hosted in a separate docker image and deployed in a high-availability environment. To strengthen support for the above features, we have made the following changes to the application architecture. Note that the content shown represents a simplified view of the architecture, and there are several non-impacting services that are not shown for clarity.
Here's a brief description of the components listed above:
Support site UI: This is a web front-end app that users use to interact with the system. Content service: This microservice performs searches and retrieves information about many different types of documents from the content repository. It uses couchdb in conjunction with solr to facilitate this task. Apache Solr: A fast, open-source search indexing engine. Aggregation Service: This is a newly created service that can be invoked by a single composite UI to retrieve and combine information from Watson and existing app sources. We provide this service as an extension point when integrating more Watson services and external data from other systems. Orchestration service: Since we'll be using the same workspace for its other layers (for example, chatbots or voice response units), we created that layer to serve as a wrapper and integration service for Watson Assistant. It allows us to augment and refine the responses from the conversation flow with information stored outside of the conversation. Apache OpenNLP: Our open-source natural language processing component for identifying parts of speech. When a user enters a search term, the following flow is used. The first process represents the autocomplete process. Note that the search all happens inside the app, and the indexed intent represents a potential entry point into Watson Assistant.
The UI sends the currently entered search query along with other user-related information (personalization, authentication level, etc.) to the autocomplete service entry point. The aggregation service determines the size of the input phrase. If the specified size of the number of words is exceeded, then OpenNLP is called to extract the key noun verb phrase. The aggregation service calls solr to see if the current phrase matches the indexed intent example. Since this is from the user's perspective, the response must be fast, and the solr query is only used for the first 2 or 3 intent matches. At the same time, the aggregator service also calls the entry point of the existing content service to retrieve the autocomplete results. Depending on what happened in step 2 above, it passes in the original search string or noun verb phrase. This result set is likewise limited to the first 2 or 3 results. As a result of this flow, users will see a set of potential primary hits from existing content and the primary intent (the first example of the intent we show the user intent) as conversation entry points. If the user selects the Watson intent, it triggers the hybrid search flow described below. In a hybrid search flow, watson is called as part of this flow. If Watson returns results with a high enough confidence level to help the user execute the query, it guides the user through the process through the interaction and the existing results of a traditional search. The process is:
The UI sends the search query along with other user-related information, such as personalization or authentication level, to the autocomplete service entry point. The aggregation service calls the orchestration service to start a session through Watson. The orchestration service calls watson and creates a session. The orchestration service determines whether Watson's response has a high enough confidence level, and then augments the information with any necessary data. If the confidence level is not high enough, the orchestration service tells the aggregation service that Watson can't provide any results to the user. The aggregation service determines the size of the input phrase. If the specified size of the number of words is exceeded, then OpenNLP is called to extract the key noun verb phrase. The aggregator service also calls the entry point of the existing content service to retrieve keyword-based results. Depending on what happened in step 2 above, it passes in the original search string or noun verb phrase. As a result of this flow, users will see a set of potential primary hits from existing content and the primary intent (the first example of the intent we show the user intent) as conversation entry points. If the user selects the Watson intent, it triggers a hybrid search flow. Hybrid search is a new way to use Watson Assistant that does away with some of the limitations of traditional chatbots. This approach allows you to realize the best of both worlds: on the one hand, a powerful search engine that is compatible with AI technology; AI technology, on the other hand, can evolve as Watson continues to receive more training. Advanced users retain the power of keywords and advanced search methods, while others simply ask questions using natural language. While integrating with search and fast results presents some unique architectural challenges, we've outlined some technical solutions that can help other adopters get their hybrid search applications off the ground.