"Intelligent assistants recognize what users are looking for; and provide precise and relevant answers."
How do you go from the search system you have today, to one that serves as an intelligent assistant to customers, partners, and/or employees?
To begin with, you need to start looking at search as an application, rather than as an enterprise-wide utility. Searchers are always going to enter very little information about their query. It is the goal of the search system to ask the searcher what they mean, to probe about their intent or to present possible choices both before they search and throughout the process of reviewing results. This can’t be done in a general way for John Q. User.
So the first step, as with any serious application development project is to understand user’s requirements. The second step is to identify and map content to query intent.
Understanding User’s Intent
The problem with most content management systems or knowledge management systems is that they are developed with generic use cases or no use cases at all. People set up the goal of wanting to “make information easier to use” and when you ask “what information?” they say “all of it”. Well, perhaps we can make “all” of the information easier to use, but we have to break the problem down first and start to characterize and describe its various components.
The question is what makes information easy to use? Usefulness is based on the value of that information in helping the user in achieving a goal. Making information easier to use is an iterative process of classification - classification of the user, the objective, the task, and the information.
We have to ask “What goals are the users trying to achieve?” This gets back to the purpose of the application or intent of the user. If we are developing a help system, inherent within that objective are the circumstances of who we are helping. What kinds of users do we have? This is the first instance of classification.
The second, what kind of things are they trying to accomplish? This is another instance of classification. Loan officers (type of user) underwrite loan applications (type of task). With these two questions, we begin to clarify user objectives and establish a context for search. But to provide value, user analysis for an intelligent assistant application needs to go deeper.
Take the example of a web application to underwrite a particular type of loan for a loan officer at a bank. Help can take the form of assistance with the mechanics of a simple task. For example, “How do I create a new loan application?” For straightforward tasks, the steps can be broken down and articulated and no matter where the user is in the task, instructions can be written to tell the user what to do next: Step 1, do this, step 2, do that, step 3, click here, etc.
Adding complexity
The above is a simple example. However, the problem can quickly become orders of magnitude more difficult as task complexity grows. As processes become more interrelated and encompass edge cases and more unusual sets of circumstances, instructions grow more difficult to articulate and sequence. The task may still be mechanical, but instructions become more nuanced and complex. Many of us have had the experience at an airline check in where some combination of conditions cause the agent to have to call over a more experienced colleague or contact a help center to update a flight or make a change to an itinerary. The combination of inputs and relationship of processes were never anticipated by the systems help instructions.
Moving beyond the mechanics of a system or process, the next level of required information can be expressed as solving a business problem which ranges from clearly defined policies, rules, and procedures to broad guidelines and authority levels that are dependent upon talent, judgment and the ability to interpret ambiguous signals. “Does the bank have the appetite to write this kind of loan?” This now becomes a problem that can be described at the business level.
What is the type of loan? Auto, home, business? What is the level of funding? What are the risks associated with the loan? Consider a business loan. Business financial risk assessment is never done in isolation – there are hundreds of factors that enter into the equation. What are the risks associated with the borrower? Beyond the details of the type of business, quantifiable financial ratios and metrics, there are larger market and economic trends, and then the subjective assessments of strategy, product plans, competitive advantages, unique products and offerings, go to market activities, leadership and executive talent, manager track record, cultural issues, etc.
It is possible to begin to create the rules for understanding whether it makes sense to write a loan depending on aggregate credit scores and financial ratios. However, the dozens of factors (that need to be aggregated and summarized from hundreds of details and signals) are still often best interpreted by an expert experienced in the subtle interpretation of these inputs.
So, how far can we go in replacing the human expert? In many cases, this is what financial firms and insurance companies are trying to do – weigh multiple inputs and factors to determine a simple answer – yes or no, but then provide the details of terms, conditions, caveats, covenants, pricing and then to place this risk (whether a loan or an insurance policy) into a package that can be valued for short term and long term financial management and trading.
In the case of insurance, the “help desk” is a hotline to an underwriter who assesses the information provided, asks additional questions and provides the nuances and details of how to write the business. This is expert work with the highest value and utility. It is also costly and difficult to scale as a business expands or new business lines and financial products are introduced.
Identifying and Mapping Content
Imagine if the expert’s role could be automated? How could that expertise be extracted and associated with end-user queries? This is precisely the challenge ahead of us with a particular organization who is trying to embody the process into a help system. The system is centered on an intelligent assistant - an animated “Siri” type of character who interprets the things that they user asks in the search box, and presents the result – the response – in the form of text-to-speech instructions delivered by a interactive animated character. The goal of the project is to structure the inputs and outputs so that the question can be asked unambiguously, and the responses are relevant to the (often unstated) intent of the user.
But what about all of that judgment that needs to be codified? Didn’t I just say that this was exceedingly difficult? Well, it can be. But one starts with the basics. Modeling of content, modeling the gamut of user intent, and modeling of search queries so we understand how they map to user intent. Users typically ask questions with one- or two-word search phrases which need to be disambiguated. That is a straightforward task. This is done by looking at the searches that are executed over time and writing the use cases and problem statements that embody that query. By testing the query against the content, rules can be written to improve the results, so that the system returns a response that is relevant to what the user intended, or at least can provide the user with more clues and hints to allow them to submit a more meaningful query.
This still begs the question of how to get that expert content and how to get the user to ask the correct question. This is not as straightforward. This can be accomplished through exhaustive interviews of experts to model every decision that they make and create the rules for walking users through the same interview process that they might engage in with someone on the phone who called the underwriting desk.
Mine and analyze
But there is a better way. It has to do with mining and analyzing the past interactions. In our example, help desk interactions with users have been recorded over a period of years. These recordings can be converted to text and mined for patterns that provide clues to user intent. Rules-based classifiers do part of the job for known situations, but emerging themes can be derived using other techniques.
The classifier determines that two things are conceptually related but it does not know why. Enter LDA or “latent pattern recognition” (technically called “latent dirichlet allocation or LDA – a branch of computational linguistics) software. LDA is a form of semantic analysis that can find meaning, hidden in large amounts of text. This can be used in help desk applications to help surface the user intent in search queries.
LDA algorithms can also find the answers to questions that have not yet been asked. LDA can discover relationships between various bits of content that may not be captured in a rules-base or a taxonomy. These relationships help us identify answers to previously asked questions that aren’t in the content. They also provide an “early warning system” for new categories of questions that are entering the system.
Summary and Conclusion
We use LDA to associate content with user queries. The methodology is actually straight forward.
- Model the search use cases (who is searching for what, when?)
- Map them to content types (what is the object of their search in the document-based system)
- Map them to custom search result types for each content type (How can we best present the relevant information instead of just a link to a document?)
There is lots of unstructured information that contains meaning and value, and our job is to uncover the meaning and extract the value. These hidden relationships can be surfaced in search queries, and can be used to derive attributes and rules for complex decision support applications such as loan or insurance underwriting.
Banking and insurance companies are finding tremendous value in these approaches for mining meaning from text and large amounts of unstructured audio data. These tools can help us capture the intelligence that is inherent in the patterns of interactions. All we have to do is mine with advanced algorithms, model users and their interactions, and develop structured content in order to present that information in the right situations. LDA gives us insights to model user intent faster, and identify the content that they seek. The value is enormous and is the next generation of knowledge processing and artificial intelligence.
For a look at how we use information architecture to design and build an Intelligent Virtual Assistant download our white paper: Making Intelligent Virtual Assistants a Reality. In this paper we show how to deploy intelligent applications in your enterprise to achieve real business value.