What is Qlik Answers?

Perhaps the biggest announcement at Qlik Connect 2024 was a brand-new product: Qlik Answers. At the time, Qlik promised it would launch in July this year, and it delivered: Qlik Answers has been available to the general public since 31 July. In this blog, we explain what Qlik Answers is, what you can do with it and what you can't do (yet).

Retrieval Augmented Generation

Qlik Answers is an application of a technique called Retrieval Augmented Generation (RAG). You may be wondering right away: what is Retrieval Augmented Generation? To explain it properly, we need to start with generative AI technologies such as ChatGPT:

By now, you have probably used ChatGPT. It works like this: you ask a question in plain language and get an answer back in plain language. There are, however, two important caveats to the answers you get from ChatGPT:

• Hallucinations: First, there is the risk of so-called "hallucinations". The answer you get sounds convincing, but is partly or entirely made up.
• Not up to date: The other major limitation of ChatGPT is that it is limited to a certain point in time. To train the model, OpenAI's developers feed it all the publicly available information on the internet. The problem with the internet, however, is that millions of new pages and articles are added every day. To make sure training the model doesn't go on forever, the developers set a cut-off point. The latest model, GPT-4, for example, was trained on data up to April 2023. If you ask ChatGPT for Apple's stock price this month, it will tell you it has no answer.

With Retrieval Augmented Generation, the language model is extended with search capabilities. When you ask a question, the language model doesn't come up with an answer itself. Instead, a function is called that searches for the right answer in a predefined source. Once the answer is found, the language model is used to return it to the user in plain language. This approach solves the problems above: the function that is called can look at new information that may not even have existed when the language model was created. In addition, RAG can include references to the sources consulted by that function in its answer, so you can always verify the accuracy of the answer yourself.

How does it work?

To get started with Qlik Answers, you first need to define which sources can be used to answer users' questions. Qlik Answers makes this very easy: you can upload documents manually, or you can connect to a storage location such as OneDrive, Amazon S3 or Google Drive where these documents are kept.

In the knowledge base, you upload the files your assistant bases its answers on

The set of documents you upload becomes part of a so-called "knowledge base". Once the documents are uploaded, you click the "index all sources" button. This makes Qlik Answers read through the documents and try to understand them as well as possible, so that it can then use the content to answer questions.

Once you have created one or more knowledge bases, you can create a so-called assistant. This assistant is essentially the chatbot you talk to. You then link one or more knowledge bases to an assistant. The big advantage of this approach is that you have a lot of control over which information may and may not be used to generate answers for different user groups.

You link an assistant to one or more knowledge bases, after which you can start asking questions

Once you have created the assistant, you're done and your users can start asking questions. When an answer has been generated, you'll see four buttons in the answer's speech bubble. First, there are the thumbs-up and thumbs-down buttons. Your users can press these when they are satisfied with the answer, or when they didn't get the answer they were looking for. As a developer, you can find this feedback later on, so you can see where the gaps in your knowledge base are. There is also a button to easily copy the text of the answer. The last button has the form "View x sources". When you press it, a second panel opens showing the pieces of text the answer is based on. This window also includes a download button, which allows the user to download the original document.

Applications and limitations

With Qlik Answers, Qlik now also has a product in its portfolio to extract value from unstructured data. For many Qlik users, this is naturally still relatively unfamiliar territory. A logical question is therefore: how can I apply this in my organization? The possibilities seem endless, but to get you started, we have listed a few ideas.

• Make onboarding colleagues a lot easier by using Qlik Answers as an assistant. You can add all the manuals and work instructions for a department as sources.
• Bundle all product specifications, frequently asked questions and answers, and past support questions to enable your support staff to find an answer even faster.
• Combine applicable laws and regulations with internal policy documents and procedures so your employees can easily answer legal questions.

There are surely many more possible applications. The longer an organization has existed, the more documents it has. So the first release of Qlik Answers immediately opens up a lot of new possibilities. There are, however, a number of limitations you need to take into account:

• Qlik Answers can't read images yet and only works with text files. A scan of a handwritten document, for example, can't be processed properly yet.
• It is also important to realize that tables can be read in, but can't yet be used to answer questions. So if your document contains a table with last quarter's revenue per department, for example, you unfortunately can't query it based on that document.
• At the moment, it isn't yet possible to combine the unstructured data with structured data, such as the data in Qlik dashboards. It is reasonable to expect that Qlik is working hard behind the scenes to make this possible, especially because Qlik Talend Cloud already offers the ability to generate so-called "AI-ready" data products.
• Finally, it is good to realize that Qlik Answers taps into a new type of data source. For structured data, we have often already put quite a few data quality and data governance measures in place, but this is often not yet the case for unstructured data. To make Qlik Answers a real success, you will have to start thinking about measures in this area.

In short, Qlik Answers makes the valuable information in those documents much more accessible to all your employees. This can be hugely valuable in the move towards data-driven working. Do you have a good use case for Qlik Answers, or do you want to explore Qlik Answers further? Let us know, we are happy to think along with you!