Build a retrieval augmented generation (RAG) AI app using SDKs
This tutorial includes the following SDK languages and versions:
TypeScript v1 TypeScript v2 Java Python v1 Python v2 C# Go PHP ❌ ❌ ❌ ❌ ❌ ✅ ❌ ❌
Everyone has heard of ChatGPT, the popular large language model (LLM) that can generate human like text, answering any question you may have with a certain degree of correctness. However, the biggest problem with these large language models is that they only have a limited amount of 'knowledge' to draw on - they are trained using data from the internet up to a particular date, and that is it.
For example, if you ask an LLM what todays date is, or what the weather is like in a particular city, it will not be able to answer you - it just doesn't have that data.
This lack of data also limits an LLMs ability to answer questions, or reason, based off your own data. For example, if you have a database of customer reviews, and you want to ask the LLM a question about the reviews, it will not be able to answer you as it simply doesn't have access to that data. This is where retrieval augmented generation (RAG) comes in, allowing you to retrieve data from your own systems to augment what the LLM can reason over.
This tutorial will show you how to develop an AI powered app that uses SDKs generated by liblab to provide RAG capabilities to an LLM. Specifically using augmenting ChatGPT with cat facts! Your app will use Microsoft Semantic Kernel as a AI app framework, and ChatGPT from OpenAI as the LLM.
Prerequisites
This tutorial assumes you already have:
- The liblab CLI installed and you are logged in
- An OpenAI account and API key1
- .NET 8.0 or later installed2
1 There is a small cost associated with using the OpenAI API, so make sure you have enough credits to run the code in this tutorial. This should only use a few cents to get started. 2 There is a template repository that you can use to get started with this tutorial, which has a pre-configured dev container. To use this, you will need to have Docker installed but won't need .NET installed.
This tutorial also assumes that you have a basic understanding of what an LLM is, and have created prompts with tools like ChatGPT before.
Overview of some relevant concepts
Before we get started, let's go over some concepts that will be useful to know. If you already know these concepts, feel free to skip ahead to creating the app.
What is RAG?
Retrieval augmented generation, or RAG, is the term for augmenting the data that goes into the LLM by retrieving data from other systems, and use that data to help the LLM generate answers to your questions. For example, if you have an app that will prompt an LLM to give you the average sentiment of a particular product, you can use RAG to augment the LLM with the customer reviews of that product by extracting the relevant data from your reviews database, then sending that data to the LLM to reason over.
How does your app implement RAG?
If you have a prompt-based app that allows the user to interact using pure text, then your app will start from a prompt that is user generated, such as "What is the average sentiment of the cuddly llama toy"
. Your app will then use some kind of LLM-powered app framework that has plugins - these are add-ons in the app that can retrieve data. This framework will use the LLM to determine which plugins it needs to call to get data, then send that data back to the LLM to reason over with an updated prompt.
Create your RAG app
This tutorial will alk you through creating your first RAG app using an SDK generated by liblab. This app will use:
- Microsoft Semantic Kernel as the AI app framework
- ChatGPT from OpenAI as the LLM
- Cat Facts API to get random cat facts
The steps you will take are:
- Use a provided template to create a new AI powered app
- Use liblab to generate an SDK for the Cat Facts API
- Create a plugin to get random cat facts and use this in your app
1. Create a new AI powered app
To get started, we have created a basic sample project that is configured to use the Microsoft Semantic Kernel as the AI app framework, and is already connected to OpenAI and has the user interaction already provided. You will start with this sample project, then add plugins using generated SDKs.
Create the app
This project is available from a template repository on GitHub.
-
Select the button below to create a new control repo using this template.
Name the repo whatever you like, such as
cat-facts-app
, then select the Create repo button. -
Open this repo. You can either clone it to your local machine, or open in a GitHub Codespace. If you open locally, or in a Codespace, you can use the included dev container to get started quickly. This includes .NET 8.0 and the liblab CLI.
-
Before you can run the app, you will need to configure your OpenAI API key in the
appsettings.json
file.-
In the
src
folder, rename the providedappsettings.json.example
file toappsettings.json
. -
Open the
appsettings.json
file and replaceOpenAI:ApiKey
with your API key.appsettings.json{
"OpenAI": {
"Key": "<Your key here>",
"ChatModel": "gpt-4"
}
} -
You can also configure the chat model in this file if you want to use a different model.
-
-
Build and run the app by running
dotnet run
from thesrc
folder. This will start the app and you can interact with through the terminal.Terminal$ dotnet run
I am an AI assistant who also knows a load of cat facts!
User > -
Ask a question and press enter to see the response.
TerminalUser > What is the airspeed velocity of an unladen swallow?
Assistant > What do you mean? African or European swallow?
Overview of the app
The app is a small console app, built around Microsoft Semantic Kernel. It has the following files:
src
├── appsettings.json
├── AppSettingsReader.cs
├── OpenAISettings.cs
├── Program.cs
└── RagWithSDKs.csproj
-
appsettings.json
- this file contains the settings for the app, including the OpenAI API key, and the model name to use for chat -
AppSettingsReader.cs
- this file contains theAppSettingsReader
class that reads theappsettings.json
file, allowing you to access the OpenAI settings, getting back anOpenAISettings
object -
OpenAISettings.cs
- this file contains theOpenAISettings
class that wraps the OpenAI settings section from theappsettings.json
file -
Program.cs
- this file contains the entry point for the app. It does the following:-
Reads the OpenAI settings from the
appsettings.json
file -
Creates a new Semantic Kernel
KernelBuilder
object that configures how your app will interact with the AI models, in this case using OpenAI chat completion - essentially ChatGPT. -
Creates a chat history object. The way you interact with the LLM in code is by sending a complete history of the conversation so far including the latest prompt from the user. This way the LLM can use previous questions and responses to help create the next response. For example you can ask
"What is the capital of England"
, then ask"How many people live there"
, and the LLM will be aware that you are referring to London, the answer to the first question, when answering the second.This history contains:
- A system prompt - this is used to give the AI some basic context and rules. In this case we are naming the chatbot
Libby the liblab llama
. Setting this system prompt is an important part of prompt engineering and can have a big impact on the quality of the responses you get back. - User prompts - these are the questions you ask the AI
- AI responses - these are the responses the AI gives back
This history will have a single system prompt, and then a user prompt and AI response for each interaction you have with the AI, ending with the latest user prompt. Although it is called the 'history', it also contains the current prompt.
- A system prompt - this is used to give the AI some basic context and rules. In this case we are naming the chatbot
-
Creates a chat completion service. The act of passing in a history and getting back a response is known as chat completion.
-
Starts a loop that will allow you to interact with the AI. This loop will:
- Get a user prompt
- Add this to the history
- Call the chat completion service with the history
- Add the AI response to the history
- Print the AI response
-
This loop ends when the user enters an empty prompt
-
-
RagWithSDKs.csproj
- the project file for the app
2. Use liblab to generate SDKs for the Cat Facts API
If you run the app and ask for a cat fact, the LLM will create one based off of whatever it learned from the internet when it was trained. This might be fine, but for our purposes, we want to limit the cat facts to a defined set. To do this, we will use the Cat Facts API to get random cat facts, so that the LLM only uses this as its cat facts data source.
Before we dig into how our app implements RAG to limit cat facts to this data source, we first need a way for it to call the cat facts API, and we will do this by generating a C# SDK that wraps this API using liblab.
The cat facts liblab config file
To make it easier to generate the cat facts SDK, the repo you created already contains a configured liblab config file.
The liblab config file provides configuration to liblab for SDK generation, such as the location of your API spec, the SDK languages you want, and other configuration options. You can read more in our config file overview.
You can find this config file in sdks/cat-facts/liblab.config.json
. This file contains the following configuration:
{
"sdkName": "CatFacts",
"specFilePath": "https://catfact.ninja/docs/api-docs.json",
"languages": [
"csharp"
],
// The API spec doesn't define the servers, so we need to provide the base URL
"baseUrl": "https://catfact.ninja",
"languageOptions": {
"csharp": {
"packageId": "liblab.Examples.CatFacts",
"sdkVersion": "0.0.1"
}
}
}
This configuration tells liblab:
- Name the SDK
CatFacts
- Use the cat facts API spec from
https://catfact.ninja/docs/api-docs.json
- Generate a C# SDK only
- The base URL for the API is
https://catfact.ninja
. The URL of the API server is normally defined in the API spec, but in this case it isn't, so we need to provide it. - Set the generated C# library's
PackageId
toliblab.Examples.CatFacts
- Set the
Version
in the generated C# library to0.0.1
Generate the cat facts SDK
To generate the cat facts SDK:
-
run the following command from the
sdks/cat-facts
folder:Terminalliblab build
-
This will generate a C# SDK in the
sdks/cat-facts/output/csharp
folder. In this folder there will be 2 projects -CatFacts
, the SDK itself, andExample
, a simple example project for the SDK.sdks/cat-facts/output/csharp
├── CatFacts
└── Example
Both the SDK and example project are .NET 6 projects. This is fine for the SDK as you can use it in any .NET project from 6 onwards, but if you want to run the example you will either need .NET 6 installed, or you can update the project to .NET 8.
3. Create a plugin to get random cat facts
We will be creating a Semantic Kernel Plugin, a class that implements one or more methods that are labelled as a KernelFunction
. These kernel functions have text decorators that tell the LLM what they can do, and return a response that the LLM can use to reason over. To implement RAG using cat facts, we will create a plugin with a kernel function that can get a random cat fact from the cat facts API.
Create the plugin
-
To use the new SDK, you first need to add a reference to it. Do this by running the following command from the
src
folder:Terminaldotnet add reference ../sdks/cat-facts/output/csharp/CatFacts/CatFacts.csproj
-
Create a new folder in the
src
folder calledPlugins
. This is where the plugin will live. -
Inside the
src/Plugins
folder, create a new file calledCatFactPlugin.cs
:src
├── Plugins
│ └── CatFactPlugin.cs
├── appsettings.json
├── AppSettingsReader.cs
├── OpenAISettings.cs
├── Program.cs
└── RagWithSDKs.csproj -
Add the
CatFactPlugin
class to this file, declaring it in theRagWithSDKs.Plugins
namespace:src/Plugins/CatFactPlugin.csnamespace RagWithSDKs.Plugins;
public class CatFactPlugin
{
} -
Add a readonly field to this class for the cat facts SDK client, and create this on declaration:
src/Plugins/CatFactPlugin.csnamespace RagWithSDKs.Plugins;
using CatFacts;
public class CatFactPlugin
{
private readonly CatFactsClient _client = new();
} -
Declare a method on the
CatFactPlugin
class to get a cat fact from the SDK and return it:src/Plugins/CatFactPlugin.cspublic class CatFactPlugin
{
private readonly CatFactsClient _client = new();
public async Task<string> GetCatFact()
{
Console.WriteLine("CatFactPlugin > Getting a cat fact from the Cat Facts API...");
var response = await _client.Facts.GetRandomFactAsync();
Console.WriteLine("CatFactPlugin > Cat fact: " + response.Fact);
return response.Fact;
}
}This method logs to the console before and after calling the SDK so that you can see that the app is using this plugin when you run it.
-
To use this method as a kernel function, you need to add 2 attributes -
Microsoft.SemanticKernel.KernelFunctionAttribute
to mark it as a kernel function, andSystem.ComponentModel.DescriptionAttribute
to provide a natural language description to the LLM of what this function actually does, so that the AI app framework can decide to use it when appropriate.Add these attributes to the
GetCatFact
method, along with the relevantusing
statements:src/Plugins/CatFactPlugin.csusing System.ComponentModel;
using Microsoft.SemanticKernel;
public class CatFactPlugin
{
...
[KernelFunction]
[Description("Gets a cat fact.")]
public async Task<string> GetCatFact()
{
...
}
}
Add the plugin to the kernel
Once the plugin is created, you need to add it to the kernel so that the AI app framework can use it. This is done in the Program.cs
file by adding it to the kernel builder before the kernel is built.
-
In the
Program.cs
file, add a using statement for theRagWithSDKs.Plugins
namespace:src/Program.csusing RagWithSDKs;
using RagWithSDKs.Plugins; -
After the
builder
is created, add theCatFactPlugin
to it:src/Program.cs// Create the kernel builder with OpenAI chat completion
var builder = Kernel.CreateBuilder().AddOpenAIChatCompletion(openAISettings.ChatModel,
openAISettings.Key);
// Add plugins to the kernel builder
builder.Plugins.AddFromType<CatFactPlugin>();This is added to the
builder
by type, and it knows how to instantiate a new instance of theCatFactPlugin
class.
Try out the cat facts plugin
Now that the plugin is added to the kernel, you can try it out by running the app and asking for a cat fact.
-
Build and run the app by running
dotnet run
from thesrc
folder. -
Ask for a cat fact:
Terminal$ dotnet run
I am an AI assistant who also knows a load of cat facts and can create images!
User > tell me a cat fact
CatFactPlugin > Getting a cat fact from the Cat Facts API...
CatFactPlugin > Cat fact: A cat's normal temperature varies around 101 degrees Fahrenheit.
Assistant > A cat's normal temperature varies around 101 degrees Fahrenheit.
User >In the console output, you can see that the plugin is being used to get the cat fact from the Cat Facts API.
-
The LLM can do more than just return the cat fact as is, you can use this fact to generate a response based off another part of the response. For example, asking for a cat fact in the style of a pirate:
TerminalUser > Give me a fact about cats in the style of a pirate
CatFactPlugin > Getting a cat fact from the Cat Facts API...
CatFactPlugin > Cat fact: A group of cats is called a clowder.
Assistant > Arr matey! Be ye knowin' that a gatherin' of meowin' seafarers,
them cats, be called a clowder? Aye, a fine group of whiskered buccaneers they be!You will see in the response that the LLM has requested a cat fact, retrieved it from the cat facts plugin, then used this fact to generate a response in the style of a pirate.
You might also notice that the prompt examples here are asking for cat facts in different ways -
"tell me a cat fact"
and"Give me a fact about cats"
. The LLM is able to reason over these prompts and the plugin description"Gets a cat fact."
to decide to use the plugin.noteOne hard part of using LLMs is prompt engineering, and this includes making sure that the description of these kernel functions is not only clear, but encompasses all the ways that a user might ask for this information. For example, if you ask
"tell me about cats"
, then the LLM might not use this plugin as it is for cat facts, and you have not asked explicitly for a fact.
Conclusion
In this tutorial, you have created an AI powered app that uses SDKs generated by liblab to provide RAG capabilities to an LLM. This is one of the powerful features of liblab - being able to quickly generate an SDK in a language of your choice, making it easier to integrate that API into your AI app.