Getting Started

QvikChat is a framework built on top of Firebase Genkit and LangChain. It allows you to quickly create and deploy a chat service, with support for advance features such as safety guardrails, authentication, response caching, chat history, Retrieval Augmented Generation (RAG) and more.

If you wish to get started quickly with minimal setup or if you are planning to build a standalone server to serve chat endpoints, its recommended that you start with the QvikChat Starter Template. It comes pre-configured with support for TypeScript, ESLint, Prettier, Jest, SWC, and more.

How it works?

Each chat service is defined by defining a chat endpoint. The chat endpoint that you define, is the code that gets called when a new request is received at that endpoint. You can define multiple chat services, each with its own endpoint. Each chat endpoint can have various attributes like input data schema, LLM model to be used for processing queries, authentication policy, etc. You defines these attributes when creating the configurations for the chat endpoint.

This is the real superpower of QvikChat. You can define a chat endpoint with support for chat history, authentication, response caching, RAG, and more, in just a few lines of code. You can even store these configurations remotely and load them dynamically at runtime.

There are two ways this can be done:

Using the configureAndRunServer method: This is the easiest way to get started with QvikChat. You can provide a list of endpoint configurations to the configureAndRunServer method, and it will start the server with all the defined chat endpoints. To learn more about the configureAndRunServer method and the different configurations that you can provide, check the Configure and Run Server section.
Using the defineChatEndpoint method: You can directly call the defineChatEndpoint method with the configurations for the chat endpoint, anywhere in your codebase. This is the method that gets called under the hood when you provide the list of endpoint configurations to the configureAndRunServer method. When using the defineChatEndpoint method, you will have to manually ensure that you setup Firebase Genkit, and that all endpoints are defined before you start the server. To learn more about defineChatEndpoint method check the Define Chat Endpoint section.

In this guide, we are going to use the configureAndRunServer method to define chat endpoints and start the server.

Installing QvikChat Package

You can install QvikChat using the following command:

npm install @oconva/qvikchat

pnpm add @oconva/qvikchat

Setup Environment Variables

By default, QvikChat uses Google's Gemini API for text generation and embedding models. If you don't yet have a Google Gen AI API key, you can get one from Gemini API - Get an API Key (opens in a new tab).

You can also use OpenAI API instead of Gemini API. You'll have to provide your OpenAI API key as the OPENAI_API_KEY environment variable and configure your chat endpoints to use the LLM model you want it to use.

QvikChat being built on top of Firebase Genkit, allows you to easily add Genkit plugins. You can easily use any LLM model available through any Genkit plugin by simply configuring that plugin when setting up Genkit.

To learn more about configuring chat endpoints to use a specific LLM model, check here.

If using the default Gemini API models or OpenAI models, there should be a .env file at the root level of your project directory, and it should have a value for at least one of the following, depending on which API you want to use:

GOOGLE_GENAI_API_KEY=
OPENAI_API_KEY=

Alternatively, you can copy the .env.tmp file or rename it to .env and fill in the values.

Usage

To use QvikChat, the typical workflow would look something like this:

Define Endpoint Configurations: Each chat service listens for incoming queries at a specific server endpoint. First step is to write the configurations for this chat endpoint. Through these configurations you can easily enable or disable features like chat history, cache, authentication, and RAG. You can also define the LLM model to be used for processing queries, besides other configurations. To learn more about the different configurations that you can provide, check the Chat Endpoint Configurations section.
Configure and Run Server: Once you have defined the chat endpoint configurations, you can use the configureAndRunServer method to start the server, passing a list of all endpoint configurations as an argument. Optionally, you can also provide configurations for the server (e.g. port number, cors, etc.), and the configurations for the Firebase Genkit (e.g. plugins you want to enable). To learn more about the configureAndRunServer method and the different configurations that you can provide, check the Configure and Run Server section.

The below code shows a simple example of defining an open-ended chat endpoint in QvikChat. An open-ended chat endpoint basically allows queries related to any topic, i.e., it doesn't place any restrictions on what context the queries should be related to.

index.ts

import { configureAndRunServer, ChatEndpointConfig } from "@oconva/qvikchat";
 
// Configurations for an open-ended chat endpoint
const endpointConfig: ChatEndpointConfig = {
  endpoint: "chat",
};
 
// Configure and run the server
configureAndRunServer({
  endpointConfigs: [endpointConfig],
});

Running the above code will run a Express (opens in a new tab) server with your defined chat endpoint accessible through it.

You should be able to access the chat endpoint defined above at the chat endpoint. To test from terminal, you can try the below command:

curl -X POST "http://127.0.0.1:3400/chat" -H "Content-Type: application/json"  -d '{"data": { "query": "Answer in one sentence: What is Firebase Firestore?" } }'

Above example points to http://127.0.0.1:3400. You can change this port and host depending on where you are running the server and on which port.

QvikChat Starter Template

To get up and running quickly, you can use the QvikChat starter template. The starter template is a pre-configured project with all the necessary configurations and setup to get you started with QvikChat quickly while ensuring code quality and reliability. It comes pre-configured with support for TypeScript, ESLint, Prettier, Jest, SWC, and more, so you can get started with developing the next revolutionary chat service right away.

Setup

Simply, clone the QvikChat starter template (opens in a new tab) to get started.

git clone https://github.com/oconva/qvikchat-starter-template.git

Environment Variables

Once you have cloned the starter template, add the API keys required to access LLM models. By default, QvikChat uses Google's Gemini API for text generation and embedding models. If you don't yet have a Google Gen AI API key, you can get one from Gemini API - Get an API Key (opens in a new tab).

To learn more about configuring chat endpoints to use a specific LLM model, check here.

GOOGLE_GENAI_API_KEY=
OPENAI_API_KEY=

Usage

To use QvikChat, the typical workflow would look something like this:

Define Endpoint Configurations: Each chat service listens for incoming queries at a specific server endpoint. First step is to write the configurations for this chat endpoint. Through these configurations you can easily enable or disable features like chat history, cache, authentication, and RAG. You can also define the LLM model to be used for processing queries, besides other configurations.
Configure and Run Server: Once you have defined the chat endpoint configurations, you can use the configureAndRunServer method to start the server, passing a list of all endpoint configurations as an argument. Optionally, you can also provide configurations for the server (e.g. port number, cors, etc.), and the configurations for the Firebase Genkit (e.g. plugins you want to enable).

Running the Project

QvikChat starter template comes pre-defined with some chat endpoints. These endpoints are defined in the src/endpoints directory. We setup Genkit and run the server in the src/index.ts file.

You can run the following commands to install the dependencies:

npm install # or pnpm install

To compile the project code and start the server, run:

npm run dev # or pnpm dev

Check the testing endpoints section to learn how to test endpoints.

To learn more about the QvikChat starter template, check the QvikChat Starter Template (opens in a new tab) repo.

Testing Endpoints

You can test the pre-defined test endpoints to see how the chat endpoints work and to confirm QvikChat setup. You can do this either using a graphical interface or by running the server locally and testing the endpoints using the terminal.

Genkit Developer UI

You can run the Genkit developer UI to test the endpoints. Testing the endpoints using a graphical interface is probably the easiest way to get started. You can know more about the Genkit Developer UI here (opens in a new tab).

Start the Genkit developer UI:

npx genkit start

OR, you can install the Genkit CLI globally:

npm i -g genkit

Then start the Genkit developer UI:

genkit start

You should be able to see your defined chat endpoints under the Flows section in the left sidebar. Simply click on the endpoint you want to test and enter the query you want to test with. Clicking the Run button will send the query to the endpoint and the response generation process will start.

Running the Server

You can also run the server locally to test the endpoints from their REST endpoints.

Before you can do this, you will need to first compile the TypeScript code.

Compile the TypeScript code. You can modify .swcrc to change the SWC configurations and package.json to adjust the build command.

npm run build

pnpm build

Then, start the server:

npm run start

pnpm start

Depending on which endpoint you wish to test, and where and on which port your server starts, you should be able to access the chat endpoints through the terminal using the curl command. The below given example sends the query to the chat endpoint:

curl -X POST "http://127.0.0.1:3400/chat" -H "Content-Type: application/json"  -d '{"data": { "query": "Answer in one sentence: What is Firebase Firestore?" } }'

Introduction Core