Create RAGs in Open WebUI

Overview

This guide explains how to create Retrieval-Augmented Generation (RAG) models in Open WebUI, leveraging knowledge collections and LLM APIs connected via GRACE. It covers the process from preparing knowledge bases to configuring and deploying custom RAG models for your organization.

User Guide

Prerequisites

• Open WebUI is deployed and connected to an LLM API via GRACE.
• You have the necessary user access to the workspace and model management features.

Step 1: Prepare a Knowledge Base

1. Navigate to the Workspace menu in Open WebUI.
2. Go to Knowledge.
3. Click Add to create a new knowledge collection.
   • Enter a name and description for the collection.
   • Set the visibility:
     • Public: Accessible to all users.
     • Private: Select specific groups or teams (e.g., Legal, IT, etc.).
4. Upload documents to the collection (e.g., PDF files, text files). You can add multiple files and collections as needed.

Step 2: Create a RAG Model

1. Navigate to Models in the workspace.
2. Click Add New to create a new model.
3. Fill in the model details:
   • Name: Enter a name for your RAG model.
   • Base Model: Select the LLM model to use (as exposed by the LLM API on GRACE, e.g., Azure ChatGPT 4o).
   • Description and Tags: Optionally add for organization.
   • Image: Optionally upload an image for the model.
   • Visibility: Set to public or assign to specific groups.
4. Configure the RAG-specific settings:
   • System Prompt: Define the assistant's behavior (e.g., "You are Super Mario from Mario Bros. Act as an assistant.").
   • Attach Knowledge Base: Click Select Knowledge and choose the relevant knowledge collections and documents.
   • Optionally enable or disable features such as tools, filters, actions, vision, and citations.
   • For this example, vision is disabled, but citations are enabled so responses reference their sources.
5. Save the model to deploy your new RAG.

Step 3: Use and Manage Your RAG

• Your RAG model is now available for use within your organization, according to the visibility and access settings you defined.
• If your RAG use case is complex or requires advanced customization, consider involving a data scientist to build and expose the RAG through the LLM API on GRACE.
• Multiple RAGs can be created, each with different knowledge bases, prompts, and settings.

Summary

• RAGs in Open WebUI allow you to combine LLMs with custom knowledge collections for advanced, context-aware responses.
• All RAG creation and management is fully integrated with the GRACE platform.
• Access and visibility can be tailored to your organization's teams and needs.