Unified AI Interface: Self-hosting LiteLLM and OpenWebUI

In the booming era of AI, we often find ourselves jumping between different services: ChatGPT for writing, Claude for analysis, Gemini for search... Managing dozens of API keys and disjointed interfaces is truly a nightmare.

My solution? Self-host a centralized Unified AI Gateway right at Homelab.

The Perfect Duo: LiteLLM + OpenWebUI

1. LiteLLM - "One API to rule them all"

LiteLLM acts as an intermediary Proxy Server. It allows you to call over 100+ different LLM services (OpenAI, Azure, VertexAI, Bedrock...) through a single standard OpenAI format.

• Benefit: You only need to write code once following OpenAI standard, then you can switch models freely (from GPT-4 to Claude 3 then to Llama 3) just by changing model name in config.
• Features: Load balancing, Budget management, Logs...

2. OpenWebUI - Top-tier User Interface

OpenWebUI (formerly Ollama WebUI) provides a web interface that is extremely beautiful, user-friendly, and identical to ChatGPT Plus.

• Features: Chat history, file/image upload support (RAG), Function Calling support, and notably full compatibility with OpenAI standard API (meaning perfect compatibility with LiteLLM).

Deployment with Docker Compose

Here is the "magical" docker-compose.yml file to set up both services in a snap:

version: '3.8'

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    ports:
      - "4000:4000"
    volumes:
      - ./litellm-config.yaml:/app/config.yaml
    environment:
      - DATABASE_URL=postgresql://... # (Optional if you want to store logs)
    command: [ "--config", "/app/config.yaml", "--port", "4000", "--detailed_debug"]

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    ports:
      - "3000:8080"
    volumes:
      - open-webui-data:/app/backend/data
    environment:
      - OPENAI_API_BASE_URL=http://litellm:4000
      - OPENAI_API_KEY=sk-1234 # Fake key because LiteLLM manages real keys
    depends_on:
      - litellm

volumes:
  open-webui-data:

Configuring litellm-config.yaml

This is where the magic happens. You declare models and real API keys here:

model_list:
  - model_name: gpt-4-turbo
    litellm_params:
      model: openai/gpt-4-turbo
      api_key: sk-proj-...

  - model_name: claude-3-opus
    litellm_params:
      model: anthropic/claude-3-opus-20240229
      api_key: sk-ant-...

  - model_name: gemini-pro
    litellm_params:
      model: vertex_ai/gemini-pro
      # ... credential setup

Result

After running docker-compose up -d, visit localhost:3000.

• Register the first admin account.
• In the chat frame, you will see a Dropdown list to select models: gpt-4-turbo, claude-3-opus, gemini-pro... all in a single interface!

This system not only helps manage API keys centrally (avoiding hardcoding keys into apps) but also makes it easy for me to share AI access with family members or project teams without revealing the root key.