A modern, self-hosted LLM chat application with RAG (Retrieval-Augmented Generation) support. Successfully deployed on Easypanel with remote LLM and embedding services integration. Inspired by AnythingLLM and OpenWebUI, but simpler and focused on quality functionality.
Privacy-First: All data stays on your infrastructure. No external data transmission except to your configured LLM/embedding services.
- Grok-like Chat UI - Clean, dark-themed interface with streaming responses
- Private Workspaces - Organize chats by topic/department with custom system prompts (private to owner + admins)
- RAG Support - Upload documents, convert to markdown, embed to vector database
- Dual Vector Store - Choose between Qdrant (hybrid search) or LanceDB (file-based)
- Advanced PDF Processing - Docling API integration with OCR, table extraction, and code detection
- Intelligent Tool Calling - LLM autonomously decides when to use web search
- Hybrid Search - Semantic + keyword (BM25) search with cross-encoder reranking
- File Upload - PDF, DOCX, TXT, MD support with automatic conversion
- REST API v1 - Developer-friendly API with API key authentication
- A/B Test Evaluator - Compare RAG system performance
- Admin Panel - User management, chat logs, API keys for debugging/development
- Remote LLM Integration - Connects to external OpenAI-compatible LLM APIs
- Remote Embeddings - Uses external embedding services for vector generation
- Easypanel Ready - Single container deployment with persistent storage
# 1. Clone the repository
git clone https://github.com/magnusfroste/private-ai-chatspace.git
cd private-ai-chatspace
# 2. Copy and configure environment
cp .env.example .env
# Edit .env with your LLM/Embedder URLs and API keys
# 3. Start the application
docker-compose up --buildAccess at http://localhost:8000
Default credentials:
- Email:
admin@localhost - Password:
changeme
Status: Successfully deployed and operational.
- Create a new App in Easypanel
- Connect to GitHub repo:
https://github.com/magnusfroste/private-ai-chatspace - Build method: Git (automatic builds)
- Set environment variables (copy from below)
- Mount persistent volume at
/data - Expose port 8000
- Deploy and access the generated URL
| Variable | Default | Description |
|---|---|---|
ADMIN_EMAIL |
admin@localhost |
Admin user email |
ADMIN_PASSWORD |
changeme |
Admin user password |
SECRET_KEY |
(random) | JWT secret key |
DEBUG |
false |
Enable debug mode |
| Variable | Default | Description |
|---|---|---|
LLM_BASE_URL |
http://172.17.0.1:8000/v1 |
OpenAI-compatible LLM API |
LLM_API_KEY |
(empty) | API key for LLM (if required) |
LLM_MODEL |
default |
Model name to use |
| Variable | Default | Description |
|---|---|---|
EMBEDDER_BASE_URL |
http://172.17.0.1:8001/v1 |
OpenAI-compatible embedder API |
EMBEDDER_API_KEY |
(empty) | API key for embedder (if required) |
EMBEDDER_MODEL |
default |
Embedder model name |
| Variable | Default | Description |
|---|---|---|
QDRANT_URL |
http://172.17.0.1:6333 |
Qdrant vector database (hybrid search support) |
LANCEDB_DIR |
/data/lancedb |
LanceDB file-based vector database directory |
VECTOR_STORE_TYPE |
qdrant |
Vector store type: "qdrant" or "lancedb" |
SEARCH_AGENT_URL |
(empty) | n8n webhook URL for web search |
DOCLING_API_URL |
(empty) | Docling API URL for advanced PDF processing |
OCR_SERVICE_URL |
(empty) | Marker API URL for PDF OCR (e.g., http://marker-api:8001) |
| Variable | Default | Description |
|---|---|---|
DEFAULT_TOP_N |
5 |
Number of document chunks to retrieve (1-20) |
DEFAULT_SIMILARITY_THRESHOLD |
0.25 |
Minimum similarity score (0.0-1.0) |
DEFAULT_USE_HYBRID_SEARCH |
true |
Use hybrid (dense + sparse) search |
DEFAULT_USE_WEB_SEARCH |
false |
Use external search agent |
DEFAULT_CHAT_MODE |
chat |
Default chat mode: "chat" or "query" |
DEFAULT_SYSTEM_PROMPT |
(see config) | Default AI instructions |
| Variable | Default | Description |
|---|---|---|
MAX_CONTEXT_TOKENS |
128000 |
Maximum context window size |
CONTEXT_HISTORY_RATIO |
0.7 |
Ratio for chat history (70%) |
CONTEXT_SYSTEM_RATIO |
0.15 |
Ratio for system prompt + RAG (15%) |
CONTEXT_USER_RATIO |
0.15 |
Ratio for user input + files (15%) |
| Variable | Default | Description |
|---|---|---|
DATABASE_URL |
sqlite+aiosqlite:////data/chatspace.db |
Database connection |
DATA_DIR |
/data |
Data directory |
DOCUMENTS_DIR |
/data/documents |
Documents directory |
ORIGINALS_DIR |
/data/documents/originals |
Original files directory |
MARKDOWN_DIR |
/data/documents/markdown |
Converted markdown directory |
┌─────────────────────────────────────────────────────────────┐
│ Private AI Chatspace Container │
│ ┌─────────────────────────────────────────────────────────┐│
│ │ FastAPI (Uvicorn) ││
│ │ - REST API (/api/*) ││
│ │ - Static files (React build) ││
│ └─────────────────────────────────────────────────────────┘│
│ │ │
│ ▼ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌──────────────┐│
│ │ SQLite/Postgres │ │ /data/documents │ │ External ││
│ │ (users, chats) │ │ (uploads, md) │ │ Services ││
│ └─────────────────┘ └─────────────────┘ │ - vLLM ││
│ │ - Embedder ││
│ │ - Qdrant ││
│ └──────────────┘│
└─────────────────────────────────────────────────────────────┘
- User - Authentication with admin/user roles
- Workspace - Isolated environments with custom system prompts
- Chat - Conversation threads within workspaces
- Message - Individual messages (user/assistant)
- Document - Uploaded files with embedding status
- ChatLog - Admin logging for all interactions
- Upload - User uploads PDF/DOCX/TXT/MD file
- Convert - File is converted to Markdown and stored
- Embed - User triggers embedding (manual or "Embed All")
- Search - On each chat message, relevant chunks are retrieved
- Augment - Context is added to the LLM prompt
POST /api/auth/login- LoginPOST /api/auth/register- Register (if enabled)GET /api/auth/me- Current user
GET /api/workspaces- List workspacesPOST /api/workspaces- Create workspacePUT /api/workspaces/{id}- Update workspaceDELETE /api/workspaces/{id}- Delete workspace
GET /api/chats/workspace/{id}- List chats in workspacePOST /api/chats- Create chatGET /api/chats/{id}/messages- Get messagesPOST /api/chats/{id}/messages- Send message (streaming)
GET /api/documents/workspace/{id}- List documentsPOST /api/documents/workspace/{id}/upload- Upload filePOST /api/documents/{id}/embed- Embed documentPOST /api/documents/workspace/{id}/embed-all- Embed all
GET /api/admin/stats- StatisticsGET /api/admin/users- List usersPOST /api/admin/users- Create userGET /api/admin/logs- Chat logs
- Python 3.10+
- Node.js 18+
- npm
cd backend
# Create virtual environment (recommended)
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
# Copy and configure environment
cp .env.example .env
# Edit .env with your LLM/Embedder URLs
# Run development server
uvicorn app.main:app --reload --port 8000cd frontend
npm install
npm run devThe frontend runs on http://localhost:5173 and proxies API calls to the backend.
Terminal 1 (Backend):
cd backend
source venv/bin/activate
uvicorn app.main:app --reload --port 8000Terminal 2 (Frontend):
cd frontend
npm run dev- Backend: FastAPI, SQLAlchemy, Python 3.10+
- Frontend: React, TypeScript, TailwindCSS
- Vector Stores: Qdrant (hybrid search), LanceDB (file-based)
- PDF Processing: Docling API (OCR, tables, code), Marker API, PyPDF2
- Search: Hybrid semantic + BM25, cross-encoder reranking
- LLM Integration: OpenAI-compatible API (vLLM, Ollama, etc.)
- Deployment: Docker, Docker Compose, Easypanel
- Paperless integration for document import
- Multiple LLM provider support
- Chat export/import
- Shared workspaces between users
- Advanced analytics dashboard
MIT