Overview
ODS (Osmantic Deployment System) is an open-source platform that turns your PC, Mac, or Linux box into a private AI server. It installs and wires together everything needed to run AI locally, including local model inference, chat interfaces, dashboards, voice agents, workflows, RAG, and privacy tools.
Key Features
- Local Model Inference: Run open models on your own hardware with llama-server and LiteLLM integration
- Chat Interface: Full-featured chat UI via Open WebUI with conversation history and web search
- Control Dashboard: Manage models, services, setup, GPU status, and extensions from one centralized place
- Voice, Agents & Workflows: Build automations that can listen, speak, call tools, and perform tasks
- RAG and Search: Connect local documents with private search and retrieval workflows
- Image Generation: Run local image tools without sending prompts to hosted APIs
- Privacy First: No cloud required – inference, chat history, and files stay on your machine

One-Command Installation
ODS supports multiple platforms with simple installation commands:
- Linux/macOS:
curl -fsSL https://install.osmantic.com/ods.sh | bash - Windows PowerShell: Guided Ubuntu/WSL2 setup with Pixel/Portal integration
- Manual Install:
git clone https://github.com/Osmantic/ODS.git && cd ODS/ods && ./install.sh
Bundled Services
ODS includes a comprehensive stack of AI services out of the box:
- Chat & Inference: Open WebUI, llama-server, LiteLLM, TEI Embeddings
- Voice: Whisper (speech-to-text), Kokoro (text-to-speech)
- Agents & Automation: Portal agent, Hermes Agent, n8n, APE, OpenCode, Memory Shepherd
- Knowledge & Search: Qdrant (vector DB), SearXNG, Perplexica, Brave Search
- Creative: ComfyUI for image generation
- Privacy & Ops: Privacy Shield, Token Spy, Langfuse observability
Hardware Support
ODS auto-detects your GPU and selects the best model for your hardware:
- NVIDIA: RTX 4060 to RTX PRO 6000 with Qwen models (4-24+ GB VRAM)
- AMD: Strix Halo with unified memory support
- Apple Silicon: M1+ chips with native Metal acceleration
- Intel Arc: Experimental support (Linux SYCL)
Extensibility
Every service in ODS is an extension – just a folder with a manifest.yaml and compose.yaml. Enable extensions with ods enable my-service and disable them when not needed. The entire system is built to be modded.
CLI Management
The ods CLI manages your entire stack:
ods status # Health checks + GPU status
ods list # All services and their state
ods restart [service] # Restart one or all services
ods mode cloud # Switch to cloud APIs via LiteLLM
ods model swap T3 # Switch to a different hardware tier