Open WebUI
Open WebUI Inc. · openwebui.com
Open WebUI is a self-hosted AI chat and agent platform that keeps all state in the owner's own SQLite database and data volume, runs with local or any OpenAI-compatible models, and sends no telemetry by default. It is strong on portability, transparency and modifiability, but has no tamper-evident or default-on action record, no owner-rooted agent identity, and limited change history, and its approval gates and boundaries can be bypassed through automations.
- All state local in documented SQLite and files (P5, T1)
- Model-independent, with local engines for embeddings, speech and images (P4)
- No telemetry, provider data holding or remote flags (T3, M5)
- Owner can add or remove tools and edit memory (M2, M3)
- Audit log off by default, with no tamper evidence (A1, A2)
- No agent identity or state change history (V2, V3)
- Automations bypass tool approval, and tool egress is unscoped (C1, C2)
- Restore needs a manual volume copy plus the secret key (P1)
All 26 findings
v0.11.4, commit 8bd8b4fac5e059578ac0c74b3c18d11139f88b7d (2026-09-21), Linux Docker image; default config (SQLite, Chroma, local storage, native function calling). Evaluated 2026-09-30. Treated as met under G11: spend (no payment capability).
Portable · Can you leave, take the whole agent with you, and keep it alive?
80%- P1Owner-usable round trip
Admin Settings > Database offers one-click config JSON export/import and a raw webui.db download, but the documented full restore is to copy the whole /app/backend/data volume to the new host and carry over WEBUI_SECRET_KEY, which the default Docker image generates into /app/backend/.webui_secret_key outside the volume; there is no single in-app import that restores a full instance. Export is immediate and needs no provider approval.
- Database.svelte (export/import UI)
- start.sh secret key file
- docker-compose.yaml
- Docs: Backups
- Docs: Database migration
› How it could pass
Suggestions. Any change that meets the test counts.
- Ship a one-action full-instance backup (DB, uploads, vector store, secret key) with a matching one-action restore.
- Store the auto-generated secret key inside the data volume, or include it in the export.
Partial - P2Complete export
All agent state (chats with tool calls, memories, models/system prompts, prompts, tools and functions as Python source, skills, automations, notes, channels, calendar, config, API keys, uploads and the Chroma vector DB) lives in the owner's data volume, which the documented migration route copies whole; the documented route also covers keeping WEBUI_SECRET_KEY, which decrypts the stored OAuth sessions. Uploaded source files and extracted text sit beside the vector index, and tool/MCP OAuth uses dynamic registration or owner-entered clients with the instance's own callback, not a provider broker.
Pass - P3Runs without the provider
Without a LICENSE_KEY (default empty) no licence server is contacted and no openwebui.com account is needed; the GitHub update check and Hugging Face model refresh are non-fatal and can be disabled with OFFLINE_MODE. The right to run rests on the non-OSI 'Open WebUI License' (BSD-3 plus a branding clause), which does not state that the grant is perpetual or irrevocable and calls branding changes a 'material breach'. Under G12 an irrevocable right to run is not established, so the pass is withheld.
- LICENSE
- Licence check only when LICENSE_KEY set
- LICENSE_KEY default and endpoints
- OFFLINE_MODE and update check flags
- Docs: Offline mode
- Docs: License
› How it could pass
Suggestions. Any change that meets the test counts.
- State in the licence that the right to run and modify is perpetual and irrevocable (or use an OSI-approved licence).
Partial - P4Model independence
Ollama and any OpenAI-compatible endpoint are first-class connections, and embeddings (SentenceTransformers), speech-to-text (faster-whisper), text-to-speech (local transformers) and image generation (ComfyUI, AUTOMATIC1111) all have local engines. Tools run in the runtime, native tool calling works with any model that supports it, and a prompt-based 'legacy' mode covers models without native tool calling. No capability is tied to one vendor's hosted model.
Pass - P5Exit survives termination
In the default self-hosted install, all state is in a SQLite database, local uploads and a local vector store inside the owner's Docker volume, so a current copy is always in the owner's hands whatever the provider does.
Pass
Transparent · Can you see everything the agent is, and everything shaping it?
88%- T1Readable without the provider
State is stored in plain SQLite, with the schema defined by public SQLAlchemy models and alembic migrations, plus raw upload files and a Chroma store; tools and functions are stored as Python source. The only encrypted fields (OAuth session tokens, and valves if opted in) use keys derived from the owner-held WEBUI_SECRET_KEY.
Pass - T2What runs is what you see
Model identity, per-model tools and system prompts, and effective config values (DB overrides env when ENABLE_PERSISTENT_CONFIG is on) are visible to the admin. No view shows the fully assembled context: the default task and tool-calling templates appear in the UI only as 'Leave empty to use the default prompt', several injected wrappers (memory context, skills list, conversation summary, Pyodide prompt) are hardcoded, and the DEBUG payload log is written before the model-level system prompt is applied. One shortfall is counted: assembled context content is not fully visible.
- Memory context wrapper
- Injected wrappers
- Empty-default templates
- Placeholder in admin UI
- System prompt applied late
- Persistent config precedence
- Docs: Logging
› How it could pass
Suggestions. Any change that meets the test counts.
- Add a per-call view or log of the exact final request sent to the model.
- Show default template text in settings instead of an empty placeholder.
Partial - T3No shadow data
In the default self-hosted install the provider holds nothing: the image disables Scarf/anonymised telemetry, OpenTelemetry is off, the licence server is contacted only with a licence key, and community sharing to openwebui.com happens only when a user clicks share. The openwebui.com privacy policy covers only the provider's own website and community service.
Pass - T4No hidden steering
The source shows no paid placement, sponsored tools or remote announcements: there are no default models or tool servers, banners come only from local config, and web search is off by default. The enterprise/sponsor notice for instances above 50 users appears only in the admin user list and is not injected into the agent.
Pass
Auditable · Can you reconstruct exactly what the agent did, and trust the record?
17%- A1Complete action record
The audit log is off by default (AUDIT_LOG_LEVEL=NONE), and even when enabled it excludes /chat, /chats and /folders by default, truncates bodies to 2048 bytes, and never sees automation runs, which bypass the HTTP middleware. Chat history does keep tool-call names and arguments, but actions inside Python tools and other side effects are not recorded separately, and there is no mechanism to alert the owner when recording fails. Counted shortfalls: unrecorded entry paths, and silent recording failure (G12).
› How it could pass
Suggestions. Any change that meets the test counts.
- Record every consequential action, including tool side effects and automation runs, in a durable log on by default.
- Alert the owner, or block the action, when recording fails.
Fail - A2Tamper-evident, owner-checkable
Audit log entries and chat records are plain JSON lines and database rows with no hash chaining, signatures or external anchor, so they can be changed without a trace.
› How it could pass
Suggestions. Any change that meets the test counts.
- Hash-chain or sign action-record entries and anchor checkpoints with the owner.
- Publish an open verifier.
Fail - A3Agent cannot rewrite its record
Chat history and the audit log are written by server middleware; the model's built-in tools can only search and view chats, not change them. However, Tools and Functions run as arbitrary Python inside the server process with full access to webui.db and audit.log, and the code-execution paths are not documented as isolated from the record. The protection is not documented as a guarantee.
› How it could pass
Suggestions. Any change that meets the test counts.
- Write the action record from a component that tools and code execution cannot reach.
- Document the protection and how the owner can check it.
Partial
Verifiable · Can you prove what it runs, who it is, and what changed?
17%- V1Obtainable source, matching build
All components needed to run are public source that may be modified, and third parties can maintain forks; the branding clause restricts only rebranding above 50 users. Default Docker images are built by a public GitHub Actions workflow that emits an SBOM, but no signing, provenance attestation or reproducible-build procedure lets the owner tie the default ghcr.io image to the source.
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish signed build provenance (SLSA/Sigstore) or reproducible builds for the Docker images.
- Document how an owner verifies the image they run.
Partial - V2Owner-rooted identity
There is no agent identity: actions run under local user accounts, with JWTs or sk- API keys issued by the instance, and automations and subagents use short-lived JWTs for the owning user. Nothing lets a third party check that an action comes from the owner's agent.
› How it could pass
Suggestions. Any change that meets the test counts.
- Give the agent an owner-held key or DID and sign outbound actions.
- Document how recipients verify that attribution.
Fail - V3Change history and rollback
Only the prompt library has version history, with diff and restore. Model definitions and their system prompts, config, memories, tools, skills and permissions are single rows with only an updated_at timestamp and no record of what changed or who changed it. For the agent's state as a whole, the product offers no way to see or attribute changes.
› How it could pass
Suggestions. Any change that meets the test counts.
- Version every class of agent state with attribution (owner, agent, import, update).
- Offer one-step restore to an earlier state.
Fail
Modifiable · Can you change anything, and can only you change it?
80%- M1Instructions and logic
The admin can edit the RAG template, the task templates (title, tags, follow-up, query, autocomplete, image, tool calling, voice), the code-interpreter and compaction templates, and per-model, per-user and per-chat system prompts. Several injected instructions are hardcoded and editable only in source: memory wrappers and the memory-review prompt, the Pyodide environment prompt, skill wrappers, the conversation-summary wrapper, the emoji and MOA templates, and the built-in tool descriptions.
› How it could pass
Suggestions. Any change that meets the test counts.
- Expose every injected wrapper and built-in tool description as an owner-editable setting.
Partial - M2Memory
Settings > Personalization lists every memory item with edit and delete controls, plus delete-all. The other stores the agent reads across sessions (chats, notes, knowledge, calendar, automations) are itemised in the UI with edit and delete controls.
Pass - M3Tools
The owner can add Python tools, OpenAPI tool servers and MCP servers with no provider review, and every built-in tool category can be switched off per model; many also have global switches.
Pass - M4Owner has the final say
Most protections are owner-configurable: code-execution engine, safe mode, pip installs for plugins, file limits, and local web fetch. The built-in web-fetch blocklist (cloud metadata endpoints and reserved ranges) is always merged with the owner's list and cannot be removed without editing source.
› How it could pass
Suggestions. Any change that meets the test counts.
- Let the owner disable the default web-fetch blocklist with a warning.
Partial - M5No unilateral change
The self-hosted instance changes only when the owner pulls a new image; the docs recommend pinning :vX.Y.Z tags, and Watchtower auto-updates are optional. The only remote data the code fetches from the provider is licence data, and only when LICENSE_KEY is set; there are no remote behaviour flags. Models are the owner's own connections.
Pass
Controllable · Is your word final?
50%- C1Boundaries
Per-user and per-group permissions and per-tool access grants exist, but admins bypass them, and in a one-person business the owner is the admin. Web fetch and web search can be domain-filtered, but Python tools and MCP servers have no egress limits (on/off only). The model can create, update and toggle automations, which then run with tool approval forced to 'full', so the agent can route around the approval setting. Spend is treated as met under G11 (no payment capability). Counted shortfalls: on/off-only limits for tool egress, and a self-modification bypass path.
› How it could pass
Suggestions. Any change that meets the test counts.
- Add per-domain egress policy for tools and MCP servers.
- Stop agent-created automations from bypassing owner limits and gates, and apply per-tool scope to the admin's own agent.
Fail - C2Approval gates
A tool approval mode ('ask' or 'full') exists behind ENABLE_TOOL_PERMISSIONS (default off). It applies to all tools at once rather than per tool or action class, and automations and channel messages always run with 'full', bypassing it.
› How it could pass
Suggestions. Any change that meets the test counts.
- Allow gating per tool or action class.
- Honour the gate on automation and channel paths.
Partial - C3Immediate halt
The stop button halts the current chat's tasks, and admins can stop tasks one by one, but there is no single stop-all. Scheduled automations need a separate step (the global ENABLE_AUTOMATIONS switch or per-automation toggles).
› How it could pass
Suggestions. Any change that meets the test counts.
- Add one global stop that cancels in-flight tasks, queued work and automations until the owner resumes.
Partial - C4Data serves only the owner
No telemetry or content leaves by default, and sharing to openwebui.com is only per click. However, non-content outbound calls are on by default and not opt-in: the GitHub release check (ENABLE_VERSION_UPDATE_CHECK=true) and Hugging Face model auto-update at startup (RAG_EMBEDDING_MODEL_AUTO_UPDATE=true). Both can be turned off, for example with OFFLINE_MODE. There are no provider data terms for self-hosted use.
› How it could pass
Suggestions. Any change that meets the test counts.
- Make update and model-hub checks opt-in, or default OFFLINE_MODE on.
- Publish the full list of outbound endpoints.
Partial - C5Credential custody
Connection API keys are listed and removable under Admin > Connections, user API keys can be viewed and deleted, and tool/MCP OAuth sessions can be revoked per provider. All credentials are stored in the owner's own database; the provider holds none.
Pass - C6Full deletion
No provider-side copy exists by default, so deletion means removing the owner's Docker volume and container, which also holds the secret key file. There is no one-step in-app deletion of the whole instance, no documented full-deletion procedure, and no confirmation; in-app deletion covers individual chats, memories and non-primary users.
› How it could pass
Suggestions. Any change that meets the test counts.
- Document or provide a one-step full-instance wipe (volume, secret key, logs) with confirmation.
Partial
How this assessment was produced
One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Open WebUI (https://openwebui.com).
During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.
- Model
- claude-opus-5-5
- Harness
- Claude Code 2.1.285
- Ran
- 2026-09-30 15:41:51 UTC · 6 turns
- Criteria hash
- a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
- Full transcript
- Download (JSONL, gzip) · every tool call, page read and reply
- Run protocol
- 2026-09-30-v1
Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.
- Human review
- pending
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/open-webui.json.
Or grade it from scratch. See Check any score yourself.
Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.