Skip to content
Agents of Humanity
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
Open sourceself-hostedOpen WebUI License (BSD-3 with branding clause; not OSI-approved)

Open WebUI

Open WebUI Inc. · openwebui.com

Open WebUI is a self-hosted AI chat and agent platform that keeps all state in the owner's own SQLite database and data volume, runs with local or any OpenAI-compatible models, and sends no telemetry by default. It is strong on portability, transparency and modifiability, but has no tamper-evident or default-on action record, no owner-rooted agent identity, and limited change history, and its approval gates and boundaries can be bypassed through automations.

Strengths
  • All state local in documented SQLite and files (P5, T1)
  • Model-independent, with local engines for embeddings, speech and images (P4)
  • No telemetry, provider data holding or remote flags (T3, M5)
  • Owner can add or remove tools and edit memory (M2, M3)
Gaps
  • Audit log off by default, with no tamper evidence (A1, A2)
  • No agent identity or state change history (V2, V3)
  • Automations bypass tool approval, and tool egress is unscoped (C1, C2)
  • Restore needs a manual volume copy plus the secret key (P1)
Evidence

All 26 findings

v0.11.4, commit 8bd8b4fac5e059578ac0c74b3c18d11139f88b7d (2026-09-21), Linux Docker image; default config (SQLite, Chroma, local storage, native function calling). Evaluated 2026-09-30. Treated as met under G11: spend (no payment capability).

Portable · Can you leave, take the whole agent with you, and keep it alive?

80%
  • P1
    Owner-usable round trip

    Admin Settings > Database offers one-click config JSON export/import and a raw webui.db download, but the documented full restore is to copy the whole /app/backend/data volume to the new host and carry over WEBUI_SECRET_KEY, which the default Docker image generates into /app/backend/.webui_secret_key outside the volume; there is no single in-app import that restores a full instance. Export is immediate and needs no provider approval.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Ship a one-action full-instance backup (DB, uploads, vector store, secret key) with a matching one-action restore.
    • Store the auto-generated secret key inside the data volume, or include it in the export.
    Partial
  • P2
    Complete export

    All agent state (chats with tool calls, memories, models/system prompts, prompts, tools and functions as Python source, skills, automations, notes, channels, calendar, config, API keys, uploads and the Chroma vector DB) lives in the owner's data volume, which the documented migration route copies whole; the documented route also covers keeping WEBUI_SECRET_KEY, which decrypts the stored OAuth sessions. Uploaded source files and extracted text sit beside the vector index, and tool/MCP OAuth uses dynamic registration or owner-entered clients with the instance's own callback, not a provider broker.

    Pass
  • P3
    Runs without the provider

    Without a LICENSE_KEY (default empty) no licence server is contacted and no openwebui.com account is needed; the GitHub update check and Hugging Face model refresh are non-fatal and can be disabled with OFFLINE_MODE. The right to run rests on the non-OSI 'Open WebUI License' (BSD-3 plus a branding clause), which does not state that the grant is perpetual or irrevocable and calls branding changes a 'material breach'. Under G12 an irrevocable right to run is not established, so the pass is withheld.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • State in the licence that the right to run and modify is perpetual and irrevocable (or use an OSI-approved licence).
    Partial
  • P4
    Model independence

    Ollama and any OpenAI-compatible endpoint are first-class connections, and embeddings (SentenceTransformers), speech-to-text (faster-whisper), text-to-speech (local transformers) and image generation (ComfyUI, AUTOMATIC1111) all have local engines. Tools run in the runtime, native tool calling works with any model that supports it, and a prompt-based 'legacy' mode covers models without native tool calling. No capability is tied to one vendor's hosted model.

    Pass
  • P5
    Exit survives termination

    In the default self-hosted install, all state is in a SQLite database, local uploads and a local vector store inside the owner's Docker volume, so a current copy is always in the owner's hands whatever the provider does.

    Pass

Transparent · Can you see everything the agent is, and everything shaping it?

88%
  • T1
    Readable without the provider

    State is stored in plain SQLite, with the schema defined by public SQLAlchemy models and alembic migrations, plus raw upload files and a Chroma store; tools and functions are stored as Python source. The only encrypted fields (OAuth session tokens, and valves if opted in) use keys derived from the owner-held WEBUI_SECRET_KEY.

    Pass
  • T2
    What runs is what you see

    Model identity, per-model tools and system prompts, and effective config values (DB overrides env when ENABLE_PERSISTENT_CONFIG is on) are visible to the admin. No view shows the fully assembled context: the default task and tool-calling templates appear in the UI only as 'Leave empty to use the default prompt', several injected wrappers (memory context, skills list, conversation summary, Pyodide prompt) are hardcoded, and the DEBUG payload log is written before the model-level system prompt is applied. One shortfall is counted: assembled context content is not fully visible.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add a per-call view or log of the exact final request sent to the model.
    • Show default template text in settings instead of an empty placeholder.
    Partial
  • T3
    No shadow data

    In the default self-hosted install the provider holds nothing: the image disables Scarf/anonymised telemetry, OpenTelemetry is off, the licence server is contacted only with a licence key, and community sharing to openwebui.com happens only when a user clicks share. The openwebui.com privacy policy covers only the provider's own website and community service.

    Pass
  • T4
    No hidden steering

    The source shows no paid placement, sponsored tools or remote announcements: there are no default models or tool servers, banners come only from local config, and web search is off by default. The enterprise/sponsor notice for instances above 50 users appears only in the admin user list and is not injected into the agent.

    Pass

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

17%
  • A1
    Complete action record

    The audit log is off by default (AUDIT_LOG_LEVEL=NONE), and even when enabled it excludes /chat, /chats and /folders by default, truncates bodies to 2048 bytes, and never sees automation runs, which bypass the HTTP middleware. Chat history does keep tool-call names and arguments, but actions inside Python tools and other side effects are not recorded separately, and there is no mechanism to alert the owner when recording fails. Counted shortfalls: unrecorded entry paths, and silent recording failure (G12).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Record every consequential action, including tool side effects and automation runs, in a durable log on by default.
    • Alert the owner, or block the action, when recording fails.
    Fail
  • A2
    Tamper-evident, owner-checkable

    Audit log entries and chat records are plain JSON lines and database rows with no hash chaining, signatures or external anchor, so they can be changed without a trace.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Hash-chain or sign action-record entries and anchor checkpoints with the owner.
    • Publish an open verifier.
    Fail
  • A3
    Agent cannot rewrite its record

    Chat history and the audit log are written by server middleware; the model's built-in tools can only search and view chats, not change them. However, Tools and Functions run as arbitrary Python inside the server process with full access to webui.db and audit.log, and the code-execution paths are not documented as isolated from the record. The protection is not documented as a guarantee.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Write the action record from a component that tools and code execution cannot reach.
    • Document the protection and how the owner can check it.
    Partial

Verifiable · Can you prove what it runs, who it is, and what changed?

17%
  • V1
    Obtainable source, matching build

    All components needed to run are public source that may be modified, and third parties can maintain forks; the branding clause restricts only rebranding above 50 users. Default Docker images are built by a public GitHub Actions workflow that emits an SBOM, but no signing, provenance attestation or reproducible-build procedure lets the owner tie the default ghcr.io image to the source.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish signed build provenance (SLSA/Sigstore) or reproducible builds for the Docker images.
    • Document how an owner verifies the image they run.
    Partial
  • V2
    Owner-rooted identity

    There is no agent identity: actions run under local user accounts, with JWTs or sk- API keys issued by the instance, and automations and subagents use short-lived JWTs for the owning user. Nothing lets a third party check that an action comes from the owner's agent.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Give the agent an owner-held key or DID and sign outbound actions.
    • Document how recipients verify that attribution.
    Fail
  • V3
    Change history and rollback

    Only the prompt library has version history, with diff and restore. Model definitions and their system prompts, config, memories, tools, skills and permissions are single rows with only an updated_at timestamp and no record of what changed or who changed it. For the agent's state as a whole, the product offers no way to see or attribute changes.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Version every class of agent state with attribution (owner, agent, import, update).
    • Offer one-step restore to an earlier state.
    Fail

Modifiable · Can you change anything, and can only you change it?

80%
  • M1
    Instructions and logic

    The admin can edit the RAG template, the task templates (title, tags, follow-up, query, autocomplete, image, tool calling, voice), the code-interpreter and compaction templates, and per-model, per-user and per-chat system prompts. Several injected instructions are hardcoded and editable only in source: memory wrappers and the memory-review prompt, the Pyodide environment prompt, skill wrappers, the conversation-summary wrapper, the emoji and MOA templates, and the built-in tool descriptions.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Expose every injected wrapper and built-in tool description as an owner-editable setting.
    Partial
  • M2
    Memory

    Settings > Personalization lists every memory item with edit and delete controls, plus delete-all. The other stores the agent reads across sessions (chats, notes, knowledge, calendar, automations) are itemised in the UI with edit and delete controls.

    Pass
  • M3
    Tools

    The owner can add Python tools, OpenAPI tool servers and MCP servers with no provider review, and every built-in tool category can be switched off per model; many also have global switches.

    Pass
  • M4
    Owner has the final say

    Most protections are owner-configurable: code-execution engine, safe mode, pip installs for plugins, file limits, and local web fetch. The built-in web-fetch blocklist (cloud metadata endpoints and reserved ranges) is always merged with the owner's list and cannot be removed without editing source.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner disable the default web-fetch blocklist with a warning.
    Partial
  • M5
    No unilateral change

    The self-hosted instance changes only when the owner pulls a new image; the docs recommend pinning :vX.Y.Z tags, and Watchtower auto-updates are optional. The only remote data the code fetches from the provider is licence data, and only when LICENSE_KEY is set; there are no remote behaviour flags. Models are the owner's own connections.

    Pass

Controllable · Is your word final?

50%
  • C1
    Boundaries

    Per-user and per-group permissions and per-tool access grants exist, but admins bypass them, and in a one-person business the owner is the admin. Web fetch and web search can be domain-filtered, but Python tools and MCP servers have no egress limits (on/off only). The model can create, update and toggle automations, which then run with tool approval forced to 'full', so the agent can route around the approval setting. Spend is treated as met under G11 (no payment capability). Counted shortfalls: on/off-only limits for tool egress, and a self-modification bypass path.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add per-domain egress policy for tools and MCP servers.
    • Stop agent-created automations from bypassing owner limits and gates, and apply per-tool scope to the admin's own agent.
    Fail
  • C2
    Approval gates

    A tool approval mode ('ask' or 'full') exists behind ENABLE_TOOL_PERMISSIONS (default off). It applies to all tools at once rather than per tool or action class, and automations and channel messages always run with 'full', bypassing it.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Allow gating per tool or action class.
    • Honour the gate on automation and channel paths.
    Partial
  • C3
    Immediate halt

    The stop button halts the current chat's tasks, and admins can stop tasks one by one, but there is no single stop-all. Scheduled automations need a separate step (the global ENABLE_AUTOMATIONS switch or per-automation toggles).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add one global stop that cancels in-flight tasks, queued work and automations until the owner resumes.
    Partial
  • C4
    Data serves only the owner

    No telemetry or content leaves by default, and sharing to openwebui.com is only per click. However, non-content outbound calls are on by default and not opt-in: the GitHub release check (ENABLE_VERSION_UPDATE_CHECK=true) and Hugging Face model auto-update at startup (RAG_EMBEDDING_MODEL_AUTO_UPDATE=true). Both can be turned off, for example with OFFLINE_MODE. There are no provider data terms for self-hosted use.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make update and model-hub checks opt-in, or default OFFLINE_MODE on.
    • Publish the full list of outbound endpoints.
    Partial
  • C5
    Credential custody

    Connection API keys are listed and removable under Admin > Connections, user API keys can be viewed and deleted, and tool/MCP OAuth sessions can be revoked per provider. All credentials are stored in the owner's own database; the provider holds none.

    Pass
  • C6
    Full deletion

    No provider-side copy exists by default, so deletion means removing the owner's Docker volume and container, which also holds the secret key file. There is no one-step in-app deletion of the whole instance, no documented full-deletion procedure, and no confirmation; in-app deletion covers individual chats, memories and non-primary users.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document or provide a one-step full-instance wipe (volume, secret key, logs) with confirmation.
    Partial
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Open WebUI (https://openwebui.com).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 15:41:51 UTC · 6 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/open-webui.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding