AnythingLLM
Mintplex Labs · anythingllm.com
AnythingLLM Desktop is a local-first workspace and agent app. All state sits in a documented storage folder on the owner's machine, it runs on local open-weight models by default, and the owner can add custom skills, MCP servers and flows without review. It has no documented whole-agent export or restore, no durable tool-call record with integrity protection, and no agent identity. The Desktop app itself ships under a proprietary licence that forbids modification, while its server core is MIT.
- Local-first: all agent state stays in an owner-held folder (P5)
- Runs on local open-weight models by default, with a text-based tool-calling fallback (P4)
- Owner can add skills, MCP tools and flows without review, and disable built-in skills (M3)
- No auto-update or remote flags; owner-initiated updates act as a version pin (M5)
- No documented export/import of the whole agent; only chat logs can be exported (P1, P2)
- Agent tool calls and their arguments are not durably recorded, and no record has integrity protection (A1, A2)
- Desktop wrapper is closed source under a no-modification licence (V1)
- Telemetry with a stable instance ID is on by default, with no disclosed retention period (T3, C4)
All 26 findings
AnythingLLM Desktop v1.16.2 (released 2026-09-22). Open-source server core read at commit 067806d4bf8211699265f82302fa07a5722ea461 (server package version 1.16.2) of github.com/Mintplex-Labs/anything-llm. The Desktop Electron wrapper is not public and could not be inspected. Desk review on Linux, 2026-09-30. First-run defaults per docs: built-in local LLM (Ollama engine, labelled 'Built-in (default)'), native embedder (all-MiniLM-L6-v2), LanceDB. Agent web browsing is on by default with You.com search.
Portable · Can you leave, take the whole agent with you, and keep it alive?
50%- P1Owner-usable round trip
The only documented exports are chat logs (CSV/JSON/JSONL) and per-chat exports. Desktop documentation gives the storage folder location but no documented backup, export, import or migration procedure for the whole instance or a workspace, so there is no documented export of the agent.
- AnythingLLM Docs: Where is my data stored? (Desktop)
- AnythingLLM Docs: Chat logs
- Source: chat export formats (convertTo.js)
- GitHub issue #3087: choose storage location (open)
› How it could pass
Suggestions. Any change that meets the test counts.
- Add an in-app export of the whole instance or workspace and a matching import
- Document a restore procedure that leaves the agent working in one action
Fail - P2Complete export
The default export route covers conversation text only. Configuration, prompts, memories, custom skills, flows, scheduled jobs, documents, embeddings and integration settings are not in any export; they remain only in the local storage folder.
- AnythingLLM Docs: Chat logs
- AnythingLLM Docs: Where is my data stored? (Desktop)
- Source: export-chats endpoint
› How it could pass
Suggestions. Any change that meets the test counts.
- Include every class of state (settings, prompts, memories, skills, flows, jobs, documents with source text) in an export bundle with a manifest
- Let integrations be re-linked from the bundle
Fail - P3Runs without the provider
The free Desktop app needs no account or licence check, and the self-hosted terms say it needs no connection to Mintplex servers (the embedder has a Hugging Face primary source with a Mintplex CDN fallback). However, the Desktop Terms of Use grant only a non-exclusive, non-transferable licence that they do not state is perpetual or irrevocable, so public evidence does not establish an irrevocable right to run the distributed Desktop binary (G12). The MIT server core offers another route, but no documented migration path onto it exists.
- AnythingLLM Desktop Terms of Use (v. July 14, 2025)
- TERMS_SELF_HOSTED.md
- Source: native embedder model download with CDN fallback
› How it could pass
Suggestions. Any change that meets the test counts.
- State in the Desktop terms that the licence to run a downloaded version is perpetual and irrevocable
- Or document running the MIT core as a drop-in replacement for Desktop's storage folder
Partial - P4Model independence
The default LLM is a built-in local engine (Ollama). Many local providers are supported (Ollama, LM Studio, LocalAI, etc.), and so are local embedders, speech-to-text, text-to-speech and image generators (Ollama, LocalAI, Lemonade). Agent tools run in the runtime, and an 'UnTooled' prompt-based fallback covers models without native tool calling. The only model-locked feature found is a beta-preview computer-use feature fixed to an Anthropic model, which is not part of the evaluated default.
Pass - P5Exit survives termination
Desktop needs no account. All state (anythingllm.db, documents, vector DB, plugins, .env) lives in a local storage folder on the owner's device, so a current copy is always in the owner's hands whatever happens to the provider.
Pass
Transparent · Can you see everything the agent is, and everything shaping it?
63%- T1Readable without the provider
State is kept in SQLite with a public Prisma schema, JSON documents and vector cache, LanceDB (an open format), JSON flow definitions, JavaScript skill source and a plaintext .env, all in an owner-held folder. Built-in skill and prompt logic is published as MIT source, and no provider-held keys are involved.
Pass - T2What runs is what you see
The UI shows the workspace system prompt, the model, and enabled skills and MCP tools. Some context additions can be seen only in source code: the agent role prompt, the appended clarifying-questions instruction, the UnTooled function-picker prompt and the memory header. No view or log of the fully assembled context was found, so G3 caps this test at partial.
- Source: agent defaults (role and clarifying-question prompts)
- Source: memory header injection
- AnythingLLM Docs: System prompt variables
› How it could pass
Suggestions. Any change that meets the test counts.
- Add a view or log of the full assembled context for each model call
- Surface hardcoded prompt additions in the UI
Partial - T3No shadow data
On by default, the app sends usage events (chat sent, agent tool name, documents uploaded, LLM provider and model, etc.) to PostHog with a stable per-instance UUID. The Desktop privacy policy does not disclose how long this data is kept. The README says sent events appear in the local output log, but retention is undisclosed.
- AnythingLLM Desktop Privacy Policy
- Source: telemetry model (PostHog, telemetry_id)
- README: Telemetry & Privacy
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish a bounded retention period for telemetry and any other data held
- Or keep telemetry off by default so the provider holds nothing
Fail - T4No hidden steering
The one provider-chosen default found is You.com as the agent web-search provider, which the changelog and blog disclose as a partnership. The owner can switch the search provider or disable web browsing. Source code shows no sponsored or paid placement in agent context or tool defaults. The Community Hub pages could not be opened (HTTP 403).
Pass
Auditable · Can you reconstruct exactly what the agent did, and trust the record?
17%- A1Complete action record
Interactive agent sessions store only the final response in workspace_chats. Tool calls and their arguments go only to the live websocket and are not stored (scheduled-job runs do store a full trace). No mechanism tells the owner when recording fails. Two shortfalls apply: entries lack arguments and targets, and recording can fail silently.
- Source: chat-history plugin (what is persisted)
- Source: scheduled job run persistence
- AnythingLLM Docs: Event logs
› How it could pass
Suggestions. Any change that meets the test counts.
- Persist every tool call with its arguments and target, from every entry path
- Tell the owner when an action could not be recorded
Fail - A2Tamper-evident, owner-checkable
Chats, event logs, invocations and job runs are plain SQLite rows with no hashing, signing or chaining. An admin endpoint can clear all event logs.
› How it could pass
Suggestions. Any change that meets the test counts.
- Hash-chain or sign action-record entries
- Anchor checkpoints with the owner and publish an open verifier
Fail - A3Agent cannot rewrite its record
Built-in default skills cannot write the chat or event tables. The SQL agent connects only to external databases, and the filesystem agent is sandboxed and available only in Docker. However, owner-enabled custom skills (in-process Node handlers) and stdio MCP servers run arbitrary code with access to anythingllm.db, and nothing separates the record from them. The protection holds by default except for this one path.
› How it could pass
Suggestions. Any change that meets the test counts.
- Write the record from a component that skills and MCP processes cannot reach
- Document the protection
Partial
Verifiable · Can you prove what it runs, who it is, and what changed?
33%- V1Obtainable source, matching build
The Desktop Electron wrapper, a required component of the evaluated offering, has no public source. Its Terms of Use call the software's source code a trade secret and forbid modification and reverse engineering. The server core is MIT, and release binaries ship without checksums or provenance.
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish the Desktop wrapper source under a licence that allows modification
- Provide reproducible builds or verifiable provenance for release binaries
Fail - V2Owner-rooted identity
The agent has no identity of its own. Outbound actions go through channels the owner holds, such as a Gmail Apps Script deployment the owner runs or an Outlook OAuth app the owner registers, with no Mintplex client or broker. Attribution is therefore per channel only.
- Source: Gmail agent (owner Apps Script deployment)
- Source: Outlook agent (owner Azure app)
- AnythingLLM Docs: Gmail agent
› How it could pass
Suggestions. Any change that meets the test counts.
- Give the agent an owner-held key or identity that recipients can check across channels
Partial - V3Change history and rollback
Workspace system prompt changes are kept in a prompt history with author and one-click restore, and the event log records that the LLM, embedder or vector DB changed (without values). Other settings, skills, memories and model changes have no change history or restore.
› How it could pass
Suggestions. Any change that meets the test counts.
- Log and attribute every state change (settings, skills, memories, model)
- Offer restore to a chosen earlier state
Partial
Modifiable · Can you change anything, and can only you change it?
70%- M1Instructions and logic
The workspace system prompt and the instance default system prompt can be edited in the UI, and custom skills and flows are owner-defined. Several hardcoded instructions (agent role prompt, clarifying-questions instruction, UnTooled tool-selection prompt, memory header) can be changed only by editing source.
› How it could pass
Suggestions. Any change that meets the test counts.
- Expose every hardcoded agent instruction as an owner-editable setting
Partial - M2Memory
Memories (workspace and global) are listed with create, edit and delete controls. However, the default-on rag-memory skill's 'store' action embeds text straight into the workspace vector DB with no document record, so those items are not itemised or individually editable in the UI.
- AnythingLLM Docs: Memories
- Source: memory endpoints (GET/POST/PUT/DELETE)
- Source: rag-memory store action
› How it could pass
Suggestions. Any change that meets the test counts.
- List agent-stored vector memories as items with edit and delete controls
Partial - M3Tools
The owner can add custom skills (plugins/agent-skills handler.js), MCP servers and Agent Flows without provider review. Every built-in skill, including the four default-on ones, can be disabled, and individual MCP tools can be suppressed.
Pass - M4Owner has the final say
Some protections are configurable (tool-call chain limit, approval timeout, MCP cooldown, auto-approved skills), but on Desktop only through environment variables that the docs tell Desktop users not to edit. Others are hardcoded: the 100-round agent limit, the 5-minute agent websocket timeout, and the memory count limits.
› How it could pass
Suggestions. Any change that meets the test counts.
- Expose every protection as a documented Desktop setting with a warning
Partial - M5No unilateral change
Desktop updates are owner-initiated reinstalls (an in-app banner only links to downloads), and no remote flag or server-side configuration code was found. The default model is local. Keeping the installed version acts as an indefinite pin.
Pass
Controllable · Is your word final?
42%- C1Boundaries
Contact and web reach are on/off only: the Gmail/Outlook skills and web browsing/scraping have no per-recipient or per-domain allow or deny lists. There is also no owner control over which sources the agent takes instructions from. Spend is treated as met under G11 because there is no payment capability. File scope has per-folder grants, but only in Docker.
› How it could pass
Suggestions. Any change that meets the test counts.
- Add per-recipient and per-domain allow/deny lists for messaging and web tools
- Let the owner limit which sources may instruct the agent
Fail - C2Approval gates
Approval prompts exist only for a fixed set of built-in write actions (filesystem writes, file creation, Gmail/Outlook/Calendar writes, job creation) and for custom skills that request them. MCP tools, SQL, web and flows cannot be gated, and scheduled-job runs auto-approve every tool.
- Source: requestToolApproval
- Source: scheduled job auto-approval
- AnythingLLM Docs: custom skill handler.js
› How it could pass
Suggestions. Any change that meets the test counts.
- Let the owner gate any tool, including MCP and flows
- Make scheduled runs honour gates, for example by queuing actions for approval
Partial - C3Immediate halt
The stop button fully aborts the foreground agent session and inference (v1.16.0). Scheduled jobs must be stopped per run or disabled separately, and background jobs such as memory extraction keep running. There is no single global stop.
- AnythingLLM changelog v1.16.0
- AnythingLLM Docs: Scheduled jobs, viewing runs
- Source: background workers
› How it could pass
Suggestions. Any change that meets the test counts.
- Add one global stop that cancels in-flight, scheduled and background agent work until the owner resumes
Partial - C4Data serves only the owner
Non-content telemetry is on by default and can be disabled in Settings > Privacy. By default, agent web-search queries go to You.com, a destination the owner did not specifically choose, though web browsing can be turned off. The privacy policy says data is not sold. Policy changes take effect after posted notice, without specific consent.
› How it could pass
Suggestions. Any change that meets the test counts.
- Turn telemetry off by default
- Make third-party web search opt-in
- Ask for specific consent before changing data terms
Partial - C5Credential custody
All credentials are held locally: provider keys in the plaintext storage .env, integration configs in SQLite. Mintplex holds none and uses no broker. The UI shows only whether each key is set, across several settings pages, and LLM provider keys can be overwritten but not individually revoked or listed in one place.
› How it could pass
Suggestions. Any change that meets the test counts.
- List every credential and grant in one view with a revoke control for each
Partial - C6Full deletion
No provider-side copy of the agent exists. Full deletion is a documented multi-step manual process (uninstall, then delete the storage folder) with no in-app single action or completion confirmation. Telemetry already sent to PostHog is not agent state.
› How it could pass
Suggestions. Any change that meets the test counts.
- Offer a one-step in-app 'delete everything' action with confirmation
Partial
How this assessment was produced
One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of AnythingLLM (https://anythingllm.com).
During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.
- Model
- claude-opus-5-5
- Harness
- Claude Code 2.1.285
- Ran
- 2026-09-30 13:59:33 UTC · 10 turns
- Criteria hash
- a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
- Full transcript
- Download (JSONL, gzip) · every tool call, page read and reply
- Run protocol
- 2026-09-30-v1
Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.
- Human review
- pending
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/anythingllm.json.
Or grade it from scratch. See Check any score yourself.
Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.