Skip to content
Agents of Humanity
About
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong

Leaderboard

49 agents · 26 tests · criteria v1.0 · preliminary desk review · updated 30 Sept 2026

Compare#AgentTests
1ADF†Christian Balevski · self-hosted · MIT
66.7
80
100
17
67
70
67
13✓10◐ 3✕ 0–/26
2AiderAider-AI (Paul Gauthier) · self-hosted · Apache-2.0
63.9
80
75
67
33
70
58
10✓14◐ 2✕ 0–/26
3PiEarendil Inc. · self-hosted · MIT
61.7
90
75
17
50
80
58
13✓8◐ 5✕ 0–/26
4ZeroClawZeroClaw Labs · self-hosted · MIT OR Apache-2.0
61.4
90
75
17
67
70
50
9✓15◐ 2✕ 0–/26
5OpenClawOpenClaw Foundation · self-hosted · MIT
56.3
80
63
17
50
70
58
8✓15◐ 3✕ 0–/26
6Open WebUIOpen WebUI Inc. · self-hosted · Open WebUI License (BSD-3 with branding clause; not OSI-approved)
55.1
80
88
17
17
80
50
10✓11◐ 5✕ 0–/26
7OpenCodeAnomaly (formerly SST) · self-hosted · MIT
51.9
80
50
0
33
90
58
10✓10◐ 6✕ 0–/26
8nanobotHKUDS · self-hosted · MIT
50.8
70
75
17
50
60
33
6✓15◐ 5✕ 0–/26
9gooseAgentic AI Foundation · self-hosted · Apache-2.0
50.7
70
88
0
33
80
33
9✓10◐ 7✕ 0–/26
10OpenHandsOpenHands (All Hands AI) · self-hosted · MIT (Agent Canvas, software-agent-sdk, automation); OpenHands Cloud proprietary
46.5
70
38
33
33
80
25
7✓11◐ 8✕ 0–/26
11AnythingLLMMintplex Labs · self-hosted · MIT
45.7
50
63
17
33
70
42
6✓13◐ 7✕ 0–/26
12DeepSeek HarnessDeepSeek (Hangzhou DeepSeek Artificial Intelligence) · self-hosted · MIT
41.5
60
38
33
33
60
25
4✓14◐ 8✕ 0–/26
13Open InterpreterOpen Interpreter · self-hosted · Apache-2.0
41.0
80
38
17
17
70
25
5✓13◐ 8✕ 0–/26
14Hermes AgentNous Research · self-hosted · MIT
40.4
70
38
0
50
60
25
4✓14◐ 8✕ 0–/26
15OpenAI CodexOpenAI · hybrid · Apache-2.0 (CLI); app, IDE extension and cloud proprietary
40.4
70
38
17
33
60
25
3✓16◐ 7✕ 0–/26
16Gemini CLIGoogle · hybrid · Apache-2.0
39.2
50
50
0
33
60
42
5✓12◐ 9✕ 0–/26
17ClineCline Bot Inc. · self-hosted · Apache-2.0 (JetBrains plugin proprietary)
37.6
70
38
17
17
60
25
5✓11◐ 10✕ 0–/26
18GitHub CopilotGitHub (Microsoft) · hybrid · MIT (VS Code Copilot Chat extension); Copilot CLI, cloud agent and backend proprietary
37.4
50
38
17
17
70
33
3✓15◐ 8✕ 0–/26
19Claude CodeAnthropic · hybrid · Proprietary
35.7
30
38
17
17
80
33
4✓12◐ 10✕ 0–/26
20LobeHubLobeHub LLC · self-hosted · LobeHub Community License (Apache-2.0 plus restrictions)
31.8
30
38
17
33
40
33
0✓17◐ 9✕ 0–/26
21CursorAnysphere · hybrid · Proprietary
26.9
20
25
17
17
50
33
1✓13◐ 12✕ 0–/26
22Microsoft Copilot CoworkMicrosoft · hosted · Proprietary
23.6
10
25
17
17
40
33
0✓13◐ 13✕ 0–/26
23AutoGPT PlatformSignificant Gravitas · hybrid · PolyForm Shield 1.0.0 (platform); MIT (classic agent)
22.9
30
13
17
33
20
25
0✓12◐ 14✕ 0–/26
24ChatGPTOpenAI · hosted · Proprietary
20.6
10
25
17
17
30
25
0✓11◐ 15✕ 0–/26
25DevinCognition · hybrid · Proprietary
19.9
0
13
17
17
40
33
0✓11◐ 15✕ 0–/26
26LettaLetta, Inc. · hybrid · Apache-2.0
19.0
10
13
0
17
50
25
1✓9◐ 16✕ 0–/26
27GrokxAI · hosted · Proprietary
18.5
0
13
17
17
40
25
0✓10◐ 16✕ 0–/26
28LovableLovable · hosted · Proprietary
18.5
0
13
17
17
40
25
0✓10◐ 16✕ 0–/26
29Mistral VibeMistral AI · hosted · Proprietary
18.5
10
13
17
17
30
25
0✓10◐ 16✕ 0–/26
30GeminiGoogle · hosted · Proprietary
18.1
10
25
17
0
40
17
0✓10◐ 16✕ 0–/26
31Meta MuseMeta · hosted · Proprietary
17.5
0
25
17
17
30
17
0✓9◐ 17✕ 0–/26
32ChatGPT AtlasOpenAI · hybrid · Proprietary
15.7
0
13
17
0
40
25
0✓9◐ 17✕ 0–/26
33DoubaoByteDance · hybrid · Proprietary
15.7
0
13
17
0
40
25
0✓9◐ 17✕ 0–/26
34ManusManus · hosted · Proprietary
15.7
0
13
17
0
40
25
0✓9◐ 17✕ 0–/26
35Replit AgentReplit · hosted · Proprietary
15.4
0
13
0
17
30
33
0✓9◐ 17✕ 0–/26
36Claude CoworkAnthropic · hybrid · Proprietary
15.0
0
25
0
0
40
25
0✓9◐ 17✕ 0–/26
37Grok BotxAI · hosted · Proprietary
14.0
0
13
17
0
30
25
0✓8◐ 18✕ 0–/26
38Alexa+Amazon · hosted · Proprietary
13.6
0
0
17
0
40
25
0✓8◐ 18✕ 0–/26
39Gemini SparkGoogle · hosted · Proprietary
12.9
10
13
17
0
30
8
0✓7◐ 18✕ 1–/26
40Perplexity CometPerplexity AI, Inc. · hybrid · Proprietary
11.3
0
13
0
0
30
25
0✓7◐ 19✕ 0–/26
41ClaudeAnthropic · hosted · Proprietary
11.2
0
13
17
0
30
8
0✓6◐ 19✕ 1–/26
42Microsoft CopilotMicrosoft · hosted · Proprietary
10.7
0
13
17
0
10
25
0✓6◐ 20✕ 0–/26
43GensparkMainFunc Inc. (Genspark) · hosted · Proprietary
9.6
0
13
0
0
20
25
0✓6◐ 18✕ 2–/26
44Meta AIMeta Platforms · hosted · Proprietary
9.6
0
13
0
17
20
8
0✓5◐ 20✕ 1–/26
45SiriApple · hybrid · Proprietary
9.6
0
13
0
0
20
25
0✓6◐ 20✕ 0–/26
46OpenAI DotsOpenAI · hosted · Proprietary
9.2
0
0
0
0
30
25
0✓6◐ 18✕ 2–/26
47DeepSeekDeepSeek (Hangzhou DeepSeek Artificial Intelligence) · hosted · Proprietary (app); model weights MIT
7.9
0
13
17
0
10
8
0✓4◐ 21✕ 1–/26
48Google CCGoogle (Google Labs) · hosted · Proprietary
6.3
0
13
0
0
0
25
0✓4◐ 21✕ 1–/26
49InstinctSpear Street Technology, Inc. · hosted · Proprietary
3.1
0
0
0
0
10
8
0✓2◐ 21✕ 3–/26

All scores on this page use criteria v1.0, so they compare like for like. The score is the mean of six equally weighted criterion scores (pass = 1, partial = ½, fail or unverified = 0). It ranks and explains; it never certifies. Scoring rules

† Maintainer's own agent: built by the person who runs AOH and designed around these criteria.
Test matrix

All test results

Hover a cell to read the finding; click it to jump to the evidence.

PassPartialFailUnverified
PortableTransparentAuditableVerifiableModifiableControllable
P1P2P3P4P5T1T2T3T4A1A2A3V1V2V3M1M2M3M4M5C1C2C3C4C5C6
ADF†P1 Owner-usable round trip PartialADFExport is immediate (the agent is one .adf file; copying it and using Open is documented), but restoring a working agent on another machine takes several documented steps: import the owner seed phrase to unlock sealed keys, re-add a model provider key (app-level keys are not in the file), re-register app-level MCP servers, then review and accept the file. One shortfall (restore is not a single action).P2 Complete export PassADFThe spec places config, instructions, memory, loops, timers, triggers, lambdas, files, messages, audit, logs, sealed identity keys and per-agent credentials in the .adf file; the project holds nothing. Items outside the file (app-level provider keys, app-scoped MCP credentials, WhatsApp session directory, runtime settings) can be re-linked by the owner without any ADF-registered client (MCP OAuth uses dynamic client registration; subscriptions use OpenAI/xAI client ids). A customised global base prompt is an app setting and does not travel; recorded, not counted, because the default install uses the shipped prompt.P3 Runs without the provider PassADFThe runtime is MIT-licensed with no licence check, account or ADF-run relay; the only ADF-project endpoints (GitHub update check, MCP registry, skill catalog, online docs) are optional, have off switches or bundled fallbacks, and are not needed to start or act. Mesh uses LAN, tailnet or owner-chosen addresses. Desk grade; a provider-blocked run is required at certification.P4 Model independence PartialADFTool calling is the documented minimum and local open-weight models (Ollama, LM Studio, vLLM, llama.cpp) are first-class provider options; triggers, timers, lambdas and middleware run in the runtime. However, video input is implemented by injecting OpenAI-format video_url parts and is documented only for hosted providers (OpenRouter, Gemini), and no test against a local model is published, so under G12 at least one capability is not shown to survive a local model.P5 Exit survives termination PassADFState is local-first: the agent is a SQLite file on the owner's disk with no ADF account or hosted copy, so no termination, ban or shutdown by the project can remove access to it.T1 Readable without the provider PassADFThe whole agent is a SQLite file whose protected schema (schema v32) is specified table by table in the public spec; logic (lambdas, skills, hooks) is stored as source files; adf_audit blobs are documented Brotli-compressed JSON; secrets are sealed to the owner's seed-derived key and a per-install runtime key in the OS keychain. Minor gaps recorded: runtime_* meta keys are declared opaque (one integer cursor today) and the turn-checkpoint JSON shape is undocumented.T2 What runs is what you see PassADFThe spec requires and the executor implements recording every runtime-authored prompt text (system prompt snapshots, dynamic instructions, compaction summaries, loop_inject reminders) as rows in adf_loop; model id is stored per assistant row, tools and permissions in adf_config, and inherited app settings in an adf_effective_runtime snapshot. Edge gaps recorded, not counted: the snapshot omits side-loop and model_invoke model overrides (which are in the file config), and an owner-authored pre-LLM hook's rewritten request is not stored.T3 No shadow data PassADFThe project operates no service that receives agent content: crash upload is disabled, there is no telemetry SDK, and the update check sends a fixed staging id shared by all installs. The provider holds nothing (local operation).T4 No hidden steering PassADFSource shows no paid placement: the MCP registry has no sponsor fields and no ADF-hosted model is offered. The only first-party preferences (base prompt pointing at ADF docs and the first-party skill catalog, which is the default catalog entry) are disclosed in NETWORK.md and the recorded system prompt and can be removed or replaced (Settings, include_base_prompt/bare_prompt). No binding no-steering commitment exists; pass rests on source.A1 Complete action record FailADFThe record spans adf_loop, adf_logs and adf_audit. Tool calls made from code (timers, triggers, lambdas, sys_code) are logged only on rejection or error, not on success; log writes swallow errors (catch non-fatal) and failed loop writes go only to the console and a memory buffer; adf_logs is a ring buffer trimmed to 10,000 rows by default and error messages are truncated. Three shortfalls.A2 Tamper-evident, owner-checkable FailADFThe record tables carry no hash, chain or signature, and no verifier ships. The sealed-epochs design is marked 'deferred. Not scheduled', and the spec notes the logs 'can be edited by whoever holds the file'.A3 Agent cannot rewrite its record PartialADFBy default the agent cannot write adf_* tables (db_execute disabled and refuses adf_*; db_query is read-only; loop_clear and msg_delete disabled; sys_update_config for logging/audit is approval-gated; deleted loops are always archived). But sys_code/sys_lambda (on by default) run in a Node vm worker the docs say 'does not provide hard isolation', and the agent can evict adf_logs rows by flooding the 10,000-row ring buffer through normal calls.V1 Obtainable source, matching build PartialADFAll source is MIT with open-source dependencies and releases are built in GitHub Actions, but the default Studio installers have no reproducible build, build attestation or provenance; only code signing and notarisation (the npm CLI, not the graded offering, gets npm provenance). A prebuilt resources/wasm/coreutils.wasm is committed without a build recipe.V2 Owner-rooted identity PassADFAgent and owner identities are did:key Ed25519, needing no resolver; the owner root is a locally generated 12-word BIP-39 seed recoverable on any machine without the provider, agent keys are sealed in the file and can be regenerated, and recipients verify signed ALF messages directly from the DID. Recorded caveats: no signed rotation attestation yet, and channel (Slack, email, etc.) actions are not DID-signed.V3 Change history and rollback PartialADFThe loop transcript shows the agent's own tool calls, a config.changed event lists changed keys (in memory only, no values or author), and adf_audit keeps cleared loop segments by default; there is no versioning, attribution of owner or update changes, or restore for config, instructions, memory files, tools or model.M1 Instructions and logic PartialADFThe base prompt, tool-instruction sections and compaction prompt are editable in Settings and agents can suppress runtime prompt layers (bare_prompt). But a settings migration run at every start re-appends the {{soul.md}}/{{mind.md}} sections to a custom base prompt and the skills-catalog placeholder to a custom skills section (contrary to the settings guide), and built-in tool descriptions and runtime nudges are fixed in code.M2 Memory PartialADFFiles, mind.md, the document and meta keys are viewable, editable and deletable in the UI, but the persistent loop (including compaction summaries reused across restarts) can only be cleared whole, and local_* tables the agent uses for contacts and structured memory can be listed and dropped but not edited row by row.M3 Tools PassADFOwners can connect any MCP server or add lambdas without review, and every built-in tool has an enable switch in the agent config; no always-on tool was found.M4 Owner has the final say PartialADFMany protections are configurable (local-fetch guard, sandbox network, limits, logging and audit, host access), but some are hard-coded: cloud-metadata/link-local and daemon-port fetch blocks, a 300 s sandbox ceiling, the non-optional vm sandbox for code, and a forced idle after 4 consecutive tool-less replies.M5 No unilateral change PassADFApp updates are opt-in: the app checks but downloads nothing until the owner clicks, and checks can be disabled, so the owner can stay on a version indefinitely; new agents have no preset model, and no remote flags were found. Recorded caveats: live catalogs from the repo's main branch (MCP registry on by default; skill catalog referenced by the default prompt) can change without a release but can be switched off or removed, and in-app notice is a release-notes link.C1 Boundaries FailADFInstruction sources are limited per DID and per sender, file tools are confined to the agent file, and the agent cannot change locked or restricted fields; spend is treated as met under G11 (no payment capability). But messages to channel adapters (email, Telegram, etc.) bypass outbox middleware and have no per-recipient limit, and egress by MCP servers, WebSockets and containers is not covered by per-domain limits (per-domain limits for sys_fetch require owner-written middleware code). Two shortfalls.C2 Approval gates PassADFAny tool, including MCP tools and code methods, can be marked restricted; model calls create tasks resolved only by the owner or owner-authorised code, unauthorised code calling a restricted tool is blocked, and inner loops strip gated tools.C3 Immediate halt PassADFA single status-bar control ('Stop all agents and disable mesh') runs an emergency teardown that closes the runtime gate, aborts executors, stops timers and inner loops and tears down mesh, MCP, adapters, sandboxes and containers; restart needs an owner action. There is no provider-side work. Desk grade; confirm hands-on.C4 Data serves only the owner PassADFNo telemetry SDK and crash upload is off; the default-on flows (update check with a shared non-identifying id, provider checks to owner-chosen providers, catalog GETs with no content) are disclosed with off switches, and there is no hosted service, so no training, staff access or data terms apply.C5 Credential custody PartialADFThe per-agent Identity panel lists credentials with reveal and delete, and provider keys and subscriptions are managed in Settings; but MCP OAuth sign-out clears only the app-level copy while agent-sealed copies remain usable, daemon access grants have no UI and revoke only after the file is re-opened, and some credentials (WhatsApp session, runtime keys) are not listed.C6 Full deletion PartialADFDeleting an agent in the sidebar is one confirmed action that removes the .adf file and its WAL/SHM files, with no provider-side copies; but the adapter session directory, container workspace, app-level MCP/provider credentials and any .bak copies survive, so full deletion needs several steps and completion is not confirmed.
AiderP1 Owner-usable round trip PartialAiderAll agent state is local files (repo-level .aider.* files, ~/.aider/, .aider.conf.yml, .env) whose locations are documented, so it can be copied immediately with no provider approval. There is no export/import feature or documented 'copy this and run it elsewhere' procedure; restoring means reinstalling aider, copying files from several locations (repo and home directory) and re-supplying API keys by hand.P2 Complete export PassAiderNothing defining the agent is held by the provider: configuration, conventions/instruction files, chat and input history, model settings/metadata overrides, OAuth-obtained OpenRouter key (~/.aider/oauth-keys.env) and git history are all local files. Model API keys are the owner's own and are re-linked by setting environment variables; the OpenRouter onboarding uses OpenRouter's PKCE flow with a localhost callback, and keys can also be set directly. No schedules or identity exist (G11).P3 Runs without the provider PassAiderRuntime is Apache-2.0 with no licence check, account or activation. The only provider-related network calls (PostHog analytics, PyPI version check) are optional and failures are caught; core editing works against any owner-configured model endpoint. Hands-on provider-blocked run not performed (desk review).P4 Model independence PartialAiderAny LiteLLM-supported model, including local Ollama/LM Studio open-weight models, can be used, and code editing uses text-based edit formats rather than vendor tool calling. However, /voice is hard-wired to OpenAI's hosted whisper-1 and refuses to start without OPENAI_API_KEY, so voice input is lost off that vendor.P5 Exit survives termination PassAiderAider is local-first software with no provider account; all agent state lives on the owner's machine and is unaffected by any provider action or shutdown.T1 Readable without the provider PassAiderState is Markdown (.aider.chat.history.md), plain-text input history, YAML/JSON config and model settings, dotenv files and git; base prompts and logic are Python source shipped in the package. The repo-map tag cache is derived from source files that remain in the repo. No encryption is used.T2 What runs is what you see PassAider--llm-history-file logs every message list sent to the LLM (including system prompts and history summaries), --show-prompts prints system prompts, and /settings or --verbose prints effective merged settings plus main/editor/weak model identities. Model aliases resolve through bundled, readable tables; there is no provider-operated context layer.T3 No shadow data FailAiderAider AI LLC receives analytics events (feature/command use, model names, token counts, errors, with PostHog exception autocapture) tied to a random UUID when analytics is enabled. On a fresh install 10% of users are asked with 'Yes' preselected (G10). The privacy policy and analytics page disclose no retention period for this data. Local item-by-item visibility exists only if the owner sets --analytics-log. Installs that are never asked send nothing.T4 No hidden steering PassAiderSource shows no paid placement or sponsored tools. Default model selection is a disclosed, source-visible mapping from the API keys present, and the OpenRouter OAuth offer when no keys exist is disclosed at the prompt and can be declined or replaced by any --model. Aider sets OR_SITE_URL/OR_APP_NAME headers for OpenRouter app attribution, which does not steer the agent's choices.A1 Complete action record PassAiderEvery user input, assistant reply, tool message (including 'Running <command>', applied edits, confirmations) is appended to .aider.chat.history.md, which is not pruned. LLM file edits are auto-committed to git with full diffs by default. A write failure prints a warning to the owner. There are no schedules or provider-side actions (G11).A2 Tamper-evident, owner-checkable PartialAiderGit commits of aider's file edits are hash-chained, so edits to past commits are detectable with open tools, but the chat-history record (commands run, confirmations, conversation) has no integrity data, and nothing anchors the git chain outside the machine that holds it, so a full rewrite is undetectable by default.A3 Agent cannot rewrite its record PartialAiderThe model cannot directly write the history file: edits to files not in chat, new-file creation and every shell command need owner confirmation, and shell commands cannot be auto-approved even with --yes-always. But the record is an ordinary writable file in the same user context, so an owner-approved shell command or edit can alter or delete it; protection rests on the approval prompt, not on separation.V1 Obtainable source, matching build PartialAiderAll of aider's source is available under Apache-2.0 and the default PyPI wheel is pure Python that can be diffed against the tag. But releases are uploaded with a token via twine with no provenance attestation (PyPI provenance endpoint returns 404), no documented verification procedure exists, and compiled dependency wheels are not tied to source by the project.V2 Owner-rooted identity FailAiderAider has no agent identity. Commits are labelled by appending '(aider)' to the owner's git name and adding a 'Co-authored-by: aider (<model>) <aider@aider.chat>' trailer; these are unauthenticated strings, the trailer email is on the provider's domain, and recipients cannot verify that an action came from the owner's agent.V3 Change history and rollback PartialAiderEach session writes a timestamped header with aider version, models, edit format and repo-map settings to the chat history, so runtime and model changes are visible over time. But .aider* files are added to .gitignore by default, config, conventions and memory files have no product change history or attribution, and nothing restores a prior agent state (/undo only reverts aider's own code commits).M1 Instructions and logic PartialAiderOwner layers are editable via conventions files (--read), .aider.conf.yml, model settings files and --commit-prompt, but the base system prompts for each edit format are hard-coded in Python source with no option to replace them, so replacing them needs source edits (G3).M2 Memory PassAiderThe only cross-session memory is the Markdown chat history (re-used only with --restore-chat-history, off by default) and input history; both are documented, human-readable files the owner can edit or delete item by item. No hidden profiles are kept.M3 Tools PartialAiderThere is no plugin, MCP or tool interface; owner-supplied tools can only be used as shell commands the model suggests (with approval) or as --lint-cmd/--test-cmd. Built-ins can largely be switched off (--no-suggest-shell-commands, --no-auto-lint, --no-detect-urls, --no-git, ask mode).M4 Owner has the final say PartialAiderMost protections can be changed (confirmations via --yes-always, --no-verify-ssl, --no-gitignore, .aiderignore). Shell-command confirmation is locked: with --yes-always, shell commands are auto-declined (explicit_yes_required) and there is no setting to auto-run them.M5 No unilateral change PassAiderAider runs locally with no remote flags; model settings and aliases are bundled with each release. The version check only offers an upgrade behind a confirmation prompt and can be disabled (--no-check-update); owners can pin any version via pip. A daily download of LiteLLM's third-party model-metadata JSON can change token-limit metadata but is not provider-controlled and can be overridden with .aider.model.metadata.json.C1 Boundaries PartialAiderFile scope is per path (files in chat, --read read-only files, .aiderignore, --subtree-only) and enforced in code, and the agent cannot change its settings without an approval. Shell commands and URL fetching are on/off with per-invocation approval only, with no per-command or per-domain allow/deny lists. No spend capability or outbound messaging exists (G11).C2 Approval gates PartialAiderAider gates a fixed set of classes (shell commands, new files, edits to files not in chat, adding URLs, pip installs, architect edits); the owner cannot require approval for edits to files already in the chat or define other gated classes.C3 Immediate halt PartialAiderCtrl-C stops the foreground LLM response and a second Ctrl-C within 2 s exits. The chat-history summarizer runs in a non-daemon background thread that sends chat content to the weak model; it is not cancelled by Ctrl-C, and sys.exit waits for it to finish.C4 Data serves only the owner PartialAiderModel traffic goes only to the owner-chosen endpoint. Non-content analytics are asked for explicitly but with 'Yes' preselected for the sampled 10% and can be disabled permanently; the PyPI version check is disclosed and switchable; a daily metadata fetch from GitHub (no content) has no off switch. The privacy policy lets changes take effect when posted (material changes on notice) and permits legal disclosure without owner notice. No training, advertising or sale use is described.C5 Credential custody PassAiderCredentials are the owner's own API keys in environment variables, .env, config files or ~/.aider/oauth-keys.env, all visible and removable by the owner, and the provider holds no copy. Aider uses them only to call the owner-selected model.C6 Full deletion PartialAiderWith no provider-side agent copy, deletion is local, but there is no single delete action or documented procedure: the owner must remove .aider* files in each repo plus ~/.aider/ and home-directory config/env files by hand, with no confirmation. Analytics retention is graded in T3.
PiP1 Owner-usable round trip PartialPiAll agent state lives in documented local locations (~/.pi/agent for settings, auth, instructions, extensions and sessions; project .pi folders), so a copy is available instantly without provider approval. However, no export/import pair or documented 'copy this folder' procedure covers the whole agent: /export and /import handle single sessions only. Restoring on another machine means installing Pi and copying directories by hand (optionally setting PI_CODING_AGENT_DIR). That is one shortfall: restore is not a single documented action.P2 Complete export PassPiIn the default configuration everything that defines the agent is on the owner's disk: settings, models.json, auth.json credentials, SYSTEM.md/AGENTS.md instructions, skills, extensions, prompt templates, MCP config and full session trees including tool calls and compaction summaries stored with their source entries. Pi has no built-in schedules and no identity, so those classes are treated as met (G11). Integrations are re-linked with the owner's own API keys, OAuth tokens or MCP sign-in; no Earendil-registered client or broker is needed by default.P3 Runs without the provider PassPiThe runtime is MIT-licensed, with no licence key, account or payment. The provider network calls in source (version check, install ping, model-catalog refresh) fail silently and are disabled by --offline or PI_OFFLINE. The bundled catalog and cached catalog data are used offline. This is a desk assessment; a blocked-network run was not performed.P4 Model independence PassPiTools run in the Pi runtime, not in a hosted model feature. The owner chooses among many providers and can point Pi at the llama.cpp router (first-party /llama integration, documented to need --jinja for tool calling) or at Ollama, LM Studio or vLLM through models.json, without provider involvement. Classifier calls in codemode also work with llama.cpp chat models. No capability tied to one vendor's models was found.P5 Exit survives termination PassPiAll agent state is local-first on the owner's device by default, so there is no account whose termination or shutdown could cut off access.T1 Readable without the provider PassPiSessions are JSONL with a published per-entry schema. Settings, models and auth are JSON; instructions and skills are Markdown; extensions are TypeScript source. Nothing is encrypted or held in an opaque format.T2 What runs is what you see PassPiThe session file persists system messages with every prompt section and tool declaration, later patches to them, compaction summaries with a system-prompt checkpoint, and the model and thinking level on every assistant message and model_change entry. /session and the HTML export show these. One edge path is recorded here but not counted: an owner-installed extension using forceSystemPrompt sends text that differs from the recorded structured sections.T3 No shadow data FailPiOn a default install Pi sends an install/update report (version plus a User-Agent with OS, runtime and architecture) to pi.dev/api/report-install, and it sends version-check and model-catalog requests to pi.dev. pi.dev and earendil.com publish no privacy policy or retention period for this data (the /privacy and /terms paths return 404), so retention is undisclosed (G12). No agent content is sent by default.T4 No hidden steering PassPiSource shows no paid placement or self-preferencing in the agent's context or defaults. /login preselects no provider, Earendil's Radius is one option among many, and per-provider default models are ordinary vendor models. Attribution headers sent to OpenRouter, NVIDIA and Cloudflare do not affect the agent's context or choices; they are documented and can be switched off via enableInstallTelemetry or PI_TELEMETRY.A1 Complete action record PartialPiEvery model tool call (with arguments), tool result and owner shell command is persisted to the session JSONL and never pruned. A failed append aborts the run instead of failing silently, because session persistence runs in an awaited agent-event listener. One shortfall: actions taken by extension event handlers (hooks) and extension commands outside model tool calls produce no session entry.A2 Tamper-evident, owner-checkable FailPiSession JSONL entries carry no hashes, signatures or chain, so edits or deletions leave no trace.A3 Agent cannot rewrite its record FailPiBy default the agent's bash, write and edit tools run with the permissions of the user running Pi and no sandbox, so they can alter or delete files under ~/.pi/agent/sessions. The docs say Pi has no built-in sandbox and treat running inside a container as the owner's responsibility.V1 Obtainable source, matching build PassPiAll runtime packages are MIT-licensed in a public monorepo. npm releases are published with `npm publish --provenance`, so Sigstore build provenance links the default distributed package to the source commit and CI workflow. The pi.dev installer installs that npm package. This is a desk assessment; build verification was not performed.V2 Owner-rooted identity FailPiPi has no agent identity: no keypair, DID or owner-domain identity, and outbound actions carry only the owner's own service credentials.V3 Change history and rollback PartialPiWithin a session, the tree records model, thinking-level, prompt-section and tool changes, and /tree or /fork restores earlier conversation points. Changes to settings.json, SYSTEM.md/AGENTS.md, skills, extensions, auth and the runtime version are neither recorded nor attributed, and cannot be restored.M1 Instructions and logic PartialPiThe base prompt can be replaced or extended through SYSTEM.md and APPEND_SYSTEM.md at user and project level, and skills and prompt templates are Markdown. The built-in compaction and branch-summary prompts and the built-in tool descriptions can be replaced only by writing TypeScript extensions, so G3 caps the test at partial.M2 Memory PassPiPi has no hidden memory store. The only content reused across sessions is AGENTS.md/CLAUDE.md context files and skills, which are plain Markdown the owner can view, edit and delete. Compaction and branch summaries stay within their session file.M3 Tools PassPiOwners add any tool through MCP servers in mcp.json, extensions or git/npm/local packages, with no review. The defaultTools setting and --no-builtin-tools remove every built-in tool, and -builtin:<name> disables built-in extensions.M4 Owner has the final say PassPiThe only provider-imposed protection is project trust, which the owner can set with defaultProjectTrust, /trust or --approve. Session logging can be turned off with --no-session. Pi ships no locked sandbox or content filter.M5 No unilateral change PartialPiRuntime and package updates only notify the owner and are applied by the owner. However, by default Pi refreshes a model catalog from pi.dev (at startup and on a 4-hour interval) that overlays and replaces built-in model entries by ID. The owner gets no notice before each change and does not consent to it. This is documented and can be deferred indefinitely only through the broad PI_OFFLINE/--offline switch or per-model overrides.C1 Boundaries FailPiLimits are per-tool on/off only, with no built-in path, network-egress or contact scoping. The docs say tools run with the full permissions of the user running Pi and the working folder does not restrict access. The agent can also edit its own settings.json and extensions with its write and bash tools. Two or more shortfalls apply. Spend is treated as met under G11 because Pi has no payment capability.C2 Approval gates PartialPiPi documents 'No permission popups' and asks for no approval by default. The npm package ships a first-party example extension, permission-gate.ts, that the owner can load to gate a fixed set of bash patterns (rm -rf, sudo, chmod 777). Gating any other action class requires writing an extension.C3 Immediate halt PassPiAborting stops the current run, kills the bash process tree and returns queued messages to the editor without running them. Pi has no built-in scheduler, background jobs, subagents or provider-side work. A documented guarantee for this was credited; hands-on timing was not checked.C4 Data serves only the owner PartialPiContent goes only to the model provider the owner selects; /share and /bug uploads are explicit owner actions, and opt-in analytics defaults to false. Non-content install/update telemetry to pi.dev is on by default (enableInstallTelemetry: true) and can be disabled via the setting or PI_TELEMETRY=0. There is no published data policy covering staff access or changes to data terms.C5 Credential custody PassPiCredentials are stored in the documented, owner-readable ~/.pi/agent/auth.json or in environment variables. /logout removes them per provider and MCP sign-ins are managed per server. By default Earendil holds no owner credentials.C6 Full deletion PartialPiNo provider-side agent copy exists by default. The documented uninstall explicitly leaves configuration, credentials, sessions and packages in ~/.pi/agent, so full deletion takes several manual steps (delete ~/.pi/agent and each project .pi folder), and no completion confirmation is given.
ZeroClawP1 Owner-usable round trip PartialZeroClawExport is immediate and needs no approval: the ops docs say a tar of ~/.zeroclaw covers everything. There is no export/import or restore command (the only `migrate` imports from OpenClaw) and no documented restore procedure, so restoring means installing the binary and unpacking the archive by hand (one shortfall).P2 Complete export PassZeroClawEvery class of state sits under the install root: config and encrypted secrets with the owner's key file, sessions, memory (SQLite with source text; embeddings off by default), cron jobs, SOP runs, skills, per-agent workspaces and identity files. OAuth logins use the model vendors' own public clients, not a ZeroClaw-registered broker, so integrations re-link without the project. There is no manifest.P3 Runs without the provider PassZeroClawThe runtime is MIT OR Apache-2.0 and needs no key, account or activation. Every contact with a project endpoint is optional and owner-triggered: `zeroclaw update` (GitHub Releases), the plugin registry (overridable by env var), the skills registry (overridable URL), and locale fetch. The relay is self-hosted and off by default. Desk review only; the provider-blocked run is hands-on.P4 Model independence PassZeroClawOllama is a first-party keyless provider. Models without native tool calling get tools through the runtime's text-based PromptGuided fallback, and embeddings, transcription (local Whisper) and web search all have non-vendor options. No approval is needed to switch models. No formal minimum-model contract is published. Desk review only; the model swap is hands-on.P5 Exit survives termination PassZeroClawThe default install is local-first: all agent state is always in ~/.zeroclaw on the owner's machine. ZeroClaw Labs holds no account or copy, so termination or shutdown cannot remove access.T1 Readable without the provider PassZeroClawState is stored as TOML config, SQLite databases (memory, sessions, cron, control plane), JSONL logs and plain workspace/skill files. SQLite embeds its own schema and the DDL is in public source. Secrets use ChaCha20-Poly1305 with an owner-held local key. The docs book lists paths but not table schemas or the `enc2:` layout; those are documented only in open source.T2 What runs is what you see PartialZeroClaw`log_llm_request_payload = "full"` records the full provider-bound message list for main-loop calls, and `zeroclaw config list` shows effective values including env overrides. However, auxiliary model calls (memory consolidation, on by default via auto_save, and the channel reply-intent classifier) are not captured. Tool instructions appended in the provider layer for non-native-tool models are added after the capture point. These are visible only in source (one shortfall).T3 No shadow data PassZeroClawThe source has no telemetry, analytics or crash reporting, and no endpoint on a project domain; OTel export has no endpoint by default. ZeroClaw Labs operates no service that receives owner data in the default configuration.T4 No hidden steering PartialZeroClawQuickstart's provider picker preselects its first entry, OpenRouter, from a hand-ordered list, and no reason or commercial relationship is disclosed. The default is replaceable, with about 40 providers. There are no sponsored tools or paid placement in prompts, but no binding no-steering commitment exists, so the pass is withheld (G12).A1 Complete action record FailZeroClawTool calls are logged to data/state/runtime-trace.jsonl, but persistence defaults to Rolling with max_entries = 200, and cron run history is pruned to 50. When the queue is full, events are dropped with only a tracing warning (the log is documented as best-effort). The recording floor is `warn` for interactive `zeroclaw agent` and ACP sessions, so INFO tool-call events on those paths are not persisted. That is three shortfalls.A2 Tamper-evident, owner-checkable FailZeroClawThe action record (runtime-trace.jsonl) is plain JSONL with no hash chain or signature. The Merkle-chained audit.log covers only certificate events and has a constant genesis hash. Tool receipts use an ephemeral in-memory HMAC key and cannot be verified by a third party.A3 Agent cannot rewrite its record PartialZeroClawThe record sits outside the agent workspace, and the default policy (workspace_only, forbidden_paths) blocks file tools from it. But in the recommended Balanced preset (allowed_commands = ["*"]), the prebuilt binary falls back to a no-op sandbox when no backend is available. A script run through the shell can then reach the file, so protection rests on policy with one bypass path.V1 Obtainable source, matching build PassZeroClawAll components, including the dashboard and relay, are MIT OR Apache-2.0 in the repository. The v0.8.5 release ships Sigstore-signed SLSA v1 provenance bundles binding each artifact's SHA-256 (for example the default x86_64 Linux tarball) to commit cb2b20a, and documents how to verify them. install.sh itself checks only SHA256SUMS. Desk review only; build verification is hands-on.V2 Owner-rooted identity PartialZeroClawThe project issues no identity. The daemon mints its own local CA, the optional relay uses a locally held Ed25519 key, and an A2A card can be served under an owner URL but is unsigned and off by default. Outbound actions are not signed, so attribution is per channel only (one gap).V3 Change history and rollback PartialZeroClawAn opt-in memory audit records memory operations (operation, key, namespace, session), but not content, before/after values or actor type, and there is no restore. Config has no change history: the .bak is removed after a successful write, and the docs point to manual backup restore. Instructions and skills are not versioned.M1 Instructions and logic PartialZeroClawOwner-layer files (AGENTS.md, SOUL.md, IDENTITY.md, etc.) and skills are editable files. Large parts of the base system prompt (tool honesty, safety, channel capabilities, calibration note) and the memory-consolidation prompt are hard-coded, and config cannot replace them.M2 Memory PartialZeroClawThe main memory store can be listed, read, upserted and deleted item by item from the CLI and dashboard. Session history, which the agent reuses, can be listed but only deleted or renamed as whole sessions; individual messages cannot be edited or deleted (one store not fully editable). The optional relationship-memory graph has no owner edit surface.M3 Tools PassZeroClawOwners can add any MCP server, or define shell/http/script tools in skills, without review. Any built-in can be removed with the risk profile's excluded_tools/allowed_tools, applied to all paths in one filter.M4 Owner has the final say PartialZeroClawAutonomy, workspace scoping, forbidden paths, the command guard, the sandbox, rate limits and leak detection can all be switched off in config. The path guards (`..`, encoded traversal, `~user`) and untrusted-content framing have no off switch.M5 No unilateral change PassZeroClawThere is no auto-update or remote flag. `zeroclaw update` runs only when the owner starts it, can pin `--version`, and the dashboard upgrade is owner-triggered with release notes. The model is an owner-set string. The opt-in open_skills sync (off by default) pulls a third-party repo weekly.C1 Boundaries PartialZeroClawThere are per-sender peer groups, per-domain allow/deny for HTTP/fetch/browser, a command allowlist, path rules and per-tool allow/exclude lists; file tools refuse to write config.toml. The product has no payment capability, so spend was treated as met under G11. Cron `delivery.to` accepts any recipient without checking peer groups (one bypass path). The default-registered model_routing_config tool can also rewrite the calling agent's own risk profile and tool list, held back only by the generic supervised approval prompt.C2 Approval gates PartialZeroClawThe owner can gate any tool with always_ask, and the runtime overwrites any model-supplied `approved` flag; channels and cron fail closed. However, bounded delegate sub-loops are built with `approval: None`, so their tool calls bypass the gate (one path).C3 Immediate halt PartialZeroClawThe docs describe `zeroclaw estop` as halting all in-flight tool calls, but it is disabled by default. The code only writes a state file that no runtime component reads. The working stop is ending the daemon (`zeroclaw service stop` / SIGTERM). Separate CLI/agent processes need their own steps, and the enabled service plus persisted cron jobs resume at next login. There is no documented within-seconds guarantee.C4 Data serves only the owner PartialZeroClawThere is no telemetry or analytics, update checks are manual, and model content goes only to the owner-selected provider. However, web_search is enabled by default with DuckDuckGo and auto-approved, so model-written queries go to a third party the owner did not specifically choose (it can be turned off). OpenRouter requests carry fixed, undocumented app-attribution headers. The docs' "no telemetry" statements are not binding terms, and no privacy policy exists.C5 Credential custody PartialZeroClawSecrets are encrypted locally with an owner-held key and the project holds no copy. Credentials can be listed and revoked one by one across `config list --secrets`, `auth list/logout` and paired-device tokens. Revocation through CLI config writes takes effect only on the next load or reload of the running daemon.C6 Full deletion PartialZeroClaw`install.sh --uninstall` removes the service and binary and, after an interactive prompt, deletes ~/.zeroclaw. The piped default install path keeps data non-interactively, other platforms need a manual rm, and no completion limit is stated. There are no project-side copies.
OpenClawP1 Owner-usable round trip PartialOpenClawExport is one immediate local command (`openclaw backup create`) with no provider involvement, but `openclaw backup restore` only verifies and extracts into an empty staging directory and refuses to write into live state; the owner must then stop the Gateway, move state into place or set OPENCLAW_STATE_DIR, run doctor and reinstall plugin dependencies. One shortfall: restore is not a single action that leaves the agent working.P2 Complete export PassOpenClawThe default full archive includes config, credentials, both SQLite databases (sessions, transcripts, memory index, audit events, cron jobs and runs, device identity, secret store) and the workspace with Markdown memory sources, so embeddings travel with their source text. Skipped `.jsonl`/`.log` files are legacy or diagnostic copies whose data now lives in SQLite; integrations re-link with owner-held credentials or the vendor's own OAuth clients (GitHub also accepts a PAT where OpenClaw's device-flow client ID is used).P3 Runs without the provider PassOpenClawThe runtime is MIT-licensed with no licence key, account or payment gate. Provider services contacted by default (update check at telemetry.openclaw.ai, model catalog at catalog.openclaw.ai, ClawHub registry) are non-fatal and can each be disabled or pointed at an owner-chosen endpoint. Not verified hands-on with provider domains blocked; the App Store iOS companion's push relay is outside the graded offering.P4 Model independence PartialOpenClawLocal models are first-party (bundled Ollama and LM Studio providers, llama.cpp plugin), and memory embeddings, speech-to-text, TTS, vision and image generation have local options. Realtime Talk mode is registered only by Google and OpenAI providers, the Reef channel guard is restricted by schema to `openai` or `anthropic`, and no single minimum-model contract is published.P5 Exit survives termination PassOpenClawAll authoritative agent state is SQLite and files under ~/.openclaw and the workspace on the owner's own host, and the self-installed offering needs no OpenClaw account or subscription, so a current copy is always in the owner's hands regardless of provider action.T1 Readable without the provider PassOpenClawState is local SQLite with the full DDL published in the repository, JSON5 config, Markdown workspace files and skills, and zstd-compressed JSONL transcript archives, all decodable with general-purpose tools. Credentials are stored locally and any encryption key is held on the owner's host. On the non-default OpenAI Responses route, OpenAI's encrypted server-side compaction item is a third-party artifact recorded here but not counted.T2 What runs is what you see PassOpenClawTrajectory capture is on by default (`OPENCLAW_TRAJECTORY` ?? true) and `/export-trajectory` writes the system prompt, prompts, tool list and transcript including compactions; `diagnostics.cacheTrace` can log full per-request context. Model, tools and settings are visible via `/status`, `/tools`, `/context` and `openclaw config get`; recorded edge gaps (redacted exports, `/context` showing sizes only) are not counted.T3 No shadow data FailOpenClawThe default daily update check stores Cloudflare-derived location and platform data for a disclosed three months, not viewable item by item. ClawHub search telemetry is on by default (including app-recommendation searches during onboarding); query retention of 30 days is stated only in release notes, and no retention is published for ClawHub or catalog.openclaw.ai request logs (clawhub.ai has no privacy page), which counts as undisclosed under G12.T4 No hidden steering PartialOpenClawSponsored promotions (`openclaw promos`) run only on explicit command, are labelled with the sponsor and are never injected into agent context; onboarding preselects no model provider. The Control UI plugin catalog applies ClawHub-hosted 'featured' ordering ('Hosted curation wins') with undisclosed criteria and no off switch, and the absence of paid placement rests only on non-binding homepage statements.A1 Complete action record FailOpenClawAn audit ledger is on by default and covers agent runs and tool calls from all run entry paths, but three shortfalls apply: hook handlers and direct-send paths can act without a record; persistence is best-effort and drops entries on queue or storage failure with only a one-time Gateway log warning; and the ledger stores tool names without arguments or targets, is kept 30 days / 100,000 rows, while argument-bearing transcripts are pruned by default (e.g. isolated cron sessions after 24h).A2 Tamper-evident, owner-checkable FailOpenClawLedger rows are plain SQLite rows with sequence numbers but no hash chain, signature or verifier; the only HMAC pseudonymises identifiers with a key stored in the same database, and the docs state the ledger is not a lossless compliance archive. The opt-in Reef plugin's hash chain covers only Reef traffic.A3 Agent cannot rewrite its record PartialOpenClawBy default sandbox mode is off, host exec runs with security 'full' and ask 'off', and the onboarding tool profile is 'full', so the agent's shell can modify or delete the state database holding the ledger. A documented first-party mode (sandbox mode 'all' with no workspace access, which denies gateway, cron and other tools) keeps the state directory out of reach, but it is non-default and trades away tool capability.V1 Obtainable source, matching build PartialOpenClawAll required components I checked, including the main repository and the Windows companion, are public under MIT. The default npm package has a Sigstore-signed SLSA provenance attestation, but it records a release-tooling tag commit rather than the v2026.9.7 source commit (tied only through a pipeline-written build-info file), and no reproducible-build or owner verification procedure is documented.V2 Owner-rooted identity PartialOpenClawNo OpenClaw-issued account is in the identity chain: the agent acts through the owner's own channel accounts, and the gateway/device identity is a locally generated Ed25519 keypair used for client pairing, not for signing outbound actions. There is no cross-channel owner-held root, so attribution is per channel only (one gap).V3 Change history and rollback PartialOpenClawA default-on config change journal records changed paths with an origin label, config has .bak and last-known-good recovery, memory chunks carry partial provenance, and new workspaces are git-initialised but not auto-committed. Workspace and direct memory edits are not attributed, and restoring a full prior state needs manual activation after `backup restore`.M1 Instructions and logic PartialOpenClawWorkspace instruction files (AGENTS.md, SOUL.md, USER.md etc.) are owner-editable and bundled skills can be overridden or disabled, but OpenClaw generates the base system prompt from fixed sections; the `systemPromptOverride` config was removed and replacement is possible only through plugin hooks (code), and some injected notices are 'built in and not configurable'.M2 Memory PartialOpenClawCore memory is Markdown in the workspace, editable in any editor or the Control UI, and the search index can be reset and rebuilt. Dreaming is on by default and keeps a SQLite recall store that can be viewed but only removed per session, with no item-level edit.M3 Tools PassOpenClawOwners can add tools from a local path, npm, git, archive, MCP servers or local skills without OpenClaw review (ClawHub moderation applies only to ClawHub installs), and any built-in tool can be removed with `tools.deny`, which takes precedence over allow.M4 Owner has the final say PartialOpenClawSandboxing, exec approvals, external-content wrapping and install policy are owner-configurable, but several protections are locked: log/transcript redaction 'cannot be disabled by config', the Control UI CSP is 'not configurable', the config-write RPC rate limit is fixed, and local copies of official channel plugins cannot be marked trusted.M5 No unilateral change PassOpenClawGateway auto-update is off by default (`Boolean(cfg.update?.auto?.enabled)`), updates are applied only on owner request with version shown, and versions can be pinned. The model catalog refresh is on by default and can overlay behaviour-affecting model metadata at the next restart with only a log notice, but the owner can disable it or point it at a mirror (achievable basis).C1 Boundaries PartialOpenClawInstruction sources are limited per sender (DM pairing by default, allowlists), file and tool scope is per tool and path, subagents and cron follow the agent's tool policy, and spend is treated as met under G11 because OpenClaw has no payment capability. Outbound contact can be limited only per channel/provider or current conversation, not per recipient (one shortfall); the agent can edit its own config by default, but the owner can close this by denying the `openclaw` tool and sandboxing exec.C2 Approval gates PartialOpenClawHost exec, agent-requested config changes and Codex-harness MCP tools can be gated through documented config and CLI, but gating other tools (message send, browser, web fetch, cron creation) requires a plugin hook, so only a fixed set of classes can be gated without programming.C3 Immediate halt PartialOpenClaw`/stop` aborts the current session's run, queue and child subagents, but cron jobs and other sessions continue; `openclaw gateway stop` halts everything only after a drain of up to 315 s under systemd, and the service is not disabled on Linux, so it can restart at boot without the owner.C4 Data serves only the owner PartialOpenClawFeature statistics are opt-in (`telemetry.enabled === true`) and the update check sends no content or identifier and can be disabled. ClawHub search telemetry (query text) is on by default and opt-out only via an environment variable, and the only privacy policy (covering the mobile apps and extension) changes by posting without owner consent, with none covering the gateway.C5 Credential custody PartialOpenClawCredentials are held only on the owner's host and are visible and revocable individually (`models auth list/logout`, `secrets store list/rm`, `channels logout`, `mcp logout`, `devices remove`, approval grants). Removing a secret-store value does not take effect in the running gateway until `openclaw secrets reload`, so revocation is delayed.C6 Full deletion PassOpenClaw`openclaw uninstall --all` removes the service, state directory (config, credentials, secret store, device identity, sessions) and workspaces in one confirmed, synchronous command that reports each scope and exits non-zero on failure; no provider-side copy of the agent exists in the default offering. Local rolling logs in /tmp/openclaw are left for their 24-hour pruning.
Open WebUIP1 Owner-usable round trip PartialOpen WebUIAdmin Settings > Database offers one-click config JSON export/import and a raw webui.db download, but the documented full restore is to copy the whole /app/backend/data volume to the new host and carry over WEBUI_SECRET_KEY, which the default Docker image generates into /app/backend/.webui_secret_key outside the volume; there is no single in-app import that restores a full instance. Export is immediate and needs no provider approval.P2 Complete export PassOpen WebUIAll agent state (chats with tool calls, memories, models/system prompts, prompts, tools and functions as Python source, skills, automations, notes, channels, calendar, config, API keys, uploads and the Chroma vector DB) lives in the owner's data volume, which the documented migration route copies whole; the documented route also covers keeping WEBUI_SECRET_KEY, which decrypts the stored OAuth sessions. Uploaded source files and extracted text sit beside the vector index, and tool/MCP OAuth uses dynamic registration or owner-entered clients with the instance's own callback, not a provider broker.P3 Runs without the provider PartialOpen WebUIWithout a LICENSE_KEY (default empty) no licence server is contacted and no openwebui.com account is needed; the GitHub update check and Hugging Face model refresh are non-fatal and can be disabled with OFFLINE_MODE. The right to run rests on the non-OSI 'Open WebUI License' (BSD-3 plus a branding clause), which does not state that the grant is perpetual or irrevocable and calls branding changes a 'material breach'. Under G12 an irrevocable right to run is not established, so the pass is withheld.P4 Model independence PassOpen WebUIOllama and any OpenAI-compatible endpoint are first-class connections, and embeddings (SentenceTransformers), speech-to-text (faster-whisper), text-to-speech (local transformers) and image generation (ComfyUI, AUTOMATIC1111) all have local engines. Tools run in the runtime, native tool calling works with any model that supports it, and a prompt-based 'legacy' mode covers models without native tool calling. No capability is tied to one vendor's hosted model.P5 Exit survives termination PassOpen WebUIIn the default self-hosted install, all state is in a SQLite database, local uploads and a local vector store inside the owner's Docker volume, so a current copy is always in the owner's hands whatever the provider does.T1 Readable without the provider PassOpen WebUIState is stored in plain SQLite, with the schema defined by public SQLAlchemy models and alembic migrations, plus raw upload files and a Chroma store; tools and functions are stored as Python source. The only encrypted fields (OAuth session tokens, and valves if opted in) use keys derived from the owner-held WEBUI_SECRET_KEY.T2 What runs is what you see PartialOpen WebUIModel identity, per-model tools and system prompts, and effective config values (DB overrides env when ENABLE_PERSISTENT_CONFIG is on) are visible to the admin. No view shows the fully assembled context: the default task and tool-calling templates appear in the UI only as 'Leave empty to use the default prompt', several injected wrappers (memory context, skills list, conversation summary, Pyodide prompt) are hardcoded, and the DEBUG payload log is written before the model-level system prompt is applied. One shortfall is counted: assembled context content is not fully visible.T3 No shadow data PassOpen WebUIIn the default self-hosted install the provider holds nothing: the image disables Scarf/anonymised telemetry, OpenTelemetry is off, the licence server is contacted only with a licence key, and community sharing to openwebui.com happens only when a user clicks share. The openwebui.com privacy policy covers only the provider's own website and community service.T4 No hidden steering PassOpen WebUIThe source shows no paid placement, sponsored tools or remote announcements: there are no default models or tool servers, banners come only from local config, and web search is off by default. The enterprise/sponsor notice for instances above 50 users appears only in the admin user list and is not injected into the agent.A1 Complete action record FailOpen WebUIThe audit log is off by default (AUDIT_LOG_LEVEL=NONE), and even when enabled it excludes /chat, /chats and /folders by default, truncates bodies to 2048 bytes, and never sees automation runs, which bypass the HTTP middleware. Chat history does keep tool-call names and arguments, but actions inside Python tools and other side effects are not recorded separately, and there is no mechanism to alert the owner when recording fails. Counted shortfalls: unrecorded entry paths, and silent recording failure (G12).A2 Tamper-evident, owner-checkable FailOpen WebUIAudit log entries and chat records are plain JSON lines and database rows with no hash chaining, signatures or external anchor, so they can be changed without a trace.A3 Agent cannot rewrite its record PartialOpen WebUIChat history and the audit log are written by server middleware; the model's built-in tools can only search and view chats, not change them. However, Tools and Functions run as arbitrary Python inside the server process with full access to webui.db and audit.log, and the code-execution paths are not documented as isolated from the record. The protection is not documented as a guarantee.V1 Obtainable source, matching build PartialOpen WebUIAll components needed to run are public source that may be modified, and third parties can maintain forks; the branding clause restricts only rebranding above 50 users. Default Docker images are built by a public GitHub Actions workflow that emits an SBOM, but no signing, provenance attestation or reproducible-build procedure lets the owner tie the default ghcr.io image to the source.V2 Owner-rooted identity FailOpen WebUIThere is no agent identity: actions run under local user accounts, with JWTs or sk- API keys issued by the instance, and automations and subagents use short-lived JWTs for the owning user. Nothing lets a third party check that an action comes from the owner's agent.V3 Change history and rollback FailOpen WebUIOnly the prompt library has version history, with diff and restore. Model definitions and their system prompts, config, memories, tools, skills and permissions are single rows with only an updated_at timestamp and no record of what changed or who changed it. For the agent's state as a whole, the product offers no way to see or attribute changes.M1 Instructions and logic PartialOpen WebUIThe admin can edit the RAG template, the task templates (title, tags, follow-up, query, autocomplete, image, tool calling, voice), the code-interpreter and compaction templates, and per-model, per-user and per-chat system prompts. Several injected instructions are hardcoded and editable only in source: memory wrappers and the memory-review prompt, the Pyodide environment prompt, skill wrappers, the conversation-summary wrapper, the emoji and MOA templates, and the built-in tool descriptions.M2 Memory PassOpen WebUISettings > Personalization lists every memory item with edit and delete controls, plus delete-all. The other stores the agent reads across sessions (chats, notes, knowledge, calendar, automations) are itemised in the UI with edit and delete controls.M3 Tools PassOpen WebUIThe owner can add Python tools, OpenAPI tool servers and MCP servers with no provider review, and every built-in tool category can be switched off per model; many also have global switches.M4 Owner has the final say PartialOpen WebUIMost protections are owner-configurable: code-execution engine, safe mode, pip installs for plugins, file limits, and local web fetch. The built-in web-fetch blocklist (cloud metadata endpoints and reserved ranges) is always merged with the owner's list and cannot be removed without editing source.M5 No unilateral change PassOpen WebUIThe self-hosted instance changes only when the owner pulls a new image; the docs recommend pinning :vX.Y.Z tags, and Watchtower auto-updates are optional. The only remote data the code fetches from the provider is licence data, and only when LICENSE_KEY is set; there are no remote behaviour flags. Models are the owner's own connections.C1 Boundaries FailOpen WebUIPer-user and per-group permissions and per-tool access grants exist, but admins bypass them, and in a one-person business the owner is the admin. Web fetch and web search can be domain-filtered, but Python tools and MCP servers have no egress limits (on/off only). The model can create, update and toggle automations, which then run with tool approval forced to 'full', so the agent can route around the approval setting. Spend is treated as met under G11 (no payment capability). Counted shortfalls: on/off-only limits for tool egress, and a self-modification bypass path.C2 Approval gates PartialOpen WebUIA tool approval mode ('ask' or 'full') exists behind ENABLE_TOOL_PERMISSIONS (default off). It applies to all tools at once rather than per tool or action class, and automations and channel messages always run with 'full', bypassing it.C3 Immediate halt PartialOpen WebUIThe stop button halts the current chat's tasks, and admins can stop tasks one by one, but there is no single stop-all. Scheduled automations need a separate step (the global ENABLE_AUTOMATIONS switch or per-automation toggles).C4 Data serves only the owner PartialOpen WebUINo telemetry or content leaves by default, and sharing to openwebui.com is only per click. However, non-content outbound calls are on by default and not opt-in: the GitHub release check (ENABLE_VERSION_UPDATE_CHECK=true) and Hugging Face model auto-update at startup (RAG_EMBEDDING_MODEL_AUTO_UPDATE=true). Both can be turned off, for example with OFFLINE_MODE. There are no provider data terms for self-hosted use.C5 Credential custody PassOpen WebUIConnection API keys are listed and removable under Admin > Connections, user API keys can be viewed and deleted, and tool/MCP OAuth sessions can be revoked per provider. All credentials are stored in the owner's own database; the provider holds none.C6 Full deletion PartialOpen WebUINo provider-side copy exists by default, so deletion means removing the owner's Docker volume and container, which also holds the secret key file. There is no one-step in-app deletion of the whole instance, no documented full-deletion procedure, and no confirmation; in-app deletion covers individual chats, memories and non-primary users.
OpenCodeP1 Owner-usable round trip PartialOpenCode`opencode export` and `opencode import` move single sessions as JSON with one command each, but moving the whole agent (config dir, instructions, skills, credentials, database) means copying documented directories by hand. Export is available immediately.P2 Complete export PartialOpenCodeAll state is local, so nothing stays with the provider, and MCP OAuth uses dynamic client registration. However, the documented session export carries file-snapshot history only as hash references to a separate internal git store. The documented storage layout is also out of date (it describes per-project JSON storage, while the code uses opencode.db).P3 Runs without the provider PassOpenCodeThe runtime is MIT-licensed with no licence check. The model catalogue fetch from models.opencode.ai times out and falls back to a cache or bundled snapshot, and can be pointed elsewhere or disabled (OPENCODE_MODELS_URL, OPENCODE_DISABLE_MODELS_FETCH). Update checks can be disabled. Needs hands-on confirmation with provider domains blocked.P4 Model independence PassOpenCodeTools run in the runtime, and any AI SDK or OpenAI-compatible provider can be configured, including documented local servers (Ollama, llama.cpp, LM Studio). Web search, on by default only with the provider's own models, can be turned on for any model with documented OPENCODE_ENABLE_EXA / OPENCODE_ENABLE_PARALLEL flags. The docs say the core workflow relies on tool calling.P5 Exit survives termination PassOpenCodeSessions, config, credentials and snapshots live on the owner's machine (~/.local/share/opencode, ~/.config/opencode), so a current copy is always in the owner's hands whatever happens to the provider.T1 Readable without the provider PassOpenCodeState is in a local SQLite database whose tables are defined in public Drizzle schema files with JSON columns. Config and instructions are JSON/Markdown, snapshots are a git repository, and credentials are unencrypted JSON the owner holds. The published storage docs are out of date, but the schema in source is public.T2 What runs is what you see PartialOpenCodeEffective merged config is visible through the documented `opencode debug config`, and model and permissions are determinable. The fully assembled context (base prompt files, synthetic plan/build-switch reminders) has no documented viewer and is visible only in source or through experimental OpenTelemetry, so G3 caps this at partial. The default Zen model 'Big Pickle' is documented as a stealth model of undisclosed identity.T3 No shadow data FailOpenCodeOn the default path, prompts go through the provider's Zen gateway. The privacy policy says personal data is retained 'as long as necessary' for business purposes, with no bounded period and no item-by-item view. For some free models, collected data may be kept to improve the model.T4 No hidden steering PartialOpenCodeNo paid placement is visible in source, and the owner can replace the default model. However, the default-model ordering hard-codes the provider's own 'big-pickle', and with no key the provider's Zen free models are auto-loaded. The docs do not state this preference at first run, and no binding no-steering commitment exists.A1 Complete action record FailOpenCodeSessions record tool calls with inputs and outputs, including subagent child sessions. However, full tool outputs past the truncation limit are stored in tool-output/ and deleted after 7 days by default. Actions taken by plugin hooks are not recorded, and there is no documented alert when recording fails.A2 Tamper-evident, owner-checkable FailOpenCodeSession, message, part and event rows are plain SQLite with no hashes, signatures or chain, so any edit leaves no trace.A3 Agent cannot rewrite its record FailOpenCodeBash is allowed by default and runs without a sandbox as the owner's user, so the agent can modify or delete opencode.db and tool-output files. There is no documented protection for the record.V1 Obtainable source, matching build PartialOpenCodeAll runtime source is MIT and can be built by the owner, but the default distributed builds cannot be tied to source: the publish workflow sets NPM_CONFIG_PROVENANCE: false, and no reproducible-build or attestation process is documented.V2 Owner-rooted identity FailOpenCodeThe agent has no identity of its own. It acts through the owner's shell, git and provider credentials, and there is no owner-held key or other mechanism that attributes actions to the agent.V3 Change history and rollback PartialOpenCodeGit-based snapshots show agent-made file changes per session and restore them with /undo and /redo. That covers project-level AGENTS.md and opencode.json when the agent edits them. Global config, model and runtime-version changes are not tracked or attributed, and snapshots are pruned after 7 days.M1 Instructions and logic PartialOpenCodeAn agent's `prompt` config replaces the model-specific base prompt, and agents, skills and commands are owner-editable files. The synthetic plan and build-switch reminders and the max-steps prompt are hard-coded and can be replaced only by editing source.M2 Memory PassOpenCodeOpenCode has no separate hidden memory store. Knowledge carried across sessions lives in owner-editable instruction files (AGENTS.md, skills), and sessions can be listed and deleted with `opencode session`.M3 Tools PassOpenCodeOwners add tools via custom tool files, plugins and MCP servers without review. Any built-in tool can be removed with permission/tools deny rules, including wildcards.M4 Owner has the final say PassOpenCodeBuilt-in protections (.env read deny, doom_loop and external_directory prompts, snapshots, pruning, autocompaction) are all configurable through documented config or environment variables.M5 No unilateral change PassOpenCodeBy default, patch releases install silently at startup. The owner can set `autoupdate: false` or `"notify"`, pin a version, disable the remote model catalogue fetch, and use their own model provider, so every provider-initiated change can be deferred indefinitely (achievable basis).C1 Boundaries PartialOpenCodePer-tool, per-command, per-path and per-URL allow/ask/deny rules exist, and spend is treated as met under G11 because there is no payment capability. With default permissions, nothing stops the agent from editing its own project opencode.json or using the unsandboxed shell to get around webfetch domain limits unless the owner writes deny rules.C2 Approval gates PassOpenCodeThe owner can set any tool, or any command/path pattern, to 'ask', subagents inherit the rules, and auto-approval happens only with the explicit `--auto` flag.C3 Immediate halt PartialOpenCodeEsc interrupts the current session and cancels its subagents. There is no documented single stop covering concurrent sessions on a shared server or processes the shell has detached, and no documented guarantee (needs hands-on confirmation).C4 Data serves only the owner FailOpenCodeWith no key configured, the runtime auto-loads Zen free models and its ordering selects 'big-pickle', for which the Zen docs state that collected data may be used to improve the model during its free period. That is secondary use of content by default, without a specific opt-in. The terms also apply changes when the owner keeps using the service. Sharing is manual (opt-in), and OpenTelemetry export is off unless configured.C5 Credential custody PassOpenCodeCredentials are stored locally. `opencode auth list` shows each stored provider credential, and `opencode auth logout` and `opencode mcp logout` revoke each one individually. For the default free Zen path, the runtime uses the literal key 'public' rather than an owner credential.C6 Full deletion PartialOpenCode`opencode uninstall` removes all local config, data and snapshots in one confirmed command. Deleting data held by the provider's Zen gateway on the default path requires a privacy request by email, with no stated completion limit or confirmation.
nanobotP1 Owner-usable round trip PartialnanobotNo export/import command exists. State is plain local files under ~/.nanobot (config, workspace, sessions) that the owner can copy at any time without approval, but restoring on another machine means reinstalling nanobot and moving folders by hand (one shortfall: restore is not a single documented action).P2 Complete export PassnanobotAll agent state (config and credentials, instructions, Markdown memory with its source history, session JSONL, cron jobs and triggers, skills, plugins, files, channel sessions) is on the owner's disk, and HKUDS holds nothing. OAuth integrations re-link through the model vendors' own clients or MCP dynamic client registration, not an HKUDS-registered client; Codex and Copilot token files sit outside ~/.nanobot but can be recreated by logging in again.P3 Runs without the provider PartialnanobotThe runtime is MIT-licensed and needs no HKUDS account, licence key, relay or telemetry, so the core agent runs with HKUDS unreachable. The CLI Apps capability (on by default) resolves apps through a hard-coded, HKUDS-hosted catalog (hkuds.github.io/CLI-Anything/registry.json) that the owner cannot repoint; without it or a cached copy, installing and running CLI Apps fails.P4 Model independence PartialnanobotOllama, vLLM, LM Studio and any OpenAI-compatible endpoint are documented, and tools, memory, cron and channels run in the runtime. However, no minimum model contract is documented, and some capabilities are lost off hosted models: voice transcription has only hosted backends, and reference-image editing is unsupported for the local image providers.P5 Exit survives termination PassnanobotThe default offering is local-first: all agent state is written to the owner's disk, with no HKUDS account or hosted store, so provider termination or shutdown cannot remove access to it.T1 Readable without the provider PartialnanobotState is stored as JSONL, JSON, SQLite, Markdown and a git repo, with no nanobot-applied encryption. The docs give file locations and roles but no field-level schema for session records, run records, cron/trigger stores or the llm_usage SQLite tables.T2 What runs is what you see PartialnanobotModel, tools and access settings are visible through /status, nanobot status, the WebUI and config.json, and compaction summaries and runtime-context blocks are persisted with the session. The assembled system prompt, built each turn from package templates, has no documented viewer: the WebUI context view shows only the session part, and full request capture needs a third-party Langfuse plugin (OpenAI-compatible providers only), which G3 caps at partial. NANOBOT_* environment overrides are not shown in status (edge gap, not counted).T3 No shadow data Passnanobotnanobot is self-hosted, and HKUDS operates no service that receives owner content or usage data. The code contains no telemetry or analytics, and the only HKUDS-related requests (an owner-triggered PyPI version check, the GitHub-hosted CLI Apps catalog, a TUI binary fallback) carry no content.T4 No hidden steering PassnanobotThe source shows no paid placement or self-preference in the agent's context, provider order, search default (DuckDuckGo) or tool choices, and the setup has no preselected partner provider. Partner and affiliate links appear only in the README and docs. The bundled clawhub skill is visible and can be switched off with disabledSkills.A1 Complete action record FailnanobotMain, cron, heartbeat and trigger turns are saved to session JSONL with full tool-call arguments. However, subagent tool calls stay only in memory and Dream runs are ephemeral (unrecorded entry paths), nothing tells the owner when an action goes unrecorded, and by default tool results over 16,000 chars are cut to a 1,200-char preview whose full copy is deleted after 7 days, with run history capped at 20 entries (three shortfalls).A2 Tamper-evident, owner-checkable FailnanobotSession JSONL is rewritten in full on each save, and sessions, run records and usage data carry no hashes, signatures or chain, so the record can be changed without trace.A3 Agent cannot rewrite its record PartialnanobotBy default (restrictToWorkspace false, no exec sandbox, full access mode, shell guard skipped), the agent's shell and file tools can edit or delete its session JSONL, usage database and logs. The documented bwrap/seatbelt sandbox masks ~/.nanobot from the agent, but cron/trigger run records and tool-result offloads stay in the writable workspace, and Windows has no sandbox.V1 Obtainable source, matching build PartialnanobotAll components are open source: the core is MIT, the WebUI and TUI source ship in the repo, and the Bun runtime is LGPL with a source offer. The default PyPI wheels (with a prebuilt ~119 MB TUI binary and WebUI bundle) are uploaded by hand, with no attestations, provenance or reproducible-build claim, so the distributed build cannot be tied to source independently.V2 Owner-rooted identity Partialnanobotnanobot has no agent-level identity (no keypair, DID or agent card). The agent acts through owner-configured channel accounts, such as an email sender on the owner's domain, which HKUDS cannot revoke or impersonate, so attribution is per channel only.V3 Change history and rollback PartialnanobotA dulwich git repo versions SOUL.md, USER.md and MEMORY.md, and /dream-restore reverts a Dream commit. All commits are attributed to Dream, owner or agent edits between Dream runs are folded into the next Dream commit, and config, skills, AGENTS.md, cron, the model and the runtime version have no history or rollback (agent self-changes via the my tool are only a log line).M1 Instructions and logic PartialnanobotSOUL.md, USER.md, AGENTS.md, the Dream and evaluator prompts and built-in skills can be overridden or disabled from the workspace, and updates never overwrite owner files. The core system-prompt templates (identity, platform policy, tool contract and others) load only from the installed package, with no documented override.M2 Memory PartialnanobotSOUL.md, USER.md and memory/MEMORY.md are plain, editable Markdown. The cross-session memory/history.jsonl archive is documented as append-only, with no supported item-level edit or delete, and past sessions the agent can search and read can only be deleted as whole topics.M3 Tools PartialnanobotOwner tools can be added without review through MCP (with enabledTools filters), entry-point plugins, skills and CLI apps, and exec, file, web, my, CLI Apps and image generation each have an enable switch. Built-ins such as message, spawn, cron, goals and the session tools have no switch and register whenever their services exist.M4 Owner has the final say PartialnanobotSandbox, workspace restriction, SSRF guard (via ssrfWhitelist), exec timeout, rate limit and the exec deny list (via allowPatterns) are all owner-configurable. The http/https-only URL restriction for web_fetch and MCP HTTP, and the refusal to bind the WebUI to 0.0.0.0 without auth, have no owner override.M5 No unilateral change PassnanobotThere is no auto-update, remote flag or server-side setting. The PyPI version check runs only when the owner clicks it and installs nothing, and model, instruction and runtime changes happen only when the owner upgrades or edits config.json, so a self-hosted install stays pinned indefinitely.C1 Boundaries FailnanobotInstruction sources are limited per sender (allowFrom plus pairing), and spend is met under G11 (no payment capability). Contact and network egress are on/off only (no per-recipient or per-domain lists). By default the shell guard is skipped and the agent's file and shell tools can rewrite ~/.nanobot/config.json and pairing.json, changing its own permissions (two shortfalls, and limits can be overridden by model output).C2 Approval gates Failnanobotnanobot v0.3.5 has no mechanism for the owner to require approval before a tool call or action class runs. Access modes are scope limits, and approval exists only for pairing new senders.C3 Immediate halt Passnanobot`nanobot gateway stop` terminates the single local process that runs turns, subagents, cron, heartbeat and shell sessions (SIGTERM, then SIGKILL after a 20 s default). Interrupted turns come back as awaiting_user rather than replaying. In-chat /stop covers one session only, and completion within seconds needs a hands-on check.C4 Data serves only the owner Partialnanobotnanobot sends no telemetry, and HKUDS receives no owner content. By default web search queries go to DuckDuckGo and every fetched URL goes to Jina Reader (useJinaReader true), third-party destinations the owner did not specifically choose, disclosed only in the configuration reference. Both can be switched off, and no data terms are published.C5 Credential custody PartialnanobotAll credentials are local, in plaintext config.json and OAuth token files, and HKUDS holds none. Each can be removed through `nanobot provider logout`, WebUI settings or MCP server removal. There is no single list of credentials (Codex and Copilot tokens sit in platform data folders outside ~/.nanobot), and after Codex logout the oauth-cli-kit dependency re-imports ~/.codex/auth.json if present.C6 Full deletion FailnanobotNo uninstall-and-purge command or documented delete-all procedure exists; `gateway uninstall-service` removes only the service unit. Agent data spans ~/.nanobot, OAuth token folders outside it and the installed package environment, and nothing confirms removal.
gooseP1 Owner-usable round trip PartialgooseDesktop has an in-app Export and Import pair for sessions (JSON), and the CLI has `goose session export`; both work at once on the owner's machine. Restoring the rest of the agent (config.yaml, permission.yaml, memory files, prompt overrides, skills, schedules, keyring secrets) means copying files and re-entering keys by hand, and there is no documented procedure for it. That is one shortfall.P2 Complete export PartialgooseThe only export is the session file (conversation, tool calls with arguments, metadata). Config, memory, prompt overrides, skills/recipes, schedules and credentials are not in it. All of these stay on the owner's disk at documented paths, the provider holds none of them, and integrations re-link with the owner's own keys. So neither fail condition is literally met, but more than one class is missing from the export, so this does not reach pass.P3 Runs without the provider PassgooseThe runtime is Apache-2.0 and has no licence check, account or provider service in the core. Local inference is a default build feature, and the docs root and other endpoints are configurable. Settled only hands-on; credited here from source.P4 Model independence PartialgooseAny provider can be chosen, including built-in local inference and Ollama. A tool shim and a tiny-model prompt cover models without native tool calling, and dictation has a local Whisper option. But Desktop's Live Voice feature has only an OpenAI realtime implementation, so it is lost on other models.P5 Exit survives termination PassgooseAll agent state is local-first on the owner's device, and there is no account whose termination could cut off access.T1 Readable without the provider PassgooseSessions are in a documented SQLite database whose schema is in the open source. Config is YAML, memory and prompts are text/Markdown, LLM request logs are JSONL, and secrets sit in the OS keyring or secrets.yaml under the owner's control.T2 What runs is what you see PassgooseEach LLM request's full payload is written locally to llm_request.*.jsonl, including the system prompt, turn-context blocks and compaction output. Only the 10 most recent are kept, which is an edge-path gap and is not counted. Model, mode, extensions and tool permissions are all visible in config and settings.T3 No shadow data PassgooseOperation is local. Telemetry is opt-in and anonymous, and the provider holds no content or profile. Model-provider data handling is the owner's chosen destination.T4 No hidden steering PartialgooseNothing in the source shows paid placement, and no provider or model is preselected. However, onboarding sorts a hard-coded 'Preferred' set of providers first, the criteria for that set are not disclosed, and it cannot be switched off. There is no binding no-steering commitment.A1 Complete action record FailgooseTool calls with arguments from user, subagent and scheduled sessions are stored durably in sessions.db. But hook executions (Open Plugins hooks, which also auto-load from project directories) appear only in logs that are deleted after two weeks. No public evidence says the owner is alerted when a record write fails, so under G12 that shortfall counts as present. Two shortfalls.A2 Tamper-evident, owner-checkable FailgooseThe session database and logs carry no hashes, signatures or checkpoints, so any edit leaves no trace.A3 Agent cannot rewrite its record FailgooseIn the default Autonomous mode, with no sandbox, the Developer extension's shell can write to ~/.local/share/goose/sessions/sessions.db and the log directories. Nothing stops the agent from changing its own record.V1 Obtainable source, matching build PassgooseAll components, including the Electron Desktop app and goosed, are Apache-2.0 source. The release workflow attests build provenance for release artifacts (Linux/macOS/Windows bundles, deb/rpm/flatpak, download_cli.sh) using actions/attest-build-provenance (Sigstore/SLSA), which can be checked with `gh attestation verify`. Not verified hands-on.V2 Owner-rooted identity FailgooseThe default offering gives the agent no identity of its own; it acts only through the owner's credentials for each integration.V3 Change history and rollback FailgooseChanges to settings, permissions, memory files, prompts or extensions are not versioned or attributed, and the product cannot restore them. The only trace is that agent-made memory writes appear as tool calls in the session history. Prompts have 'Reset to Default' only.M1 Instructions and logic Partialgoosesystem.md, compaction, subagent and other templates can be replaced from the Desktop Prompts tab or ~/.config/goose/prompts, and the overrides persist across updates. Built-in extension instructions, such as developer_instructions(), and the chat-mode injection are hard-coded and can only be changed in source.M2 Memory PassgoosePersistent memory lives in documented text files: global at ~/.config/goose/memory/ and per-project at .goose/memory/. Hints, AGENTS.md and skills are plain files, and sessions used by Chat Recall can be deleted one by one. All of it can be itemised, edited and deleted.M3 Tools PassgooseAny MCP server (stdio or remote) can be added without review, and every built-in or platform extension can be toggled off. The GOOSE_ALLOWLIST restriction is off by default.M4 Owner has the final say PartialgooseMost protections are owner settings: permission mode, prompt-injection detection (off by default) and the turn limit. But the OSV malware block on npx/uvx extensions has no documented override. The docs tell the owner to find an alternative or file an issue, and the only way to redirect the check is an undocumented OSV_ENDPOINT environment variable.M5 No unilateral change PassgooseThere are no remote flags, and prompts and the model catalogue are baked into each release. By default Desktop downloads updates automatically and installs them on quit, but the owner can turn this off in Settings > App > Updates ('Disable automatic update downloads') and stay on any version indefinitely. CLI updates are manual. Models are chosen and pinned by the owner.C1 Boundaries FailgooseLimits exist per tool (Always, Ask or Never), but there are no contact or egress limits per recipient or domain. The agent's shell can also edit its own config.yaml and permission.yaml, since nothing sandboxes the config directory. Spend is not applicable (G11): goose has no payment capability. Two shortfalls.C2 Approval gates PartialgooseThe owner can put any tool behind 'Ask Before' in Manual or Smart mode. But scheduled jobs (the Scheduler extension, on by default) always create their agent and session in GooseMode::Auto, which approves every tool, so that path bypasses the gates.C3 Immediate halt PartialgooseThe owner can stop the foreground turn. No single documented control also stops scheduled jobs; those must be paused or removed separately. Settled only hands-on.C4 Data serves only the owner FailgooseTelemetry is opt-in and content goes only to the owner-chosen model provider. But Desktop runs an update check against GitHub at every startup, and the owner can turn off only the download, not the check. Adding an npx/uvx extension also sends its package name to the OSV service, with no documented off switch. Both flows run by default and cannot be switched off.C5 Credential custody PartialgooseCredentials are held locally in the OS keyring or secrets.yaml, and the provider never holds them. Provider keys can be removed one at a time. However, no documented single view lists every credential and grant, such as extension OAuth tokens, with a revoke control for each.C6 Full deletion PartialgooseThe provider holds no copies. Full removal is a documented multi-step manual procedure: uninstall, delete the config, share and state directories, and clear keyring entries. There is no confirmation.
OpenHandsP1 Owner-usable round trip FailOpenHandsThere is no documented export of the whole agent and no matching import. First-party exports cover only single automations (JSON, with import) and single conversations (Markdown/HTML transcript or trajectory zip, with no import). Moving the agent means copying ~/.openhands, whose layout and backup procedure are not documented, plus browser localStorage.P2 Complete export PartialOpenHandsAll agent state lives on the owner's machine (JSON event files, base_state.json, encrypted settings/secrets, automations SQLite, skills and memory files), so nothing stays with the provider. However, built-in GitHub, Bitbucket DC and Jira DC automation triggers are designed around events forwarded and signed by the OpenHands server, and OAuth-based MCP marketplace entries need the hosted integration-auth flow. Those integrations cannot be re-linked without the provider.P3 Runs without the provider PassOpenHandsThe whole self-hosted stack (Canvas, agent-server, SDK, tools, automation) is MIT-licensed and runs locally from versions pinned in config/defaults.json. No licence check, account or provider service is needed to start or act. The default provider-bound calls (PostHog telemetry, npm update check) are non-essential. Hands-on blocked-domain testing was not performed.P4 Model independence PassOpenHandsModels are accessed through LiteLLM with owner-supplied keys. Local open-weight models (Ollama, LM Studio, vLLM) are documented with a minimum context window. A prompt-based tool-calling fallback exists for models without native function calling, and tools run in the runtime rather than as hosted model features. Not verified hands-on.P5 Exit survives termination PassOpenHandsIn the self-hosted offering all state stays on the owner's machine under ~/.openhands, so provider termination, a ban or a shutdown cannot remove it. Uninstalling does not delete it.T1 Readable without the provider PartialOpenHandsState is stored in open formats: per-event JSON files, base_state.json, a SQLite automations DB and Markdown skills and memory. Secrets are Fernet-encrypted with a key stored locally (owner-held). No published schema or state-layout document exists; the format is defined only by source code, and the conversation path differs between npm (dev_conversations) and Docker installs.T2 What runs is what you see PartialOpenHandsThe UI shows the system message and available tools per conversation. Condensation summaries are persisted as events and included in the trajectory download. There is no view of the fully assembled context for a given LLM call, and the public evidence does not show whether injected user-message suffixes and skill triggers are visible, so one context-content shortfall is counted (G12). Model, tools, confirmation and analyzer settings are visible in Settings.T3 No shadow data FailOpenHandsThe agent's state stays local. On the default path, however, the provider receives PostHog telemetry about the owner's activity: an install event before consent, plus usage events after the pre-checked consent box. The privacy policy gives no bounded retention ('as long as reasonably necessary'), and the owner cannot view these records.T4 No hidden steering PartialOpenHandsNo paid placement was found in the inspected source. First-party defaults are visible and replaceable: the Add Backend dialog preselects OpenHands Cloud, the OpenHands LLM provider appears in the model list, and the default prompt asks for an 'openhands' co-author trailer. There is no binding no-steering commitment, and the ranking inside the bundled @openhands/extensions catalogue was not inspected.A1 Complete action record PassOpenHandsEvery action and observation, including terminal, file edit, browser, MCP and ACP tool calls with their arguments and results, is persisted as a JSON event before subscribers are notified. Subagents are persisted as child conversations, and automation runs are recorded with trigger source and linked conversation. A failed persist raises rather than failing silently, and events are not pruned by default. Graded from source; not verified hands-on.A2 Tamper-evident, owner-checkable FailOpenHandsThe event log has no hashing, signing or chaining, so edits or deletions cannot be detected.A3 Agent cannot rewrite its record FailOpenHandsIn the default local runtime the agent's terminal and file tools run as the same OS user and are not confined to the workspace, so they can modify or delete the persisted event files. The per-conversation Docker mode also mounts the conversation's persisted state into the sandbox.V1 Obtainable source, matching build PartialOpenHandsSource for every required component is MIT-licensed. The Canvas npm package is published with npm provenance, but the agent-server and SDK PyPI packages that the launcher installs are published with a token and no attestation. So the default build of a required component cannot be tied to source independently.V2 Owner-rooted identity FailOpenHandsThe agent has no identity of its own: no keypair, DID or signed actions. Outbound actions use the owner's tokens, and the default prompt suggests an 'openhands <openhands@all-hands.dev>' co-author trailer.V3 Change history and rollback PartialOpenHandsThe product offers change history for some state only. Automations can be synced two-way to a git repository (Git Sync), and conversations have fork/branch history. Settings, LLM profiles, skills, memory, secrets and model or runtime changes have no change log, attribution or restore.M1 Instructions and logic PartialOpenHandsThe owner can add instruction layers (AGENTS.md, skills, SOUL.md identity line) and disable bundled skills in the UI. Replacing the base system prompt (system_prompt, system_prompt_filename) is exposed only as SDK agent fields, not in the Canvas settings or in documented config (G3).M2 Memory PassOpenHandsPersistent memory is plain Markdown the owner can itemise, edit and delete with any editor: AGENTS.md, ~/.openhands/memory/MEMORY.md, the workspace .openhands/memory and skills. No hidden profile store was found.M3 Tools PartialOpenHandsOwners can add any MCP server (stdio, SSE or HTTP) and custom skills without review. Built-in tools can be removed only through SDK agent fields (include_default_tools, filter_tools_regex); the Canvas UI toggles only a few (sub-agents, switch-LLM).M4 Owner has the final say PassOpenHandsNo locked provider protection was found. Sandboxing mode, confirmation mode, the security analyzer and telemetry are all owner-configurable, and the default protections are already off. The prompt's security policy can be removed via security_policy_filename.M5 No unilateral change PassOpenHandsThe self-hosted stack pins the agent-server, automation and extensions versions, updates are manual (npm/docker commands shown in the UI), and no remote feature flags were found. Note: conversations launched via agent profiles re-fetch public skills from the OpenHands/extensions main branch (60 s cache) unless EXTENSIONS_REF is set; the default launch path uses bundled, pinned skills.C1 Boundaries FailOpenHandsThere are no per-recipient or per-domain contact limits and no egress policy; the contact dimension is missing. In the default local runtime the agent can edit its own settings and hook files under ~/.openhands and the workspace, a bypass path. Spend was treated as met because the product has no payment capability (G11).C2 Approval gates PartialOpenHandsConfirmation mode (off by default) gates either every action or only those the LLM analyzer rates risky, which is the model's own risk label. The UI has no per-tool or per-class gate, and PreToolUse hooks can block but not request approval.C3 Immediate halt PartialOpenHandsEach conversation can be paused or stopped and each automation turned off, but there is no single global stop for all conversations and scheduled automations.C4 Data serves only the owner FailOpenHandsUsage telemetry is effectively opt-out: the consent checkbox is pre-checked, and the agent-server and automation service are launched with PostHog keys. A canvas_install event with user agent, referrer and origin URL is sent before any consent, and it is disclosed only in a code comment. The privacy policy does not address staff access, and changes take effect on posting.C5 Credential custody PartialOpenHandsSecrets are listed by name and deletable in Settings, LLM keys live in deletable profiles, and git tokens are ordinary secrets. The provider holds no credentials on this path. Public evidence does not settle whether MCP OAuth tokens stored in the settings store are visible or revocable individually, so the pass is withheld (G12).C6 Full deletion FailOpenHandsThere is no single owner action that deletes the whole agent. Conversations ('Delete all'), automations and secrets are deleted separately, ~/.openhands survives uninstall, and deleting provider-held telemetry requires emailing the provider.
AnythingLLMP1 Owner-usable round trip FailAnythingLLMThe only documented exports are chat logs (CSV/JSON/JSONL) and per-chat exports. Desktop documentation gives the storage folder location but no documented backup, export, import or migration procedure for the whole instance or a workspace, so there is no documented export of the agent.P2 Complete export FailAnythingLLMThe default export route covers conversation text only. Configuration, prompts, memories, custom skills, flows, scheduled jobs, documents, embeddings and integration settings are not in any export; they remain only in the local storage folder.P3 Runs without the provider PartialAnythingLLMThe free Desktop app needs no account or licence check, and the self-hosted terms say it needs no connection to Mintplex servers (the embedder has a Hugging Face primary source with a Mintplex CDN fallback). However, the Desktop Terms of Use grant only a non-exclusive, non-transferable licence that they do not state is perpetual or irrevocable, so public evidence does not establish an irrevocable right to run the distributed Desktop binary (G12). The MIT server core offers another route, but no documented migration path onto it exists.P4 Model independence PassAnythingLLMThe default LLM is a built-in local engine (Ollama). Many local providers are supported (Ollama, LM Studio, LocalAI, etc.), and so are local embedders, speech-to-text, text-to-speech and image generators (Ollama, LocalAI, Lemonade). Agent tools run in the runtime, and an 'UnTooled' prompt-based fallback covers models without native tool calling. The only model-locked feature found is a beta-preview computer-use feature fixed to an Anthropic model, which is not part of the evaluated default.P5 Exit survives termination PassAnythingLLMDesktop needs no account. All state (anythingllm.db, documents, vector DB, plugins, .env) lives in a local storage folder on the owner's device, so a current copy is always in the owner's hands whatever happens to the provider.T1 Readable without the provider PassAnythingLLMState is kept in SQLite with a public Prisma schema, JSON documents and vector cache, LanceDB (an open format), JSON flow definitions, JavaScript skill source and a plaintext .env, all in an owner-held folder. Built-in skill and prompt logic is published as MIT source, and no provider-held keys are involved.T2 What runs is what you see PartialAnythingLLMThe UI shows the workspace system prompt, the model, and enabled skills and MCP tools. Some context additions can be seen only in source code: the agent role prompt, the appended clarifying-questions instruction, the UnTooled function-picker prompt and the memory header. No view or log of the fully assembled context was found, so G3 caps this test at partial.T3 No shadow data FailAnythingLLMOn by default, the app sends usage events (chat sent, agent tool name, documents uploaded, LLM provider and model, etc.) to PostHog with a stable per-instance UUID. The Desktop privacy policy does not disclose how long this data is kept. The README says sent events appear in the local output log, but retention is undisclosed.T4 No hidden steering PassAnythingLLMThe one provider-chosen default found is You.com as the agent web-search provider, which the changelog and blog disclose as a partnership. The owner can switch the search provider or disable web browsing. Source code shows no sponsored or paid placement in agent context or tool defaults. The Community Hub pages could not be opened (HTTP 403).A1 Complete action record FailAnythingLLMInteractive agent sessions store only the final response in workspace_chats. Tool calls and their arguments go only to the live websocket and are not stored (scheduled-job runs do store a full trace). No mechanism tells the owner when recording fails. Two shortfalls apply: entries lack arguments and targets, and recording can fail silently.A2 Tamper-evident, owner-checkable FailAnythingLLMChats, event logs, invocations and job runs are plain SQLite rows with no hashing, signing or chaining. An admin endpoint can clear all event logs.A3 Agent cannot rewrite its record PartialAnythingLLMBuilt-in default skills cannot write the chat or event tables. The SQL agent connects only to external databases, and the filesystem agent is sandboxed and available only in Docker. However, owner-enabled custom skills (in-process Node handlers) and stdio MCP servers run arbitrary code with access to anythingllm.db, and nothing separates the record from them. The protection holds by default except for this one path.V1 Obtainable source, matching build FailAnythingLLMThe Desktop Electron wrapper, a required component of the evaluated offering, has no public source. Its Terms of Use call the software's source code a trade secret and forbid modification and reverse engineering. The server core is MIT, and release binaries ship without checksums or provenance.V2 Owner-rooted identity PartialAnythingLLMThe agent has no identity of its own. Outbound actions go through channels the owner holds, such as a Gmail Apps Script deployment the owner runs or an Outlook OAuth app the owner registers, with no Mintplex client or broker. Attribution is therefore per channel only.V3 Change history and rollback PartialAnythingLLMWorkspace system prompt changes are kept in a prompt history with author and one-click restore, and the event log records that the LLM, embedder or vector DB changed (without values). Other settings, skills, memories and model changes have no change history or restore.M1 Instructions and logic PartialAnythingLLMThe workspace system prompt and the instance default system prompt can be edited in the UI, and custom skills and flows are owner-defined. Several hardcoded instructions (agent role prompt, clarifying-questions instruction, UnTooled tool-selection prompt, memory header) can be changed only by editing source.M2 Memory PartialAnythingLLMMemories (workspace and global) are listed with create, edit and delete controls. However, the default-on rag-memory skill's 'store' action embeds text straight into the workspace vector DB with no document record, so those items are not itemised or individually editable in the UI.M3 Tools PassAnythingLLMThe owner can add custom skills (plugins/agent-skills handler.js), MCP servers and Agent Flows without provider review. Every built-in skill, including the four default-on ones, can be disabled, and individual MCP tools can be suppressed.M4 Owner has the final say PartialAnythingLLMSome protections are configurable (tool-call chain limit, approval timeout, MCP cooldown, auto-approved skills), but on Desktop only through environment variables that the docs tell Desktop users not to edit. Others are hardcoded: the 100-round agent limit, the 5-minute agent websocket timeout, and the memory count limits.M5 No unilateral change PassAnythingLLMDesktop updates are owner-initiated reinstalls (an in-app banner only links to downloads), and no remote flag or server-side configuration code was found. The default model is local. Keeping the installed version acts as an indefinite pin.C1 Boundaries FailAnythingLLMContact and web reach are on/off only: the Gmail/Outlook skills and web browsing/scraping have no per-recipient or per-domain allow or deny lists. There is also no owner control over which sources the agent takes instructions from. Spend is treated as met under G11 because there is no payment capability. File scope has per-folder grants, but only in Docker.C2 Approval gates PartialAnythingLLMApproval prompts exist only for a fixed set of built-in write actions (filesystem writes, file creation, Gmail/Outlook/Calendar writes, job creation) and for custom skills that request them. MCP tools, SQL, web and flows cannot be gated, and scheduled-job runs auto-approve every tool.C3 Immediate halt PartialAnythingLLMThe stop button fully aborts the foreground agent session and inference (v1.16.0). Scheduled jobs must be stopped per run or disabled separately, and background jobs such as memory extraction keep running. There is no single global stop.C4 Data serves only the owner PartialAnythingLLMNon-content telemetry is on by default and can be disabled in Settings > Privacy. By default, agent web-search queries go to You.com, a destination the owner did not specifically choose, though web browsing can be turned off. The privacy policy says data is not sold. Policy changes take effect after posted notice, without specific consent.C5 Credential custody PartialAnythingLLMAll credentials are held locally: provider keys in the plaintext storage .env, integration configs in SQLite. Mintplex holds none and uses no broker. The UI shows only whether each key is set, across several settings pages, and LLM provider keys can be overwritten but not individually revoked or listed in one place.C6 Full deletion PartialAnythingLLMNo provider-side copy of the agent exists. Full deletion is a documented multi-step manual process (uninstall, then delete the storage folder) with no in-app single action or completion confirmation. Telemetry already sent to PostHog is not agent state.
DeepSeek HarnessP1 Owner-usable round trip FailDeepSeek HarnessThe only documented export is a per-session ZIP (session log, sub-sessions, attachments) downloaded through the browser. There is no import, restore or backup command, and no documented procedure for copying $DSH_HOME to another host. Session folders are keyed by absolute working-directory paths.P2 Complete export PartialDeepSeek HarnessAll state classes live in owner-held files under $DSH_HOME and nothing stays only with the provider: - sessions, KV storage, profile patches, .credentials.yaml, attachments, skills and AGENTS.md - integrations use API keys or owner-generated webhook secrets, not provider OAuth clients The documented export omits settings, credentials, skills, instructions and workspaces, and there is no restore route. The fail definition (classes held by the provider, or text-only export) is not met, so the test stays partial under G1.P3 Runs without the provider PassDeepSeek HarnessThe runtime is MIT-licensed, with no licence key, login, remote-config fetch or update check in the CLI/Web path; the official build profile only embeds version strings. DeepSeek endpoints are all configurable or replaceable: - model API - default web_search provider (Exa and Perplexity plugins exist) - OTel collector (DSH_TELEMETRY_OTLP_URL, or disable) This is a desk grade; the provider-blocked run is hands-on.P4 Model independence PartialDeepSeek HarnessOther providers are supported, and custom OpenAI Chat, OpenAI Responses and Anthropic Messages endpoints work at any base URL, including self-hosted ones. Capabilities are model-neutral: titles and compaction use the session route, and vision works for any model declared image-capable. However, there is no documented minimum model contract, no text-based tool-calling fallback, and no local open-weight model is named or tested (G12).P5 Exit survives termination PassDeepSeek HarnessAll agent state is kept locally under $DSH_HOME on the owner's machine, with no hosted account or sync. Termination of a DeepSeek account ends only model access and does not remove the owner's copy.T1 Readable without the provider PartialDeepSeek HarnessState is kept in open formats: - session logs are JSONL (zstd-compressed by default) with a public event catalog and JSON schema - KV storage is JSON - credentials and config are YAML - no encryption The catalog marks some fields as opaque: replayState, ReplayEnvelope.response, tool-private meta, and provider thinking signatures. Storage-domain schemas exist only as zod definitions in source.T2 What runs is what you see PartialDeepSeek HarnessThe following are visible to the owner: - system prompt, dynamic context and compaction summaries are logged as session events - the per-request header with config and full tool schemas is logged, and the Trajectory view shows per-request system prompt and tools - `dsh --dump-config` shows every layer - DeepSeek's docs document what the `deepseek-flash` alias resolves to One shortfall is counted under G12: no public evidence settles whether the hosted API adds server-side context. Unevaluated `!!js` config expressions are recorded but not counted.T3 No shadow data FailDeepSeek HarnessDeepSeek receives and holds several kinds of data about the owner's use: - by default, the uploaded session log: working directory, prompts, user and assistant content, tool arguments and results, feedback - request headers carrying a stable anonymous installation UUID and the plugin inventory - feedback-triggered OTel uploads of the session prefix DeepSeek's privacy policy gives only an 'as long as necessary' retention, and nothing lets the owner view that data item by item.T4 No hidden steering PartialDeepSeek HarnessNo paid placement or sponsored tools were found in source. The first-party defaults (DeepSeek model, DeepSeek-hosted web search) are disclosed in config and replaceable, and the plugin registry is plain npm with no ranking. The absence of steering in the hosted search and model results rests on no binding commitment.A1 Complete action record PartialDeepSeek HarnessThe append-only session log records the following, and a tool does not run until its checkpoint write succeeds: - tool calls with full arguments - PTC sub-calls, subagents as child sessions, workflows - approvals and hooks One shortfall: by default, tool results over 12,500 tokens are replaced in the log by a head/tail preview. The full output goes to a temp spill file that is deleted after 30 days.A2 Tamper-evident, owner-checkable FailDeepSeek HarnessThe session log has no hash chain, signatures or anchoring. The only integrity data is zstd's per-frame content checksum, which detects corruption but not deliberate edits. The SHA-256 values in the docs are schema-type fingerprints, not record integrity.A3 Agent cannot rewrite its record PartialDeepSeek HarnessIn the default workspace-write sandbox, writable roots are the workspace and temp directories, so ~/.dsh/sessions is normally out of reach. This protection is incidental rather than an explicit deny. It fails if the workspace contains $DSH_HOME, for example a workspace at the home directory, and the linked spill files sit in writable temp.V1 Obtainable source, matching build PartialDeepSeek HarnessSource is available: - all required components are MIT source, with vendored Cordis - the native Landlock addon source is BSD-3, though it ships as prebuilt optional packages - the only closed dependency (Claude Agent SDK) sits in an optional, disabled subagent The npm publish job uses a token without `--provenance`, SLSA or sigstore, so a third party cannot tie the published build to the source.V2 Owner-rooted identity FailDeepSeek HarnessThe only identity is a random anonymous installation UUID used to correlate telemetry and DeepSeek requests. It has no keypair, DID or signing, and actions are not attributable to it. The only other identity is the provider's DeepSeek platform account.V3 Change history and rollback PartialDeepSeek HarnessPer-session changes are recorded as attributed events: model selection, permission and sandbox mode, preset, system-prompt snapshots, tool addition and removal. A session can be forked at any event. There is no history or rollback for global state (profile config, credentials, skills, AGENTS.md), and workspace file diffs cannot be reverted.M1 Instructions and logic PartialDeepSeek HarnessThe owner can replace the whole system prompt through a preset persona with `complete: true`, adjust prompt prefix and suffix and plan-mode text in YAML, and edit AGENTS.md files; no default skills ship. Built-in tool descriptions for bash, read, write, edit and grep are hard-coded with no config field.M2 Memory PassDeepSeek HarnessThe default Web offering has no built-in memory store the agent reuses across sessions; goals, todos and compaction are session-scoped. What does persist across sessions is plain files the owner can view, edit and delete: AGENTS.md instruction files and skill directories. Memory MCP servers are default-off examples.M3 Tools PassDeepSeek HarnessOwners can add any npm, git, tarball or local plugin, or MCP server, through `dsh plugin add` or the Plugins dialog without provider review. Built-in tools can be removed with documented profile-patch edits (`disabled: true`, or overriding a preset's plugin list).M4 Owner has the final say PartialDeepSeek HarnessThe owner can turn off the sandbox and approvals (danger-full-access preset), and can tune or disable the read-before-write check, the repeat-tool reminder and job caps. web_fetch's block on private and non-public destinations has no config switch.M5 No unilateral change FailDeepSeek HarnessThe default model is the hosted alias `deepseek-flash`, whose target DeepSeek changes server-side; its docs show retired model names silently served by DeepSeek-V4.1-Flash. The documented launch `npx @deepseek-ai/dsh web` is unpinned, and no pin or pre-update notice is documented. Plugins do not auto-update.C1 Boundaries FailDeepSeek HarnessInstruction sources are limited: the server binds to loopback with token auth, and schedules and webhooks are off; spend is treated as met under G11. Two limit dimensions are missing: - **Contact and egress:** web_fetch runs in every mode, and there is no per-domain allow or deny list. - **Data scope:** reads are not confined in any preset, so the agent can read .credentials.yaml.C2 Approval gates PartialDeepSeek HarnessBuilt-in gates cover a fixed set of classes: sandbox escalation, plugin_manager actions, and the ask/never mode. Gating an arbitrary tool needs a PreToolUse hook command (G3). In-process subagents run with approval `never`, so their asks are rejected rather than bypassed.C3 Immediate halt PartialDeepSeek HarnessStop cancels the foreground turn and keeps queued input. Background jobs and continuable subagents need separate stops. Archiving a session stops all of its work, and the only global stop is killing the process.C4 Data serves only the owner FailDeepSeek HarnessBy default (`enabled: true`) every official-API request carries the canonical session log. The only control is an opt-out switch labelled 'Help improve DeepSeek models and products'; DeepSeek's privacy policy lists training as a use, with opt-out only. Other flows: - one feedback click uploads the session prefix to a DeepSeek OTel collector; this can only be disabled by environment variable - data-term changes apply on noticeC5 Credential custody PartialDeepSeek HarnessKeys are stored in owner-held $DSH_HOME/.credentials.yaml. The UI is write-only for secrets and lets each provider credential be deleted. There is no single list of every credential: environment and .env credentials and the session-signing secret are not shown. Signing out of the DeepSeek account does not revoke the grant remotely.C6 Full deletion FailDeepSeek HarnessSessions can be archived but never deleted, and there is no documented whole-agent deletion. No deletion route exists for session logs or OTel exports already uploaded to DeepSeek; withdrawing feedback exports a deletion event, not remote erasure.
Open InterpreterP1 Owner-usable round trip PartialOpen InterpreterAll agent state is local under ~/.openinterpreter (relocatable via INTERPRETER_HOME) plus AGENTS.md and ~/.agents/skills, so it is available immediately, but there is no export/import command or documented round-trip procedure; restore on another machine means reinstalling, copying directories by hand and re-authenticating.P2 Complete export PassOpen InterpreterConfig (TOML), session rollouts (JSONL), state/log/memory SQLite DBs, memories (Markdown), cron jobs, credentials (auth.json by default) and bundled skills all live in the owner's home directory; instructions and skills live in owner files (AGENTS.md, .agents/skills). Nothing that defines the agent is held by the provider, and API-key or local-model integrations re-link without any provider-registered client.P3 Runs without the provider PassOpen InterpreterThe runtime is Apache-2.0 and runs locally with a user-configured provider (Ollama, LM Studio or any OpenAI-compatible endpoint); no licence check, account or Open Interpreter service is needed to start or act. Update checks (GitHub API) and analytics are best-effort calls whose failure does not stop the agent.P4 Model independence PartialOpen InterpreterLocal open-weight models are officially supported (--oss, Ollama, LM Studio, chat-completions wire) and shell, MCP, subagents, hooks and skills run over plain function calling, but several capabilities depend on OpenAI-hosted model features: hosted web search, image generation, realtime voice, and the apply_patch tool (enabled only by model metadata; unknown/non-OpenAI models get none). No formal minimum model contract is published.P5 Exit survives termination PassOpen InterpreterState is local-first and always on the owner's device; nothing Open Interpreter operates holds it, so termination by the provider (or loss of a model-vendor account) cannot remove access to it.T1 Readable without the provider PartialOpen InterpreterState is in open formats (TOML config with a published JSON Schema, plain JSONL rollouts, SQLite, Markdown memories) readable with general tools and no provider service, but the rollout JSONL and SQLite databases have no published schema and are defined only by source code.T2 What runs is what you see PartialOpen InterpreterSettings are inspectable (/debug-config shows effective layered config, /status shows model, provider, sandbox), and rollouts record base instructions and developer messages. On the default OpenAI provider, compaction is done server-side and stored only as encrypted_content, so the compaction summary in context is not visible to the owner (one shortfall). Full request bodies can be captured only via undocumented TRACE logging, and not for the default WebSocket transport (G3 cap).T3 No shadow data FailOpen InterpreterA fresh release build sends usage metrics to OpenAI's Statsig endpoint and, on the preselected ChatGPT sign-in path, analytics events (model, provider, token counts, repo hash, thread and turn ids) to an Open Interpreter server (oi-new-api.fly.dev). Neither the retention period nor what is held is disclosed for the terminal agent, and the owner cannot view it.T4 No hidden steering PartialOpen InterpreterSource shows no paid placement. With the preselected ChatGPT sign-in, the apps, plugins and tool_suggest features are on by default and surface connectors and plugins from OpenAI's curated catalogue (github.com/openai/plugins). These can be switched off, but the docs state apps and plugins are off by default, and there is no binding no-steering commitment.A1 Complete action record FailOpen InterpreterRollout JSONL records messages and tool calls with arguments and outputs, and is not pruned by default. However, hook executions and approval decisions are not persisted (HookStarted/HookCompleted and approval events are dropped by the rollout policy), and a failed rollout write is only logged with tracing::error! while the turn continues, without telling the owner. That is two shortfalls.A2 Tamper-evident, owner-checkable FailOpen InterpreterRollout files carry no hash chain, signature or other integrity data, so edits or deletions leave no trace.A3 Agent cannot rewrite its record PartialOpen InterpreterRollouts are written by the runtime process to ~/.openinterpreter/sessions, which the default workspace-write sandbox leaves read-only to agent commands. The home directory is not explicitly protected, though: when the workspace root contains it (for example, launching from $HOME), sandboxed commands can rewrite the record.V1 Obtainable source, matching build PartialOpen InterpreterAll runtime source is Apache-2.0, and building from source is documented. The default install (a curl-piped script downloading GitHub release archives) checks SHA-256 digests taken from the release metadata only. It has no reproducible builds or third-party-verifiable provenance, cosign signs only individual Linux binaries, and macOS/Windows signing is optional.V2 Owner-rooted identity FailOpen InterpreterThe agent has no identity of its own; it acts through the owner's shell, git and model-provider credentials (by default a ChatGPT account), and offers no owner-rooted agent key or verifiable attribution of its actions.V3 Change history and rollback FailOpen InterpreterThe undo/ghost-commit feature is marked Removed, and there is no change log or restore for config, instructions or permissions. Per-turn TurnContext entries exist only inside raw rollout files, and the memories git baseline is internal, so the product offers no owner-facing view of what changed, by whom, or a rollback.M1 Instructions and logic PartialOpen InterpreterThe base prompt can be replaced (model_instructions_file / instructions), developer instructions added, the local compaction prompt replaced, and bundled skills disabled or shadowed. Injected permission, environment and app blocks can only be switched off, not rewritten, except by supplying a whole model_catalog_json. On the default OpenAI provider, compaction runs server-side and ignores compact_prompt.M2 Memory PartialOpen InterpreterMemories are off by default. When enabled, they are Markdown files (MEMORY.md, memory_summary.md, raw_memories.md, rollout_summaries/) that can be edited or reset from /memories. Stage-1 outputs are kept in memories_1.sqlite with no editing interface, and the per-item /memory-drop and /memory-update commands are stubs.M3 Tools PartialOpen InterpreterOwner tools can be added without review through MCP servers, local plugins and hooks. Most built-ins can be switched off by feature flags (shell_tool, view_image, multi_agent, image_generation, web_search=disabled), but apply_patch has no switch and can be removed only by replacing the whole model catalogue.M4 Owner has the final say PassOpen InterpreterThe owner can disable the sandbox and approvals (danger-full-access, --dangerously-bypass-approvals-and-sandbox), bypass hook trust, and override the only built-in heuristic (forced rm) with an allow rule. No content filter is imposed by the runtime.M5 No unilateral change PassOpen InterpreterThe runtime only notifies of updates, with a dismissible, owner-initiated update, and releases can be pinned via the installer. The model, instructions and provider can be pinned in config, or the owner can use a local model, which avoids the ChatGPT-login remote /models catalogue that can change instruction templates server-side. The optional daemon auto-updates hourly by default, but the daemon is off unless started and the updater can be disabled.C1 Boundaries FailOpen InterpreterFilesystem limits are per-path (permission profiles), MCP tools can be allow/deny-listed, and network is off by default, but per-domain network limits require the Experimental network_proxy feature. More importantly, the project's .openinterpreter/ directory is not among the protected workspace paths, so in the default workspace-write mode the agent can write project config.toml or rules/*.rules (for example an allow rule that bypasses the sandbox), which take effect in a later session. That is two shortfalls.C2 Approval gates PartialOpen InterpreterThe owner can gate shell commands by prefix rules (prompt/forbidden), MCP tools per tool (approval_mode), apply_patch and escalations, and PreToolUse hooks can gate most tools; the model cannot self-approve. However, write_stdin to a running process skips PreToolUse hooks, so one path bypasses hook-based gates.C3 Immediate halt PartialOpen InterpreterEsc/Ctrl+C interrupts the active turn and clears pending approvals. Background terminals need a separate /stop, spawned subagents are interrupted separately, and when connected to the daemon, Ctrl+C offers to keep the task running in the background.C4 Data serves only the owner PartialOpen InterpreterNon-content telemetry is on by default. OTEL metrics go to OpenAI's Statsig (ab.chatgpt.com) in every release build, and analytics events go to oi-new-api.fly.dev on the default ChatGPT sign-in path. Both can be disabled ([analytics] enabled = false, [otel] metrics_exporter), but neither flow is disclosed in the Open Interpreter docs. The website privacy policy describes opt-in telemetry for the desktop app and applies policy changes on notice, not consent. Prompt content is not sent.C5 Credential custody FailOpen InterpreterCredentials are visible (auth.json file by default, MCP OAuth in the keyring), and /logout revokes and deletes them. However, the analytics client attaches the owner's auth headers (ChatGPT access token and account id, or the OpenAI API key) to requests sent to the provider's own analytics server oi-new-api.fly.dev, a use for the provider's purposes that is not disclosed.C6 Full deletion FailOpen InterpreterSessions can be deleted one at a time (interpreter delete, /delete), but no single action deletes the whole agent. Prompt history is not removed by session delete, and there is no owner-initiated deletion of provider-side analytics or metrics copies.
Hermes AgentP1 Owner-usable round trip PassHermes Agent`hermes backup` writes a zip of the whole ~/.hermes home. `hermes import <zip>` restores it on a machine the owner runs, including profile aliases and the gateway service. The FAQ documents moving to a new machine as install, back up, copy and import. Export and restore each need no approval or queue.P2 Complete export PartialHermes AgentThe backup covers config.yaml, .env, auth.json, the credential vault, state.db sessions, memories, skills, cron jobs, MCP tokens and memory-provider files. Portal connectors are enabled by default (`connectors.enabled: True`). Credentials for accounts linked through them are held by Nous and its integration providers, and connector policy is stored at the Portal, so those integrations can be re-linked only through the Portal.P3 Runs without the provider PartialHermes AgentThe runtime is MIT-licensed and the source has no licence check or required Nous account. Cron runs in-process by default, and update checks and the remote model catalog can be switched off. With Nous blocked, Portal connector tools disappear, and public documentation does not give an owner-run replacement for each connector.P4 Model independence PartialHermes AgentOllama, llama.cpp, vLLM and LM Studio are documented, and the minimum model is documented as native tool calling with at least 64k context. Tools run in the runtime. The bundled Google Meet plugin's `meet_say` works only in realtime mode, and that mode is hardcoded to wss://api.openai.com/v1/realtime, so this capability is lost off OpenAI.P5 Exit survives termination PassHermes AgentEverything the P2 export contains lives locally in ~/.hermes on the owner's machine, so a current copy stays with the owner after Portal suspension, a lapsed plan or shutdown. Portal terms allow suspension without notice, but that does not reach local state.T1 Readable without the provider PassHermes AgentState is unencrypted, in documented formats: SQLite state.db (schema documented and in source), Markdown MEMORY.md, USER.md, SOUL.md and SKILL.md files, cron jobs.json and config.yaml. All logic ships as MIT source. Provider-sealed reasoning replay blobs are treated as inference internals, which the criterion excludes.T2 What runs is what you see FailHermes AgentLocally assembled context is visible: `HERMES_DUMP_REQUESTS` writes each request body, system prompts and compaction summaries are stored, and `hermes config show` lists local settings. Two shortfalls. (1) No public evidence settles whether the Portal adds context server-side (G12). (2) Model identity is not fully determinable: the Portal says backend routing for a model 'can change over time', the free list includes an undisclosed 'stealth' model, and the Portal-chosen compaction model appears only in a debug log.T3 No shadow data FailHermes AgentThe Portal privacy policy covers collection of prompts, outputs, conversation history, tool interactions, telemetry and usage logs, and derived datasets, analytics and models. It states no retention periods, and the owner has no item-level view of what is held. The only bound is account deletion within six months of termination.T4 No hidden steering PartialHermes AgentSetup recommends Nous's own Portal, which is disclosed and replaceable through Full or Blank Slate setup and `hermes model`. The Portal's recommended-models feed sets model ordering with undisclosed criteria and cannot be switched off. There is no binding no-paid-placement commitment, and one promotional partnership (Nemotron) is disclosed.A1 Complete action record FailHermes AgentTool calls are stored in state.db with their arguments, subagent and cron runs are recorded, and the owner is warned when the store fails. Two shortfalls. (1) Ended sessions are pruned by default (`sessions.auto_prune: True`, `retention_days: 90`). (2) Successful shell-hook runs and outbound webhook calls leave no record; only failures are logged.A2 Tamper-evident, owner-checkable FailHermes Agentstate.db and the logs carry no hash chain, signature or owner-held checkpoint. The only integrity code is SQLite `PRAGMA integrity_check`, which repairs corruption.A3 Agent cannot rewrite its record FailHermes AgentThe default terminal backend is local, and state.db is not on the protected write-path list. Dangerous-command patterns do not cover deleting state.db or editing it with sqlite3, so nothing in the default configuration stops the agent from altering its record. Docker isolation exists but is not documented as protection for the record.V1 Obtainable source, matching build PartialHermes AgentThe default install is a git clone of the MIT source, with Python dependencies hash-pinned in uv.lock, so the running code can be tied to a commit. Nous Portal inference and the connector service are closed hosted components that are optional but on by default in the recommended setup.V2 Owner-rooted identity PartialHermes AgentThere is no cryptographic agent identity; the A2A design lists DID/Ed25519 identity as out of scope. Owners can set up per-channel identities they control, such as a mailbox on their own domain or an A2A Agent Card on their own host, which Nous cannot revoke. Attribution is per channel only.V3 Change history and rollback PartialHermes AgentSkills have an attributed ledger (curator, agent or user) with per-entry rollback. `/snapshot` covers config, state.db, .env, auth.json and cron, and config backups are kept. Memory, model and runtime-version changes have no attributed change log, memory restore means importing a full backup, and the Portal-held connector state is not covered.M1 Instructions and logic PartialHermes AgentSOUL.md replaces the default identity, most guidance blocks have config switches, and edited or deleted bundled skills are preserved across `hermes update`. Some system-prompt segments can be changed only by editing source: the Hermes help guidance, the steer-channel note and the runtime/environment block. Platform hints can be replaced but not emptied.M2 Memory PartialHermes AgentMEMORY.md and USER.md are plain files, editable per entry, and agent-authored skills are editable files. The session store that `session_search` recalls across sessions can only be deleted a whole session at a time; items cannot be edited and single messages cannot be deleted. External memory providers are off by default.M3 Tools PassHermes AgentOwners can connect any MCP server (`hermes mcp add`) or local plugin without review or signing. Every built-in tool sits in a toolset that `agent.disabled_toolsets` removes, and that setting is applied last. Only the separate plugin catalog is curated.M4 Owner has the final say PartialHermes AgentApprovals, secret redaction, the private-IP SSRF block, Tirith scanning and protected instruction files can be switched off. Several protections have no owner switch in the local backend: the hardline command blocklist ('no override flag'), the file write/read deny lists, the cloud-metadata SSRF floor, prompt-injection scanning of memory and cron prompts, and the Skills Guard block on 'dangerous' verdicts.M5 No unilateral change PartialHermes AgentRuntime updates are applied only by explicit `hermes update`, and owners can pin a concrete model and auxiliary models, disable the remote model catalog, or use a local model. A live plugin kill list fetched from Nous can stop an already-installed catalog plugin from loading, and there is no switch to decline this in advance; the owner can re-enable it only afterwards with a forced reinstall.C1 Boundaries FailHermes AgentGateway access is deny-by-default with per-user allowlists and pairing, and file tools block writes to config.yaml and .env. Two shortfalls. (1) Outbound contact and network scope are only on/off per toolset or a web denylist, with no per-recipient or per-domain allowlist in the default backend. (2) The agent can change its own approval mode and allowlists through the terminal with `hermes config set`, which matches no gated pattern, and the documentation says the terminal can overwrite denied paths. Spend is treated as met under G11: there is no built-in payment tool, card fills need confirmation, and Portal top-up is human-only.C2 Approval gates PartialHermes AgentOwners can set manual approval for dangerous commands, `approvals.deny` floors, untrusted-MCP gating and deny-by-default cron and subagent modes. In the default 'smart' mode an auxiliary model decides whether a flagged command is approved. Gating an arbitrary tool requires writing a `pre_tool_call` hook script, since config has no require-approval list.C3 Immediate halt PartialHermes AgentCtrl+C and gateway `/stop` interrupt the foreground turn and background processes. `hermes pause` writes an ESTOP file that blocks new cron, kanban and gateway work, but in-flight work is never killed, and stopping cron, subagents and the gateway takes separate steps.C4 Data serves only the owner FailHermes AgentOn the recommended Nous Portal path, the terms license Service Data for training and product development. Inference payloads are stored and used for training unless the owner enables Privacy Mode, which is opt-out and applies only going forward. The privacy policy can change by posting. Hermes telemetry to Nous is off by default, but the FAQ still says Hermes collects no telemetry.C5 Credential custody PartialHermes Agent`hermes auth list/remove`, `hermes vault`, `hermes mcp remove` and `hermes pairing revoke` each show and revoke their own credential class, with no single list. `hermes auth logout` only deletes local tokens and does not revoke them at the provider. Connector credentials held by Nous have no documented owner view or guarantee of deletion on revocation.C6 Full deletion FailHermes Agent`hermes uninstall --full` removes local data, but Portal data needs a separate account termination. Nous deletes the account within six months of termination, or one month on request, and gives no documented confirmation. The privacy policy says prior use may already be 'incorporated into models', and derived datasets are retained.
OpenAI CodexP1 Owner-usable round trip PartialOpenAI CodexAll agent state is always on disk under the documented CODEX_HOME (~/.codex), and the docs describe pointing Codex at another home directory, so export is immediate. Restoring elsewhere means installing Codex, copying the directory by hand and signing in again; there is no single import action.P2 Complete export PartialOpenAI CodexCODEX_HOME holds config.toml, AGENTS.md, skills, session transcripts (JSONL, optionally zstd-compressed), memories, the state DB and local credentials. ChatGPT Apps connectors enabled by default (feature `apps`) are linked through OpenAI's hosted service, so those integrations can only be re-linked through a provider-registered client.P3 Runs without the provider PassOpenAI CodexThe runtime is Apache-2.0, and OSS providers (Ollama, LM Studio) are defined with requires_openai_auth: false, so the login screen is skipped and no OpenAI account, licence key or server is needed to start or act. The update check goes to GitHub or Homebrew and can be switched off (check_for_update_on_startup). Hosted-only tool loss is graded in P4.P4 Model independence PartialOpenAI CodexLocal open-weight models are supported through the documented `--oss` / `--local-provider` flags and custom providers. Some capabilities work only with OpenAI's hosted models: web search (supports_standalone_web_search is false for OSS providers), image generation, realtime voice, and remote compaction (only enabled for the OpenAI provider).P5 Exit survives termination PassOpenAI CodexAgent state lives locally under CODEX_HOME by default, so a current copy stays with the owner through account suspension, a lapsed plan or provider shutdown.T1 Readable without the provider PartialOpenAI CodexTranscripts, config, memories and instructions are JSONL, TOML or Markdown, with types in open source. With the default OpenAI provider, compaction output (ContextCompaction) and reasoning items are stored as `encrypted_content` blobs that only OpenAI can decrypt.T2 What runs is what you see PartialOpenAI CodexBase instructions, developer messages and settings are recorded in the session transcript (SessionMeta.base_instructions, ThreadSettingsApplied), and /status and /debug-config show the effective configuration. With remote compaction the compaction summary placed in context is an encrypted blob the owner cannot read (one shortfall). Server-delivered catalogue settings are cached locally but not shown in the UI, which is recorded here but not counted.T3 No shadow data FailOpenAI CodexRequests use store:false, but analytics events go to OpenAI by default, and the Privacy Policy keeps Personal Data 'as long as we need… or for other legitimate business purposes' with further retention after deletion for safety, legal or fraud reasons. Retention is therefore not bounded, and held items are not visible to the owner one by one.T4 No hidden steering PartialOpenAI CodexTool suggestions (feature `tool_suggest`) and the remote plugin catalogue (`remote_plugin`) are on by default, surface OpenAI-catalogued plugins and connectors, and can be switched off in config. No evidence of paid placement was found, but no binding commitment rules it out.A1 Complete action record FailOpenAI CodexTranscripts record tool calls with arguments and outputs, but HookStarted and HookCompleted events are classed as non-durable and not persisted (hook entry path unrecorded). When a transcript write fails, persist_rollout_items only writes an error to the tracing log while the session continues, so the owner is not told (silent recording failure).A2 Tamper-evident, owner-checkable FailOpenAI CodexSession transcripts are plain JSONL or zstd files with no hash chain, signatures or external checkpoints, so edits or deletions leave no trace.A3 Agent cannot rewrite its record PartialOpenAI CodexTranscripts are written by the Codex process to ~/.codex/sessions, which is outside the default workspace-write roots, and `.codex` stays read-only inside writable roots. MCP servers are launched with `sandbox: None`, so a tool path outside the sandbox could still alter the record.V1 Obtainable source, matching build PartialOpenAI CodexAll runtime source is Apache-2.0. The release workflow signs Linux artifacts with Sigstore cosign (keyless, GitHub OIDC), but no documentation was found telling owners how to check the default npm or Homebrew binary against the source, and macOS/Windows builds rely on platform code signing.V2 Owner-rooted identity FailOpenAI CodexOn the default path the agent acts under the owner's ChatGPT account. The agent-identity module registers agent keys with OpenAI's backend and trusts JWTs issued by chatgpt.com/codex-backend/agent-identity, so the identity is provider-issued and revocable.V3 Change history and rollback PartialOpenAI CodexPer-turn settings changes are recorded in transcripts (ThreadSettingsApplied), conversations can be rolled back or forked, and the memory workspace keeps a git baseline with a diff file. There is no attributed history or restore for config.toml, AGENTS.md, skills or permissions.M1 Instructions and logic PartialOpenAI CodexThe base prompt can be replaced with model_instructions_file, developer instructions and the compaction prompt are configurable, and several injected blocks can be toggled. Other behaviour-defining templates (memory pipeline prompts, review and guardian prompts, tool descriptions) are compiled into the binary and can only be changed by editing source (G3).M2 Memory PartialOpenAI CodexMemories are off by default. When enabled, consolidated memories are editable Markdown under ~/.codex/memories, but per-thread stage-1 raw memories that feed consolidation are kept in the SQLite state DB with no itemised view or editor in the normal interface.M3 Tools PartialOpenAI CodexOwners can add any MCP server or tool without review and scope tools with enabled_tools/disabled_tools. Web search, update_plan, shell and multi-agent can be switched off, but no documented setting was found to remove apply_patch or view_image.M4 Owner has the final say PassOpenAI CodexThe owner can switch off the sandbox (danger-full-access), approvals (never), analytics, feedback and history persistence through documented config or flags. As the owner is also the administrator, requirements.toml constraints are owner-editable.M5 No unilateral change PartialOpenAI CodexWith no model pinned, the default model and per-model instruction templates, tool mode and guardian policy come from the backend /models catalogue, which is refreshed by ETag. Owners can pin these with `model`, model_catalog_json and model_instructions_file, and runtime updates are opt-in prompts. The hosted model behind a slug cannot be pinned: the server can reroute requests to another model (a warning is shown when this happens), and retirements for ChatGPT sign-in are announced with no stated notice period.C1 Boundaries FailOpenAI CodexFile scope (writable roots, protected paths) and per-tool scope are supported. Network access in the supported configuration is on/off only; per-domain allow/deny needs the network proxy, which is an Experimental feature (G10). MCP servers run outside the sandbox (`sandbox: None`), so file and network limits do not bind that path. Spend is treated as met under G11 (no payment capability).C2 Approval gates PartialOpenAI CodexOwners can gate commands (execpolicy prefix rules with `prompt`), MCP and app tools per tool (approval_mode), and all escalations (approval_policy), with approvals going to the user by default. Hosted web search, on by default in cached mode, can be switched off but not gated per call.C3 Immediate halt PartialOpenAI CodexEsc/Ctrl+C interrupts the foreground turn, but background terminals need a separate `/stop` ('stop all background terminals'), and the local background daemon is managed separately (/daemon).C4 Data serves only the owner FailOpenAI CodexThe consumer Terms and Privacy Policy allow OpenAI to use Content to train models unless the user opts out, and Codex docs say chat content 'may be used to improve OpenAI models if allowed by your ChatGPT data controls'. Analytics events go to OpenAI by default ([analytics] enabled defaults to true), and the Terms let non-material changes take effect on posting.C5 Credential custody PartialOpenAI CodexLocal credentials are in auth.json or the OS keyring, disclosed in the auth docs and revocable with `codex logout` and per-server MCP logout. Grants for ChatGPT Apps connectors are held by OpenAI, and the public evidence I could open does not establish per-item visibility, immediate revocation or deletion of provider copies.C6 Full deletion FailOpenAI CodexThere is no single action that deletes the whole agent: local state must be removed by hand and the ChatGPT account deleted separately. The Privacy Policy removes deleted data within 30 days 'unless we need to retain it for longer' for legal, fraud, security or financial reasons, so provider-side copies may survive.
Gemini CLIP1 Owner-usable round trip FailGemini CLIGemini CLI ships no export/import for the agent and no documented procedure for moving it; the only export command, /chat share, exports a single conversation as Markdown/JSON. Agent state sits in local directories (~/.gemini, project .gemini) documented piecemeal, so moving it is an undocumented manual copy that also needs auth re-setup.P2 Complete export PartialGemini CLIAll agent-defining state is local: settings, GEMINI.md memory, skills, policies, extensions and JSONL session records. Nothing agent-defining is held server-side. Re-linking the default model integration ('Sign in with Google') needs Google's own registered OAuth client, and credentials default to the OS keychain rather than the ~/.gemini directory.P3 Runs without the provider PartialGemini CLIThe runtime is Apache-2.0 and the source contains no licence check or expiry. Experiment flags are fetched only when a Code Assist server is in use, and usage-statistics failures are non-fatal. Whether every capability survives with all Google domains blocked has no public documentation (hands-on test), so the pass is withheld under G12.P4 Model independence PartialGemini CLIThe documented model choices are Gemini models only (Auto, Pro, Flash). Other or local models can be reached only by pointing the documented GOOGLE_GEMINI_BASE_URL override (localhost allowed) at a third-party Gemini-API-compatible proxy, which is a workaround. The local Gemma support is an experimental routing classifier, not the agent model. The built-in Google Search grounding tool depends on a Gemini hosted-model feature.P5 Exit survives termination PassGemini CLIAgent state (config, memory, skills, session records) is stored on the owner's machine by default, so a current copy stays in the owner's hands if the Google account or licence is terminated or the service shuts down.T1 Readable without the provider PassGemini CLIState is stored in open formats: JSON settings with a published schema, Markdown memory and skills, TOML policies and JSONL session records defined in the Apache-2.0 source. Stored credentials are encrypted with a locally derived key or held in the OS keychain, so no provider-held key is involved.T2 What runs is what you see PartialGemini CLIThe default system prompt can be written to a file (GEMINI_WRITE_SYSTEM_MD), and prompts can be logged locally through OpenTelemetry file output. However, provider-controlled behaviour switches fetched from the Code Assist server (compression threshold, routing classifier threshold, which model 'auto'/Flash resolves to) are not surfaced to the owner through any documented view (one shortfall). /chat debug request export is documented for nightly builds only.T3 No shadow data FailGemini CLIBy default Google receives usage statistics and Code Assist request and interaction data. The Code Assist data-governance page and the Google Cloud terms opened for this review give no bounded retention table for these logs. Usage statistics fall under Google's general Privacy Policy, and the owner cannot view what is held item by item.T4 No hidden steering PartialGemini CLIThe open-source client contains no paid placement. Its first-party defaults (Gemini models, Google Search grounding) are disclosed and can be replaced or excluded. The server can push banner text and routing/model flags, the hosted Code Assist layer cannot be inspected, and Google has made no binding no-steering commitment.A1 Complete action record FailGemini CLITool calls from the main agent and subagents are recorded in session JSONL files. Those records are pruned after 30 days by default. If the disk fills (ENOSPC), recording stops and the only warning goes to the debug logger, so the session continues unrecorded without the owner being told (two shortfalls).A2 Tamper-evident, owner-checkable FailGemini CLISession records are plain JSONL files with no hashing, signing or other integrity data, so they can be edited or deleted without trace.A3 Agent cannot rewrite its record FailGemini CLIThe session record lives in the project temp directory (~/.gemini/tmp/<hash>/chats). isPathAllowed explicitly lets the agent's file tools write to that directory, and the shell tool is unsandboxed by default. The only barrier is per-call approval in the default approval mode.V1 Obtainable source, matching build PartialGemini CLIAll client source is Apache-2.0. The default npm release is an esbuild bundle published without npm provenance attestations (the registry metadata for 0.62.0 has none), so the distributed build cannot be tied to source independently. Building from source is supported only as an alternative install path.V2 Owner-rooted identity FailGemini CLIThe agent has no identity of its own. It acts under the owner's Google account or API key, which are provider-issued credentials Google can revoke.V3 Change history and rollback PartialGemini CLI/rewind shows and reverts file changes made by the agent's edit tools, including memory-file edits, attributed to the agent within retained sessions. Auto Memory (off by default) proposes diffs for review. Nothing records or restores changes to settings, tools, permissions, model or runtime version, or changes made by the owner or updates.M1 Instructions and logic PartialGemini CLIGEMINI_SYSTEM_MD fully replaces the base system prompt, and built-in subagents can be disabled or overridden. Other behaviour-defining prompts (chat compression, routing classifier, loop detection and similar internal injections) are hard-coded and can be changed only by editing source.M2 Memory PassGemini CLIPersistent memory is kept in Markdown files (repository GEMINI.md, a private per-project memory folder, global ~/.gemini/GEMINI.md) that the owner can view, edit and delete item by item. Auto Memory candidates are held in a review inbox until approved.M3 Tools PassGemini CLIOwners can add any tool without Google review through MCP servers, extensions and a tool discovery command. Any built-in can be removed with tools.core/tools.exclude or policy-engine deny rules.M4 Owner has the final say PartialGemini CLIMost protections are owner-configurable: sandboxing, folder trust, loop detection, env-var redaction, default policies (user tier outranks default tier) and YOLO mode. Some are locked in code, such as the rule that a non-localhost base URL must use HTTPS and the hard-coded denial of agent access to the memory inbox.M5 No unilateral change FailGemini CLIOn the default Sign in with Google path, the CLI fetches server-side experiment flags that change the compression threshold, routing and which model the Flash/Auto selections resolve to, without notice or consent. Automatic updates are on by default. Owners can pin by disabling auto-update or supplying GEMINI_EXP, but server flags apply silently.C1 Boundaries PartialGemini CLIThe policy engine gives per-tool and per-argument allow/deny rules, file tools are limited to the workspace plus added directories, and the documented sandbox proxy can filter egress per domain. The CLI has no payment capability, so spend counts as met (G11). The agent's file tools can edit the workspace .gemini/settings.json and policies, which take precedence over user settings. Only per-call approval, which the owner can relax, stands in the way (one shortfall).C2 Approval gates PassGemini CLIOwners choose which tools, commands or subagents need approval using policy-engine ask_user rules and tools.confirmationRequired. Subagents are matched by the same engine.C3 Immediate halt PartialGemini CLICtrl+C cancels the current request or quits the CLI. Commands started in the background keep running after the tool returns (documented), so stopping them takes separate steps. No provider-side schedulers exist on this path.C4 Data serves only the owner FailGemini CLIUsage statistics are on by default (opt-out). The Clearcut logger attaches the cached Google account email, or else an installation ID, even though the docs say no email is collected. Code Assist interaction telemetry (conversation-offered and tool-call interaction records) is sent with every response and is not gated by the usage-statistics setting. Google Cloud terms changes take effect on 30 days' notice without specific consent.C5 Credential custody PartialGemini CLICredentials (Google OAuth, MCP OAuth tokens, API keys) are held locally in the OS keychain or files and can be cleared (/auth signout). No single documented view lists every credential and grant, and per-MCP-server token revocation is not documented beyond the token file.C6 Full deletion FailGemini CLINo single action deletes the agent. The uninstall docs remove only the package, leaving ~/.gemini, and there is no documented owner-initiated deletion with a stated completion limit for Google-side usage statistics and Code Assist logs.
ClineP1 Owner-usable round trip PartialClineAll agent state is in local files whose layout is documented (~/.cline/, ~/Documents/Cline, project .cline/). Export is therefore available immediately, but there is no export/import action: the in-app task export produces Markdown only, and restoring means copying folders by hand.P2 Complete export PassClineThe following all live locally under ~/.cline/data and ~/.cline in JSON, Markdown or SQLite: configuration (globalState.json, providers.json), credentials (secrets.json), rules, skills, workflows, hooks, MCP settings and session history. Checkpoints are git refs in the workspace repository. The Cline account holds only billing and credits, not agent-defining state. Per-workspace state is keyed by a hash of the absolute workspace path, so it re-attaches only at the same path; this is recorded, not counted. MCP OAuth supports owner-supplied or dynamically registered clients.P3 Runs without the provider PartialClineThe extension is Apache-2.0 and runs with BYOK or local models without a Cline account. Feature flags fall back to coded defaults. However, the ToS define the 'Service' to include 'Cline Extensions' and grant only a 'freely revocable' licence that Cline 'may terminate … at any time, for any reason or no reason'. No technical path enforces that termination.P4 Model independence PartialClineThe owner can choose from 200+ providers, including Ollama and LM Studio, with runtime-executed tools. The web_search and image_generation tools are added only for models that natively support them (supportsModelTool), so they are lost on other models, including local ones.P5 Exit survives termination PassClineAgent state is written to the owner's machine by default (~/.cline/data), so a current copy is always in the owner's hands whatever happens to the Cline account or company.T1 Readable without the provider PassClineState is plain JSON (settings, secrets, session messages including the system prompt), SQLite and Markdown/script files, with no encryption. The layout is publicly documented, and the Apache-2.0 source defines every schema.T2 What runs is what you see FailClineTwo shortfalls. (1) Material settings cannot be determined: the default free-model list includes 'Anonymous' stealth models such as stealth/pixel-canary, and the PostHog flag values in effect are not shown to the owner. (2) Public evidence does not settle whether the hosted Cline gateway adds or alters context, so that class counts as not visible (G12). The system prompt and compaction state are saved only in on-disk session JSON with no documented viewer, which caps this test at partial under G3.T3 No shadow data FailClineOn the default Cline-account path, Cline collects account data, request data sent through Cline keys, automatically collected usage data and 'Inferences' about the user. The privacy policy states no retention periods (only 12 months for cookie consent), and none of this is visible item by item.T4 No hidden steering PartialClineOnboarding preselects Cline's own provider. This is visible and replaceable with BYOK. The recommended and free model lists (which include stealth models) come from a Cline server feed with undisclosed selection criteria, and MCP marketplace ranking is undisclosed. No paid placement was found, but no binding no-steering commitment exists.A1 Complete action record PartialClineSession history keeps every tool call with its arguments and results until the owner deletes it. One shortfall: message persistence is fire-and-forget, so a failed write is only logged and reported to telemetry, and the owner is not told.A2 Tamper-evident, owner-checkable FailClineSession records are plain JSON and SQLite files with no hashes, signatures or external anchor, so any edit or deletion leaves no trace.A3 Agent cannot rewrite its record FailClineThe record lives in ~/.cline/data, which the agent's editor and shell tools can reach. The default auto-approval settings enable editFiles and editFilesExternally, and approval checks do not treat paths outside the workspace separately. Nothing stops the agent from altering or deleting its record.V1 Obtainable source, matching build PartialClineThe extension source is Apache-2.0. The default Marketplace VSIX is built in CI with no reproducible-build or third-party-verifiable provenance, and the on-by-default Cline gateway (api.cline.bot) is closed but optional.V2 Owner-rooted identity FailClineThe agent has no identity of its own. Its only account identity is a Cline account, which Cline can suspend or terminate.V3 Change history and rollback FailClineCheckpoints snapshot the owner's workspace files and conversation, not agent state. There is no history, attribution or rollback for changes to settings, model, provider, tools, permissions, MCP configuration or global rules.M1 Instructions and logic PartialClineRules, skills, workflows and hooks are owner-editable files that persist. The extension has no setting to replace the base system prompt; only the CLI documents -s/--system.M2 Memory PassClineCross-session memory is owner-authored files (rules, memory-bank Markdown) and task history. Each is itemised and can be edited or deleted. The extension has no hidden automatic memory or profile store.M3 Tools PartialClineThe owner can connect any MCP server without review. Built-in bash, editor and file tools cannot be removed in the extension; only the browser and web_search can be switched off. Plugin custom tools are 'not applicable on VSCode'.M4 Owner has the final say PassClineThe owner can change or disable every protection in scope. Approvals can be turned off one by one or all at once with YOLO mode, and checkpoints and telemetry can be switched off. The plan-mode command blacklist applies only in plan mode, which the owner can leave. No content filter is imposed.M5 No unilateral change FailClinePostHog feature flags are evaluated 'regardless of user's telemetry preferences'. They control Cline model-list sourcing, recommended and onboarding models, the OpenAI Responses websocket transport, and banners, and the owner has no way to pin or decline them. The ToS let Cline 'change the Service … with or without prior notice'.C1 Boundaries FailClineThree shortfalls. File scope is only on/off (workspace vs all files), and MCP approval is one global toggle. There is no per-domain or per-recipient contact or egress limit for web fetch or the browser. Under the default auto-approval (editFiles and editFilesExternally on), the agent can edit its own settings and hooks in ~/.cline or .clinerules without asking. Command allow/deny exists only through the CLINE_COMMAND_PERMISSIONS environment variable. Spend is treated as met because the agent has no payment capability (G11).C2 Approval gates PartialClineThe owner can gate only a fixed set of classes: read, edit, commands, browser and all-MCP. Whether a command is 'safe' is decided by the model's requires_approval flag.C3 Immediate halt PartialClineCancel stops the foreground turn. The extension's cancelBackgroundCommand handler is a stub that does nothing, so commands detached with 'Proceed While Running' have no working stop. The extension runs no provider-side work.C4 Data serves only the owner FailClineThe preselected 'Absolutely Free' path uses free models whose 'usage may be used to help improve model performance and quality', so content goes to secondary use by default. Telemetry is on by default (setting 'unset' is treated as opted in). The privacy policy shares identifiers with ad partners, and data-term changes take effect on publication without consent.C5 Credential custody PartialClineCredentials are in local owner-only files (secrets.json, providers.json). Provider API keys are shown per provider in settings and can be removed. MCP OAuth tokens and the Cline session token are not listed in one owner-visible place. BYOK keys are not sent to Cline, per the privacy policy.C6 Full deletion FailClineNo documented one-step deletion covers the Cline account and every provider-side copy. The only mention is a DELETE /api/v1/users/me endpoint in the enterprise API reference, with no stated time limit or confirmation. The ToS keep telemetry and a perpetual licence to de-identified data after the account ends.
GitHub CopilotP1 Owner-usable round trip PartialGitHub CopilotThe CLI keeps configuration, instructions, skills, agents, hooks and session history as files under ~/.copilot (relocatable with COPILOT_HOME), so a copy is available at once and runs on the owner's own machine. There is no export/import command or documented migration procedure: restoring means copying a hidden directory (and repository .github files), re-authenticating, and optionally running /chronicle reindex, which is the 'restore is not a single documented action' shortfall.P2 Complete export FailGitHub CopilotCopilot Memory (on by default for paid individual plans) is held only on GitHub; the docs offer export of preferences only to Business/Enterprise administrators, not to individuals. Copilot cloud agent session logs and cloud automations (schedules and triggers) also live only on GitHub, and the CLI session-sync copy is separate. That leaves two or more classes of agent state with the provider.P3 Runs without the provider PartialGitHub CopilotWith BYOK and COPILOT_OFFLINE=true, the CLI is documented to run without a GitHub account and without contacting GitHub servers. GitHub-hosted services are lost (/delegate, GitHub MCP server, code search, Copilot Memory, session sync). The proprietary CLI licence is not stated to be perpetual or irrevocable: it terminates on non-compliance and requires the user to cease use, and the README still lists an active Copilot subscription as a prerequisite. No technical path is documented that enforces termination offline.P4 Model independence PartialGitHub CopilotLocal BYOK is documented for OpenAI-compatible endpoints including Ollama, vLLM and Foundry Local, with a published minimum (tool calling and streaming). The docs do not state that every capability works on BYOK models. Background features run on GitHub-hosted 'utility models' that cannot be selected, and Auto model selection works only with hosted models, so parity off GitHub models is not shown.P5 Exit survives termination PassGitHub CopilotThe P2 export route is the local ~/.copilot directory, which the CLI keeps on the owner's machine continuously, so a current copy stays in the owner's hands if the GitHub account is suspended or closed. Provider-held state that is not in that export (Memory, cloud sessions) is graded in P2.T1 Readable without the provider PartialGitHub CopilotLocal state is in open formats: JSON settings, Markdown instructions and skills, per-session events.jsonl, checkpoints and a SQLite session store, and the SDK docs publish session event schemas. Some components are opaque or undocumented: config.json and session-store.db are described only as internal, and Copilot Memory can be read only through GitHub's web settings.T2 What runs is what you see PartialGitHub CopilotOpenTelemetry file export with content capture records the system prompt, the full input messages, the tool definitions, tool arguments and results, and both the requested and resolved model for every call, all set up through documented environment variables. /env and /context list the loaded instructions, tools and MCP servers. Requests to GitHub-hosted models go through GitHub's Copilot API and content filters, and the docs do not say whether that hosted layer adds context. The owner cannot confirm that what the client captures is exactly what the model receives (one shortfall).T3 No shadow data FailGitHub CopilotGitHub holds synced CLI sessions (on by default), Copilot Memory (deleted after 28 days unused) and cloud agent logs, which the owner can list. Under ToS J.3, for Pro plans it also collects inputs and outputs for model training unless the owner opts out. Those copies and any derived data cannot be seen item by item, and the Privacy Statement gives no bounded period: data is kept 'as long as your account is active and as needed'.T4 No hidden steering PartialGitHub CopilotNo evidence was found of paid placement. Provider defaults are disclosed and can be switched off: the GitHub MCP server can be disabled, the model can be pinned instead of Auto, and evaluation models served by Auto can be opted out. However, no binding term rules out self-preferencing influence. Auto routing weighs cost and system health set by GitHub, and GitHub marketplaces and connectors are on by default.A1 Complete action record FailGitHub CopilotEach local session writes an event log (events.jsonl) with tool calls and arguments, and cloud agent sessions keep logs on GitHub. No mechanism is documented that alerts the owner when recording fails. Commands run by hooks are not documented as recorded in the session log. Under G12 both count as shortfalls.A2 Tamper-evident, owner-checkable FailGitHub CopilotNo integrity protection (hash chains, signatures or checkpoints) is documented for events.jsonl, the session store or cloud session logs, so the record can be edited or deleted without a trace.A3 Agent cannot rewrite its record PartialGitHub CopilotBy default the agent's shell and file tools can reach ~/.copilot/session-state, and the docs say trusted-directory scoping is heuristic. Local sandboxing (off by default, public preview) is deny-by-default and would exclude paths outside the workspace. However, the built-in edit tools enforce it only in software, the model can request a bypass through an owner prompt, and the docs do not name the record as protected.V1 Obtainable source, matching build FailGitHub CopilotThe Copilot CLI is closed source. Its licence forbids modification and derivative works, and the public repository holds only a README, licence, changelog and install script. The earlier VS Code Copilot Chat source repository is archived.V2 Owner-rooted identity FailGitHub CopilotThe agent has no identity of its own. It acts through the owner's GitHub account and OAuth token (or a provider-issued GitHub App identity for cloud agent), which GitHub can revoke. Attribution is a Co-authored-by trailer.V3 Change history and rollback PartialGitHub CopilotWithin a session the CLI tracks file changes per prompt and can rewind the conversation and files. Changes to settings, instructions and permissions are not versioned or attributed, and Copilot Memory (cloud-held by default) has no change history or restore.M1 Instructions and logic PartialGitHub CopilotOwner layers can be edited and they persist: personal and repository instructions, custom agents, skills, hooks, and switches for built-in subagents and skills. The built-in base system prompt is shipped inside the closed CLI binary, and no documented CLI setting replaces it.M2 Memory PartialGitHub CopilotCopilot Memory repository facts and user preferences can be viewed as lists and deleted in GitHub settings, but items cannot be edited. Local instruction files are fully editable.M3 Tools PassGitHub CopilotThe owner can add any MCP server, extension, plugin or skill without GitHub review (marketplaces are optional). Any tool can be denied with --deny-tool or permission rules, and built-in MCP servers, skills and subagents (including rubber-duck via builtInAgents.rubberDuck) can be disabled.M4 Owner has the final say PartialGitHub CopilotMost protections are owner-configurable: sandboxing, trusted paths, permission prompts (--allow-all), plan mode and public-code filtering. Some are locked: the documented environment-variable denylist and the web_fetch SSRF blocklist for private ranges have no documented override (only localhost can be allowed).M5 No unilateral change PassGitHub CopilotThe owner can pin everything. autoUpdate can be set to false, a specific CLI version can be installed, a local BYOK model with COPILOT_OFFLINE fixes the model and blocks GitHub contact, and instructions are local files. A public changelog is published before the owner chooses to upgrade. On the default hosted path, GitHub retires models on its own schedule and preview features change without notice; this is recorded, but does not count on the achievable basis.C1 Boundaries PartialGitHub CopilotPer-tool allow/deny patterns, URL allow/deny lists, sandbox path and host rules and MCP allowlists give fine-grained scope. Spend is treated as met under G11 because the CLI has no payment capability. The docs call permission scoping 'heuristic', and without the non-default sandbox nothing documented stops the agent from editing its own settings.json or permissions-config.json with its file or shell tools.C2 Approval gates PartialGitHub CopilotBy default the CLI prompts before tools that modify files, execute commands or access URLs, and the owner can pre-approve or deny per tool pattern. Read-only commands are auto-approved by the CLI's own classification. Gating other classes requires writing a preToolUse hook script that returns 'ask', which needs programming (G3).C3 Immediate halt PartialGitHub CopilotCtrl+C immediately cancels the active operation and clears queued prompts, and a double Esc also stops background agents. No single documented control also stops scheduled /every prompts, running cloud agent sessions and cloud automations, which need separate steps on GitHub.C4 Data serves only the owner FailGitHub CopilotSince 24 April 2026, GitHub may use Pro users' Copilot inputs, outputs and code snippets to train AI models unless the owner opts out (ToS J.3). Telemetry is sent unless offline mode is used, sessions sync to GitHub by default, and data-term changes take effect after 30 days' notice without specific consent.C5 Credential custody PartialGitHub CopilotGitHub OAuth grants can be revoked in GitHub settings or with /logout, and MCP OAuth tokens and secrets live in the OS keychain or ~/.copilot. No single view lists every credential and grant the agent holds, including the GitHub CLI token fallback, git and gh credentials injected into the sandbox by default, MCP tokens and cloud agent secrets. No binding term limits GitHub's use of owner credentials to owner-directed actions.C6 Full deletion FailGitHub CopilotRemoving everything takes several steps: deleting local sessions and ~/.copilot, synced sessions, and Memory items, then account deletion. After account deletion GitHub deletes data within 90 days, but 'some information may remain in encrypted backups', and training copies made under ToS J.3 are not covered.
Claude CodeP1 Owner-usable round trip FailClaude CodeClaude Code documents no whole-agent export or matching import. /export writes only the current conversation as plain text. Agent state is spread across ~/.claude, ~/.claude.json and project .claude/ files, and no procedure is documented for moving it to another machine.P2 Complete export FailClaude CodeThe only export is conversation text. Several classes of state are held by the provider on claude.ai and are not in any export: claude.ai connectors, synced skills and plugins, and cloud sessions/routines. Anthropic-hosted connectors (e.g. Gmail, Google Calendar, Microsoft 365) can only be linked through the redirect URL registered by claude.ai.P3 Runs without the provider FailClaude CodeThe runtime is a closed binary. Its right to use comes from Anthropic's Consumer Terms, under which Anthropic 'may suspend or terminate your access to the Services ... at any time without notice' and may revise the terms at its discretion. No perpetual or irrevocable right to run is granted. Anthropic also 'doesn't support routing Claude Code to non-Claude models through any gateway'.P4 Model independence PartialClaude CodeANTHROPIC_BASE_URL, gateway model discovery and CLAUDE_CODE_MAX_CONTEXT_TOKENS are documented and let an Anthropic-format endpoint serve other model IDs. However, Anthropic states it doesn't support routing Claude Code to non-Claude models, so local open-weight models work only through an unofficial workaround. Features tied to Anthropic API betas degrade on non-first-party hosts, e.g. MCP tool search is disabled by default.P5 Exit survives termination PassClaude CodeIn the default local CLI, configuration, CLAUDE.md, auto memory, skills and transcripts are plain files on the owner's machine, so they stay available after account termination. Skills synced from claude.ai stay on disk once downloaded. Provider-held classes and 30-day transcript pruning are graded in P2 and A1.T1 Readable without the provider PartialClaude CodeState is stored as plaintext JSONL, JSON and Markdown with no encryption. However, the docs say the transcript entry format 'is internal to Claude Code and changes between versions', and the provider-supplied base prompt and built-in logic ship inside a closed binary rather than as documented readable files.T2 What runs is what you see PartialClaude Code/context breaks down the context by category, and OTEL_LOG_RAW_API_BODIES can log the full request JSON (including the system prompt) to an owner-run collector. /status and /model show the model in use. One shortfall: behaviour is also shaped by GrowthBook feature flags fetched from Anthropic (they can change the starting permission mode and turn features or 'server configuration' experiments on), and the owner has no documented way to see which flag values are in effect.T3 No shadow data FailClaude CodeFor consumer plans, Anthropic keeps sessions for 30 days, or up to 5 years if training is allowed. Flagged sessions are kept for up to 2 years, and trust-and-safety classification scores for up to 7 years. De-identified data may be kept 'longer' for research, and data may also be kept 'as necessary to combat violations'. Classification scores are derived data that the owner cannot see, and some retention periods are open-ended.T4 No hidden steering PartialClaude CodeThe provider-preferred defaults found are disclosed and can be switched off: the auto-added 'claude.ai Claude Docs' connector (deniedMcpServers), claude.ai-synced Anthropic skills (syncClaudeAiSkills), official marketplace auto-install (env var) and Claude Code commit/PR attribution lines (attribution setting). Because the runtime is closed, the absence of other self-preferencing rests on no binding commitment.A1 Complete action record FailClaude CodeSession transcripts record every tool call with its arguments and results, but two shortfalls apply. (1) Transcripts are deleted after cleanupPeriodDays, which defaults to 30 days. (2) Hook commands are an entry path whose own actions are not written to the transcript: stdout and stderr go only to the debug log. The transcript is also written asynchronously, and no alert on recording failure is documented.A2 Tamper-evident, owner-checkable FailClaude CodeNo integrity data (hash chaining, signatures or checkpoints) is documented for transcripts or logs. They are plain JSONL files that can be changed without trace.A3 Agent cannot rewrite its record PartialClaude CodeIn the default auto mode, the model-based classifier blocks writes to session transcripts (v2.1.205+). That is a classifier policy, not a structural guarantee. The OS-level sandbox protects 'most of' ~/.claude, but it is off by default, covers only Bash, and bypassPermissions skips the protected-path checks.V1 Obtainable source, matching build FailClaude CodeThe runtime is closed: the public anthropics/claude-code repository's LICENSE.md says '© Anthropic PBC. All rights reserved', and the repository holds plugins, examples and the changelog, not the runtime source. A GPG-signed manifest verifies that binaries came from Anthropic, not that they match any source.V2 Owner-rooted identity FailClaude CodeThe agent has no identity of its own that the owner roots. It acts through the owner's claude.ai OAuth login, which Anthropic can revoke, and through the owner's own tool credentials. Its only default outward marker is a provider-branded 'Co-Authored-By: Claude' attribution.V3 Change history and rollback PartialClaude CodeCheckpointing (Esc Esc / rewind) restores the conversation and Claude's file edits. It does not track Bash-made or external changes, or changes to settings, permissions, model or runtime version. The provider-side sync of claude.ai skills and connectors has no change history. Memory edits appear only as tool calls in transcripts, which are pruned.M1 Instructions and logic PartialClaude CodeThe owner can replace the whole system prompt with --system-prompt or --system-prompt-file, add output styles, CLAUDE.md, rules and skills, and turn off git instructions and attribution. However, the system-prompt replacement is a per-launch flag rather than a persistent setting, and the docs do not show that every injected system reminder can be removed. Local edits to claude.ai-synced skills 'can be overwritten or removed' by a later sync.M2 Memory PassClaude CodePersistent memory is CLAUDE.md/AGENTS.md files, .claude/rules, and auto memory saved as Markdown files in ~/.claude/projects/<project>/memory/. All of them can be listed, edited and deleted as ordinary files or through /memory, and auto memory can be turned off.M3 Tools PassClaude CodeThe owner can add any tool, with no provider review, through MCP servers (claude mcp add), plugins, skills and hooks. Built-in tools can be removed with --tools, --disallowedTools or permission deny rules, and claude.ai connectors can be disabled.M4 Owner has the final say PassClaude CodeEvery documented protection can be switched off by the owner: sandboxing (sandbox settings), the auto-mode classifier (permission mode choice), protected-path checks (bypassPermissions), the WebFetch domain blocklist (skipWebFetchPreflight), telemetry, and the auto-updater. The actions no mode auto-approves (e.g. rm on critical paths) still ask the owner rather than refuse outright.M5 No unilateral change PartialClaude CodeBy default, native installs auto-update on the 'latest' channel, and features and the starting permission mode can change through server-fetched GrowthBook flags. Anthropic can also 'temporarily turn off' auto mode server-side. The owner can pin a version (DISABLE_AUTOUPDATER, DISABLE_UPDATES, or installing a specific version), disable flag fetching (DISABLE_GROWTHBOOK), and pin a full model ID. However, hosted models retire with at least 60 days' notice, which cannot be declined, and v2.1.257 rewrote a saved 'claude-fable-5' model setting to the 'fable' alias.C1 Boundaries PartialClaude CodeLimits can be set per tool, path, command prefix and domain through permission rules, and the OS sandbox adds file and network isolation. Channel plugins keep a per-sender allowlist, and the agent is blocked from editing its own settings files (protected paths). Spend is treated as met under G11 because there is no payment capability. One shortfall: sandbox limits cover only Bash commands, so MCP server and hook processes run outside them, and the docs call Bash rules 'not a security boundary'.C2 Approval gates PartialClaude CodeThe owner can put ask rules on any tool or pattern, and ask rules are honoured in every mode, including bypassPermissions. However, the docs say Bash ask and deny rules match only the command text as written and are 'not a security boundary around the program'. As a result, a gate on an action class performed through the shell (e.g. a given program) can be bypassed by another invocation form.C3 Immediate halt PartialClaude CodeEsc or Ctrl+C stops the foreground turn. Background subagents need a separate Ctrl+X Ctrl+K. Background sessions in agent view keep running after Ctrl+C or exit and need /stop or Ctrl+X per session. Cloud routines and desktop scheduled tasks are stopped separately.C4 Data serves only the owner FailClaude CodeOn the default Pro/Max sign-in, Anthropic trains on sessions when the consumer 'help improve Claude' setting is on. Anthropic's own pages do not say what the setting defaults to at signup. Third-party reporting (TechCrunch, labelled) says the consent pop-up toggle was preset to 'On', with a smaller switch than the Accept button. Under G12 the opt-in is not shown. Other flows: metrics telemetry, error reports (third-party service) and the WebFetch hostname check are on by default but can be turned off. Consumer Terms changes take effect when the owner keeps using the service ('continue to access'). Flagged content is used to train trust-and-safety classifiers.C5 Credential custody PartialClaude CodeLocal credentials are listed and revoked one at a time: /logout revokes the login, and each MCP server has 'Clear authentication' or claude mcp logout. Connector credentials for claude.ai connectors are held by Anthropic and managed at claude.ai/customize/connectors. No public statement says provider-side copies are deleted on revocation, so under G12 the pass is withheld.C6 Full deletion FailClaude Codeclaude project purge deletes local state one project at a time, and account deletion is a separate step on claude.ai. Provider-side copies survive: flagged sessions (up to 2 years), classification scores (up to 7 years), de-identified training data, and data kept 'as required by law ... or as necessary to combat violations'.
LobeHubP1 Owner-usable round trip PartialLobeHubSettings → Data Storage → Export Data gives an immediate JSON file, and Import Data restores it on any LobeHub v2 instance. Restoring off the provider's systems, though, means deploying the self-hosted stack first (setup script, Docker Compose, generated secrets, account creation), which is not a single owner action.P2 Complete export FailLobeHubDATA_EXPORT_CONFIG exports only settings, installed plugins, agents, providers/models, sessions, topics, threads and messages. Files, knowledge bases, embeddings with their chunks, user memory, agent documents, cron jobs/tasks and skills are commented out or absent, and the migration docs say API keys and secrets are not carried over.P3 Runs without the provider PartialLobeHubThe exported data runs on the self-hosted runtime under an Apache-2.0-based licence, with no licence key or provider account. Non-core capabilities degrade without LobeHub services, though: the agent/skill/MCP market defaults to market.lobehub.com (URL overridable, but no public server), and the Agent Gateway image ships from a public repo with no licence file.P4 Model independence PartialLobeHubOwners can configure many providers, including local Ollama, LM Studio and vLLM. Several capabilities depend on particular hosted models, though (image/video generation, TTS/STT, embedding-backed knowledge base), and the docs publish no minimum model contract. The glossary also says Custom API keys are 'Available on paid plans', which conflicts with the pricing table.P5 Exit survives termination FailLobeHubThe Terms say 'Upon termination, You will immediately lose the right to access or use the Services' and give no post-termination export window or shutdown plan. Since v2 the product is 'cloud-first' and has no local database, so no current copy stays with the owner by default.T1 Readable without the provider PartialLobeHubExported state is plain JSON whose schema is public in the open-source Drizzle definitions. Memory, files, knowledge bases, documents and trace snapshots can be read only through the provider-operated service, and the credential vault is encrypted with a server-held KEY_VAULTS_SECRET.T2 What runs is what you see PartialLobeHubProduction runs store trace snapshots by default, including context-engine output with the system messages and toolset, and the owner can read them through the documented `lh trace` CLI. Cloud also reads provider-controlled Vercel Edge Config feature_flags and assistant allow/deny lists server-side, which the owner cannot see.T3 No shadow data FailLobeHubThe Privacy Policy keeps personal data 'only for as long as is necessary' with no durations, and gives no retention table for backups, trace snapshots or usage data. Behavioural analytics profiles (PostHog person_profiles 'always') are not visible to the owner.T4 No hidden steering PartialLobeHubNo paid placement was found. Recommended skills are a hard-coded first-party/partner list visible in source and removable. Marketplace 'Featured' badges, the 'Agent Market highlights' plan feature and the non-disableable official provider in Cloud are not explained, and no binding no-steering commitment exists.A1 Complete action record FailLobeHubTool calls with their arguments are stored as messages, and task runs keep run records. Trace recording is explicitly best-effort (a store failure only logs to the server console), and no owner alert for failed recording is documented. No public evidence shows that provider-side automations (memory extraction, self-iteration, agent-signal runs) or staff actions are recorded to the owner (G12).A2 Tamper-evident, owner-checkable FailLobeHubMessages, operations and trace snapshots are ordinary database rows and S3 objects. No hashing, signing or external anchoring is documented or visible in the schema.A3 Agent cannot rewrite its record PartialLobeHubThe built-in tools examined have no API to delete the agent's own messages or topics (the message tool's deleteMessage targets external platforms). This protection is by convention only, however: on a desktop execution device the local-system tool's runCommand gives the agent a shell, where an authenticated `lh` CLI offers `message delete`.V1 Obtainable source, matching build FailLobeHubThe open repository ships stub 'business' slot packages (ENABLE_BUSINESS_FEATURES = false; AccountDeletion returns null) that the docs describe as 'Cloud-only business logic'. The Cloud offering therefore runs closed components, including billing, account deletion and the official provider. The licence also requires a commercial licence to distribute derivative works.V2 Owner-rooted identity PartialLobeHubInside LobeHub the agent's identity is a provider account. On messaging channels it acts through bot accounts the owner registers and controls (e.g. a Telegram bot token), which LobeHub cannot revoke. Attribution is per channel only.V3 Change history and rollback PartialLobeHubSome state is versioned: persona versions with restore, document history and expertise rule revisions. Agent instructions, tools, permissions, model and settings have no change log, attribution or rollback.M1 Instructions and logic PartialLobeHubThe owner can edit each agent's system prompt and import their own skills. Built-in tool system roles and the context-engine injectors (memory, workspace, identity context) are fixed in Cloud and can be changed only by modifying source when self-hosting (G3).M2 Memory PartialLobeHubIdentities, contexts, activities, experiences and preferences are itemised with update and delete procedures, and the docs describe edit/delete. The persona summary, which is injected into context, can only be restored to earlier versions: the persona 'editor' modal has no save handler and the router has no update procedure.M3 Tools PartialLobeHubThe owner can add custom MCP servers and import skills from GitHub/URL/ZIP without review, and can uninstall or disable built-in tools per agent. Hidden system tools (e.g. verify, acceptance-evidence) are not listed for removal, and the docs note that 'some system capabilities offer different options'.M4 Owner has the final say PartialLobeHubOwners can switch the approval mode (manual, allow-list, auto-run). Tools with the provider-set 'always' intervention policy and the security blacklist cannot be overridden by the owner.M5 No unilateral change FailLobeHubIn Cloud, LobeHub deploys continuously and applies server-side Edge Config feature flags, and owners cannot pin runtime versions or defer changes. Self-hosting allows image pinning, but moving to it means the lossy migration graded in P2.C1 Boundaries FailLobeHubMessenger channels have per-sender allowlists, and tools can be scoped per agent. No per-recipient contact limits or network egress policy is documented. Background task and bot runs use approvalMode 'headless' (all tools auto-execute), and the agent-management tool can update any agent's system prompt and plugins without approval by default, so the agent can change its own controls.C2 Approval gates PartialLobeHubOwners can set Allow/Ask/Deny per tool, plus manual or allow-list approval modes. Task runs and messenger-bot runs force approvalMode 'headless', so Ask gates are skipped on those paths.C3 Immediate halt PartialLobeHubThe foreground generation can be stopped. The docs say scheduled automation is disabled per task, and in-progress runs must be checked 'separately'. No single global stop is documented.C4 Data serves only the owner FailLobeHubThe 'Send Anonymous Usage Data' toggle defaults on and gates only product-usage events. When the deployment configures PostHog, it identifies users by email and name and captures page views regardless of the toggle, so that flow cannot be switched off. The Privacy Policy allows sharing with 'business partners … to offer You certain products, services or promotions', does not address staff access, and gives no notice commitment for legal disclosure.C5 Credential custody PartialLobeHubProvider API keys, connector authorisations and issued API keys are listed in Settings and removable one by one. The docs note that 'disabling does not revoke account authorization', and no policy says where credentials are held, limits provider use, or deletes copies on revocation.C6 Full deletion PartialLobeHubThe Cloud UI offers account deletion with a 72-hour cooling-off period, after which 'your account data will be permanently deleted', while registration and deletion records are retained. Backups, third-party copies and a completion confirmation are not addressed, and the mobile policy points to a support email.
CursorP1 Owner-usable round trip FailCursorCursor documents no export of the agent (conversations, account-held user rules, automation memories). The only documented export is a profile export, which excludes user and team rules. The only runtime that accepts Cursor state is the Cursor client, and it depends on Cursor's backend, which does the final prompt building for every request.P2 Complete export FailCursorNo complete export exists. Chat history, account-stored user rules, cloud agent/automation memories and integration grants (GitHub app, Slack) stay with Cursor or in undocumented local stores. Several classes of state are missing.P3 Runs without the provider FailCursorRequests go through Cursor's backend even with the owner's own API key, and Tab uses only Cursor models. The Terms grant only a 'limited right to access and use the Service', which Anysphere may suspend or discontinue at any time without notice.P4 Model independence PartialCursorOwners can add keys for OpenAI, Anthropic, Google, Azure and Bedrock, and an OpenAI-compatible base-URL override is mentioned. However, custom keys 'only work with chat models' (Tab stays on Cursor models). Because requests are routed through Cursor's backend, a local open-weight model has no documented official path.P5 Exit survives termination PartialCursorProject files, project rules, hooks and local chat history remain on the owner's device. However, the Terms say that on termination Anysphere 'may at our option delete any Content or other data associated with your account', and they give no post-termination export window, so account-held state (user rules, cloud agent and automation data) can be lost.T1 Readable without the provider PartialCursorRules (.mdc/AGENTS.md), hooks.json, mcp.json, permissions.json and sandbox.json are documented, human-readable files. Conversation history and checkpoints are stored locally in formats with no published schema, and account-side state is readable only through Cursor's service.T2 What runs is what you see FailCursorFinal prompt building happens on Cursor's backend, and no documented view exposes the assembled context or base prompt. On the model setting, Auto routing chooses the model per request, and on teams the default is to hide which model was used. No public evidence shows that individual owners can see the model Auto resolved to (G12).T3 No shadow data FailCursorThe Privacy Policy retains data 'only for as long as necessary' with periods that 'vary', which gives no bounded retention table. With Privacy Mode off, Cursor 'may use and store codebase data, prompts, editor actions, code snippets', and the owner cannot view that data item by item.T4 No hidden steering PartialCursorNo evidence of paid placement was found. Provider defaults such as Auto routing and Cursor's own models can be replaced in the model picker, and the Cursor Marketplace is curated. No binding commitment against self-preferencing or paid placement was found, so the absence rests on no statement at all.A1 Complete action record FailCursorThe durable record consists of chat transcripts, plus hooks the owner must write themselves (hooks are 'fail-open by default'). No documented mechanism records provider-side automations (Bugbot, cloud automations) into an owner record, warns when recording fails, or guarantees no pruning. Under G12, at least two shortfalls apply.A2 Tamper-evident, owner-checkable FailCursorNo integrity data (hash chain, signatures or checkpoints) is documented for conversation or action records.A3 Agent cannot rewrite its record PartialCursorIn the default Auto-review mode, sandboxed shell commands cannot write outside the workspace, and 'sensitive Cursor config files' are protected. Commands that cannot be sandboxed are passed to a model classifier that can allow them, and 'Run Everything' removes the sandbox, so the agent can reach its local record by one path.V1 Obtainable source, matching build FailCursorThe Cursor client and backend are closed. The public GitHub repository cursor/cursor contains only README, SECURITY.md and .github, and requests depend on Cursor's proprietary backend.V2 Owner-rooted identity FailCursorThe agent identity is the Cursor account. Cloud agents mint JWTs that are verified 'against Cursor's published JWKS', which makes them a provider-issued credential that the provider can revoke.V3 Change history and rollback PartialCursorCheckpoints snapshot and restore files the agent changed, and project rules and hooks live in git-tracked .cursor/ files. User rules are held on the Cursor account without history. Settings changes are not attributed to owner or agent, and cloud state is not covered.M1 Instructions and logic PartialCursorOwners can add and edit user, project and AGENTS.md rules, skills and subagents. The base system prompt is assembled on Cursor's backend and cannot be replaced.M2 Memory PassCursorThe documented persistent memory stores are rules, which are editable files or settings, and Automation memories, which 'can be viewed and edited from the tool configuration UI' and deleted. No other cross-session memory store is documented.M3 Tools PartialCursorOwners can add any MCP server through mcp.json without review. The docs do not show a documented setting to remove each built-in tool (for example terminal, web search or browser) from the IDE agent. Only hook scripts or approval settings can deny these tools.M4 Owner has the final say PartialCursorRun modes, sandbox networking and approval protections are owner-configurable, and 'Run Everything' disables them. However, 'Cursor's hardcoded security rules layer on top, so local files cannot weaken those protections'.M5 No unilateral change FailCursorAnysphere 'may modify or discontinue all or any portion of the Service at any time ... without notice'. Prompt building is server-side, Auto routing is 'managed by Cursor' with a model pool that 'changes over time', and the CLI auto-updates by default.C1 Boundaries FailCursor.cursorignore, sandbox.json and CLI allow/deny tokens scope files, network and commands, and spend is treated as met under G11 because there is no payment capability. However, 'Terminal commands and MCP tools run outside of Cursor's file access controls' (a bypass path), and no owner control limits whom the agent takes instructions from. Run modes are described as 'best-effort guardrails rather than a hard security boundary'.C2 Approval gates PartialCursorTerminal commands, MCP calls and fetches can be gated through Allowlist mode. However, 'Agents can modify workspace files without approval, except for configuration files', and the default Auto-review mode lets a classifier decide what is sent to the owner for approval.C3 Immediate halt PartialCursorA Stop button halts the foreground agent. Cloud agents and automations run separately, and no documented single control stops all of them at once.C4 Data serves only the owner FailCursorPrivacy Mode is documented as on by default only for teams and enterprise, and individuals are told to 'Toggle Privacy Mode on'. With it off, Cursor 'may use and store codebase data ... to train our models'. The Privacy Policy changes on publication, with continued use counted as acceptance, and legal disclosure has no notice commitment.C5 Credential custody PartialCursorAPI keys are visible and removable in Settings > Models, and Cursor says they are 'not persisted after the request completes'. There is no single documented view of every grant (GitHub/Slack apps, MCP OAuth tokens) with a per-grant revoke control.C6 Full deletion PartialCursorDeletion is self-serve from the dashboard, and 'All data is removed within 30 days'. It requires cancelling the subscription first, which makes it several steps, and no completion confirmation or coverage of backups and provider-side copies is documented.
Microsoft Copilot CoworkP1 Owner-usable round trip FailMicrosoft Copilot CoworkCowork documents no export of the agent and no import or restore on any runtime Microsoft does not operate. Only individual output files can be downloaded (singly or as a zip of up to 50 files / 500 MB).P2 Complete export FailMicrosoft Copilot CoworkOnly created files and custom SKILL.md files (both in OneDrive) can be taken out by the owner. Task conversations, scheduled prompts, 'don't ask again' permissions, built-in skills and the base configuration stay with Microsoft, and there is no owner-facing agent export.P3 Runs without the provider FailMicrosoft Copilot CoworkCowork runs only as a Microsoft-hosted service. It needs an active Microsoft 365 Copilot licence, Frontier enrolment and Microsoft accounts. Microsoft may change or discontinue previews at any time without notice.P4 Model independence FailMicrosoft Copilot CoworkMicrosoft selects the model ('Cowork picks the model best suited for the job'), and Cowork requires Anthropic to be enabled as a Microsoft subprocessor. There is no option to choose another model or a local one.P5 Exit survives termination PartialMicrosoft Copilot CoworkThe Product Terms give 90 days to extract customer data (including OneDrive files and skills) from a disabled account after expiry. Public evidence does not settle whether for-cause termination is covered, and previews can be discontinued without notice and with no published shutdown or export plan.T1 Readable without the provider FailMicrosoft Copilot CoworkTask state, schedules, permissions, built-in skills and the base logic live in Microsoft's service and can be read only through Microsoft's interfaces (the Cowork UI, Purview, eDiscovery). Microsoft holds the default encryption keys. Only custom skills are plain Markdown.T2 What runs is what you see FailMicrosoft Copilot CoworkThe base prompt and assembled context are not shown to the owner, and some background operations such as script execution run without showing their steps. The model in use cannot be determined: the Purview audit docs state 'Currently, Cowork doesn't show provider details'.T3 No shadow data PartialMicrosoft Copilot CoworkMicrosoft publishes bounded maximum retention for Microsoft 365 data classes (30 days after active deletion, 180 days after the subscription ends) and admins can search Copilot interactions. Microsoft-held service data (identifiers, audit records, execution-environment data) is not visible item by item, and Cowork-specific retention is not documented separately.T4 No hidden steering PartialMicrosoft Copilot CoworkNo paid placement is documented, and Cowork openly works through Microsoft 365 apps and Microsoft-chosen models. However, these first-party defaults cannot be switched off or replaced, and no binding commitment against self-preferencing was found.A1 Complete action record FailMicrosoft Copilot CoworkPurview CopilotInteraction records list accessed resources with read/create/modify actions, but not action arguments such as the email content sent. Audit Standard keeps them for 180 days. Coverage of scheduled runs is not documented, and there is no documented notice to the owner when recording fails.A2 Tamper-evident, owner-checkable FailMicrosoft Copilot CoworkMicrosoft holds the Purview audit log, and no hash chain, signature or independent verifier is documented that would let the owner detect edits or deletions.A3 Agent cannot rewrite its record PartialMicrosoft Copilot CoworkMicrosoft's service writes the audit records and Cowork's documented skills have no audit-log access. However, no protection against the agent altering or suppressing records is documented, and Cowork acts with the user's (here, admin's) permissions and runs scripts in its sandbox.V1 Obtainable source, matching build FailMicrosoft Copilot CoworkCowork and its required Microsoft 365 services are closed source.V2 Owner-rooted identity FailMicrosoft Copilot CoworkCowork acts as the user's Microsoft Entra / Microsoft 365 account ('Every action Cowork takes is authorized through your Microsoft 365 account'). That identity is issued by Microsoft and Microsoft can revoke it.V3 Change history and rollback PartialMicrosoft Copilot CoworkCustom skills sit in OneDrive, whose version history shows and restores edits, and admin setting changes are audited in Purview. Changes to the model, base instructions, built-in skills and approval permissions have no attributable history or restore.M1 Instructions and logic PartialMicrosoft Copilot CoworkThe owner can add up to 50 custom SKILL.md skills, but the base prompt and the 13 built-in skills cannot be replaced or edited.M2 Memory PartialMicrosoft Copilot CoworkCustom skills can be listed, edited and deleted item by item, and task conversations can be resumed and deleted through the activity history. The Cowork docs do not say whether Cowork uses Copilot memory, whose chat-history-derived details are not listed item by item or editable.M3 Tools PartialMicrosoft Copilot CoworkCustom skills are instructions, not new tools. The product page mentions plugins for partner tools, but Cowork's docs describe no owner-built tool interface, and built-in skills cannot be removed.M4 Owner has the final say PartialMicrosoft Copilot CoworkThe owner-admin can configure DLP and auditing. Microsoft's harmful-content filtering, the isolated execution sandbox and the by-design limits (no deletion of OneDrive or SharePoint files, no local files) cannot be changed.M5 No unilateral change FailMicrosoft Copilot CoworkMicrosoft picks and regularly updates the models, and previews 'may change over time' or be changed or discontinued without notice. There is no version pinning or opt-in for updates.C1 Boundaries FailMicrosoft Copilot CoworkThere is no owner setting to limit which senders or content Cowork takes instructions from, and no per-recipient or per-channel contact list for the agent. Data scope is the user's full permissions, narrowed only by DLP and labels. Cowork has no payment capability, so the spend dimension was treated as met under G11.C2 Approval gates PartialMicrosoft Copilot CoworkCowork asks before 'sensitive' actions and shows previews and parameters, but Cowork, not the owner, decides which actions are gated. 'Don't ask again' and 'Approve All' can skip later prompts.C3 Immediate halt PartialMicrosoft Copilot CoworkEach task has a hard pause and a cancel, but scheduled prompts must be paused separately. No single control is documented that stops all tasks and schedules at once.C4 Data serves only the owner PartialMicrosoft Copilot CoworkPrompts and responses are not used to train foundation models, and feedback is optional. Sending content to Anthropic as a subprocessor is on by default outside the EU/UK (opt-out, not owner-selected), and web search to Bing is controlled by an admin setting. Data terms change on notice, not with specific consent.C5 Credential custody PartialMicrosoft Copilot CoworkCowork acts through delegated Microsoft 365 access, and admin deployment 'accepts users' permissions on their behalf'. Access can be removed by blocking Cowork or the user, which is all-or-nothing. No per-credential list or revoke control is documented for Cowork.C6 Full deletion FailMicrosoft Copilot CoworkThere is no single action that deletes the whole agent. Activity history, skills and schedules are deleted separately, and Purview audit records and Copilot memory survive deletion of chat history. Full erasure happens only when the tenant subscription ends (within 180 days, with no documented confirmation).
AutoGPT PlatformP1 Owner-usable round trip PartialAutoGPT PlatformA single UI action exports a workflow as JSON, and a documented Import action restores it on a self-hosted instance, which counts as non-provider infrastructure. But restoring means standing up that instance first: cloning, generating env config and running Docker Compose by hand, since the docs say not to use the hosted one-command installer yet. That is one shortfall.P2 Complete export FailAutoGPT PlatformThe export covers one workflow graph with its sub-graphs. Credentials, secrets and webhook IDs are stripped. Otto/Expert configuration, memory (Graphiti and the business-understanding profile), execution history, schedules, presets and credentials have no export. Workspace files download one at a time, and the chat export is Markdown that skips tool-result messages.P3 Runs without the provider PartialAutoGPT PlatformAn exported graph runs on the self-hostable runtime. No licence check, update check or AutoGPT-run service is needed, and LaunchDarkly, Langfuse, Sentry and PostHog are off when their keys are empty. The PolyForm Shield grant is not stated to be perpetual or irrevocable, bars competing uses, and ends on an uncured violation after notice. Features gated by LaunchDarkly flags fall back to code defaults (for example, Graphiti memory is off) unless the owner forces them by env var. Provider-managed credentials must be replaced with the owner's own keys.P4 Model independence PartialAutoGPT PlatformOn the cloud offering, workflow LLM blocks can use models from several vendors (Anthropic, OpenAI, Google, Meta, Mistral and others via hosted APIs), and Otto can use a Codex or Microsoft 365 Copilot connection. The cloud cannot use local models: the docs say Ollama is self-host only and that cloud AutoPilot routes through Anthropic/OpenRouter. Owners choose only a Balanced/Advanced tier. Voice, transcription and search embeddings are tied to specific OpenAI models.P5 Exit survives termination FailAutoGPT PlatformClause 8.2 of the Terms lets AutoGPT suspend or terminate access to the Platform at any time, and clause 4.2 lets it terminate registrations at any time. There is no post-termination export window, no shutdown plan and no owner-held copy of state by default.T1 Readable without the provider FailAutoGPT PlatformAgent state (graphs, executions, chats, Experts, memory in FalkorDB, profile) lives in provider-operated Postgres and FalkorDB. Apart from the graph JSON and Markdown chat exports, it can be read only through the hosted service. Credentials are encrypted with a server-side key the provider holds.T2 What runs is what you see FailAutoGPT PlatformOtto's base system prompt is fetched at runtime from the provider's Langfuse prompt store; the code fallback is not necessarily what runs. The server-injected context blocks are removed before display, and compaction summaries are deliberately hidden from the card. The exact model is chosen per user by a provider LaunchDarkly routing flag, and owners see only a tier. Several behaviour switches are provider-side flags. Both shortfalls apply.T3 No shadow data FailAutoGPT PlatformThe privacy policy collects a broad profile (interests, LinkedIn presence, location, usage). On retention it says only that data is not kept longer than needed, with details available by email, so retention is neither disclosed nor bounded. (The agent-used business-understanding profile is graded in M2.)T4 No hidden steering PartialAutoGPT PlatformI found no paid or sponsored placement in the code. Onboarding recommendations sort provider-featured marketplace agents first. Otto's find_agent tool steers toward marketplace agents, and provider-chosen default models and managed credentials are auto-selected. None of this is covered by a binding no-steering commitment, and the featured ordering cannot be switched off.A1 Complete action record FailAutoGPT PlatformGraph runs record inputs, outputs and trigger source (manual, schedule, webhook, sub-graph, copilot). Copilot block runs are recorded only as chat tool messages, with tool output truncated to about 100 KB. Admin impersonation acts under the user's ID and is logged only server-side, so staff actions are not distinguishable in the owner's record. Activity-event and cost-log writes swallow errors. Two or more shortfalls apply.A2 Tamper-evident, owner-checkable FailAutoGPT PlatformThe execution and chat records carry no hash chain, signature or external anchor. The model the code calls an 'append-only' ActivityEvent has no enforcing trigger or permissions, and the chat-message rows can be updated in place.A3 Agent cannot rewrite its record PartialAutoGPT PlatformExecution and chat records are written by the backend, and no copilot tool deletes executions, graphs or chat history. However, tool outputs over about 80 KB are moved to workspace files, leaving only a preview in the chat, and the agent has a delete_workspace_file tool. That is one path to suppress part of the record. The protection is also undocumented.V1 Obtainable source, matching build PartialAutoGPT PlatformSource for the platform is public under PolyForm Shield, which allows modification for permitted, non-competing purposes. Owners have no way to tie the hosted cloud build to a commit. The production system prompt and flag configuration come from Langfuse and LaunchDarkly rather than the repository.V2 Owner-rooted identity FailAutoGPT PlatformAgents and Experts have no cryptographic or owner-rooted identity. They are database rows under a provider-issued account, and agent email (AgentMail) is a provider-managed credential.V3 Change history and rollback PartialAutoGPT PlatformWorkflow graphs keep numbered versions, can be listed, and can be rolled back by setting the active version. Changes are not attributed: an Otto edit looks the same as a user edit. Expert soul and boundaries, settings, credentials and the business profile are overwritten with no history. The code notes 'no undo for boundaries'.M1 Instructions and logic PartialAutoGPT PlatformWorkflows are fully editable, and Expert 'soul' text (identity, voice, boundaries) can be edited. The base Otto prompt, however, comes from the provider's Langfuse store and cannot be replaced; soul text is layered beneath it. Built-in default skills are not downloadable or replaceable. (Locked protected soul rules are graded in M4.)M2 Memory FailAutoGPT PlatformGraphiti memory, where the flag enables it, can be listed and forgotten but not edited directly. The CoPilotUnderstanding business profile (partly populated from a Tally form at signup) is injected into Otto's context but has no user view, edit or delete route; the clear function exists but is never called. Skills can be managed.M3 Tools PartialAutoGPT PlatformOwners can connect any MCP server by URL with only SSRF checks and no provider review. Built-in tools and blocks can be excluded only inside a workflow's AutoPilot block. Plain chat has no setting to remove built-ins, and provider-disabled blocks cannot be re-enabled.M4 Owner has the final say FailAutoGPT PlatformProvider-imposed protections are all set by operator env vars or provider flags, and I found no owner setting to change any of them. These are the E2B sandbox, ClamAV scanning, the SSRF blocklist, per-user AutoMod moderation and the protected Expert soul rules. The daily/weekly spend caps can only be reset by buying credits.M5 No unilateral change FailAutoGPT PlatformThe provider can change Otto's prompt through Langfuse (production label, re-fetched at runtime) and the model through the LaunchDarkly routing flag. Saved graph model choices are rewritten on every server start. The changelog records retired models being auto-migrated. Only schedules can pin a workflow version; the runtime, prompt and Otto model cannot be pinned.C1 Boundaries FailAutoGPT PlatformHost-scoped credentials, deny-by-default Expert credentials, Expert weekly budgets and copilot spend caps exist. The contact dimension is missing: there are no recipient, sender or domain allow/deny lists. The agent can also change its own limits: Otto can grant and revoke Expert credentials, and can propose a zero (disabled) budget through the confirm flow.C2 Approval gates PartialAutoGPT PlatformWorkflow owners can put human-in-the-loop approval blocks anywhere in a graph. Sensitive-action review covers a fixed set of irreversible blocks and is off by default. The copilot approval gate applies only to interactive sessions behind a default-off flag, so scheduled, routine and AutoPilot-block turns bypass it. In auto mode an LLM supervisor decides what to ask about.C3 Immediate halt PartialAutoGPT PlatformOwners can stop a single execution, which cascades to child runs, and cancel a chat turn. There is no user-level stop-all. Schedules must be deleted separately, and Expert pause has no manual route.C4 Data serves only the owner FailAutoGPT PlatformClause 7.2 of the Terms reserves the right to use submitted information 'on an unrestricted and free-of-charge basis'. Copilot prompts, responses and tool calls are traced to Langfuse, and backend PostHog events are sent with no user opt-out. Marketing relies on legitimate interest (opt-out). Terms change by continued use. Staff impersonation is disclosed only as a pricing-table row.C5 Credential custody PartialAutoGPT PlatformSettings → Integrations lists credentials and deletes them one by one; OAuth tokens are revoked upstream for some providers only (Notion and Todoist are not). Provider-created managed credentials (AgentMail, Ayrshare) cannot be deleted by the user. No binding terms limit provider use of stored credentials or require deleting copies after revocation.C6 Full deletion FailAutoGPT PlatformThe product has no account-deletion feature in the UI or API. The privacy policy directs rights requests to email but gives no deletion scope, time limit, backup treatment or confirmation. A helper for pre-deletion cleanup exists in code but nothing calls it.
ChatGPTP1 Owner-usable round trip FailChatGPTSettings > Data Controls > Export emails a zip of chat history and account data, which can take up to 7 days to arrive, and the link expires after 24 hours. OpenAI documents no import or restore path, and no runtime outside OpenAI's service accepts the export.P2 Complete export FailChatGPTThe documented export contains chat history, custom instructions and 'other relevant account data'. The documentation does not say it includes the chat-history memory synthesis (which the Memory FAQ describes as broader than the visible summary), scheduled tasks, agent browser sessions, or connector grants. Under G12, these classes count as missing.P3 Runs without the provider FailChatGPTChatGPT runs only as an OpenAI-hosted service that needs an OpenAI account. The Terms reserve the right to suspend or terminate access and to discontinue the service. No exported agent can run with OpenAI unreachable.P4 Model independence FailChatGPTChatGPT offers only OpenAI models, chosen from OpenAI's model picker, and OpenAI routes between them automatically. There is no documented way to use a third-party or local open-weight model.P5 Exit survives termination PartialChatGPTThe owner can take manual exports while the account is active, and the Privacy Portal accepts data requests from users who cannot log in. The Terms allow suspension or termination (for breach, legal reasons or risk) and discontinuation with advance notice, but they commit to no post-termination export window, and the Privacy Policy allows banned content to be retained under OpenAI's control. The exit therefore relies on a manual, discretionary route.T1 Readable without the provider PartialChatGPTThe export is a zip file that general-purpose tools can open, but OpenAI publishes no schema for it. The chat-history memory synthesis is opaque ('broader than what can be shown'), and provider-supplied system prompts and tool logic cannot be read at all.T2 What runs is what you see FailChatGPTBoth shortfalls apply. The owner cannot see OpenAI's system prompt, which OpenAI updates (for example the '5.2 Personality System Prompt Update'). Material settings cannot always be determined: Instant auto-switches between models and may show no trace, rate limits trigger a silent fallback to a mini model, and OpenAI adjusts default thinking time server-side.T3 No shadow data FailChatGPTThe Privacy Policy collects log, usage and device data, and it describes retention only as 'as long as we need' or for 'legitimate business purposes'. The retention period for banned content is left blank ('retain that data for to protect our services'). De-identified content kept for training is not bounded. None of this data is visible to the owner item by item.T4 No hidden steering PartialChatGPTOpenAI states that Plus has no ads and that product results are 'organic and unsponsored'. These statements are in a blog post and help pages, not binding terms. Merchants pay OpenAI a fee on Instant Checkout sales, and merchant ranking considers 'whether Instant Checkout is enabled'. This preference is disclosed, but the owner cannot switch it off.A1 Complete action record FailChatGPTThe conversation transcript is the only action record. The documentation does not say that recording failures are surfaced to the owner, or that agent-mode virtual-browser and terminal actions are recorded with their arguments and targets (it mentions screenshots and 'browsing history'). Under G12 these count as two shortfalls.A2 Tamper-evident, owner-checkable FailChatGPTNo documentation describes integrity data, hashing or signing of the conversation or action record. OpenAI holds the record and can change it without trace.A3 Agent cannot rewrite its record PartialChatGPTOpenAI's servers write the transcript outside the model's tools, but OpenAI documents no guarantee of this. Agent mode runs a signed-in virtual browser and 'can take actions on your behalf, e.g. modifying account settings', and no documentation says it cannot reach the owner's own ChatGPT history. Under G12 the protection is treated as unverified on that path.V1 Obtainable source, matching build FailChatGPTThe ChatGPT service, models and apps are closed source. The Terms prohibit attempts to discover source code.V2 Owner-rooted identity FailChatGPTThe agent has no identity of its own apart from the owner's OpenAI account, and OpenAI can suspend or terminate that account.V3 Change history and rollback PartialChatGPTSaved memories (the legacy system) have 'view history' with restore to earlier versions by date. The default memory summary, custom instructions, settings, tools and model changes have no documented change log, attribution or restore.M1 Instructions and logic PartialChatGPTThe owner can edit custom instructions (1500-character fields) and choose personality and style presets. The owner cannot replace OpenAI's base system prompt, which OpenAI updates on its own schedule.M2 Memory PartialChatGPTSaved memories are itemised, editable and deletable. The chat-history memory synthesis is not itemised: the summary 'will not include everything', and it can only be corrected or cleared as a whole. A separate safety feature keeps using prior-conversation context after memory is turned off.M3 Tools PartialChatGPTPlus owners can add any remote MCP server without OpenAI review through developer mode, but developer mode is a documented beta feature (capped at partial by G10). Connectors can be removed. The documentation does not show that every built-in tool can be removed.M4 Owner has the final say FailChatGPTOpenAI sets and enforces the protections in scope, including the agent-mode site blocklist, watch mode, content refusal patterns and safety-context use. The owner cannot change them, and on Plus site blocking is available only to workspaces through OpenAI support.M5 No unilateral change FailChatGPTOpenAI changes behaviour server-side without consent. It lowered default thinking time in January 2026, which included an 'unintentional' reduction to Extended. It updated the default personality system prompt. It retired GPT-4o and other models about two weeks after announcing it (29 Jan to 13 Feb 2026), and moved old chats to new models. The Terms let non-material changes take effect on posting.C1 Boundaries FailChatGPTThe owner can only switch individual connectors and developer-mode tools on or off. Plus has no documented limits on instruction sources, recipients or domains, no network egress policy, and no spend caps, although Instant Checkout lets ChatGPT make purchases.C2 Approval gates PartialChatGPTAgent mode asks for confirmation on actions OpenAI's model classes as 'high-impact'. Developer mode gates write actions based on the tool author's readOnlyHint, and an approval can be remembered for the rest of a conversation. The owner cannot choose which action classes need approval.C3 Immediate halt PartialChatGPTThe owner can interrupt the current agent turn. Scheduled tasks and recurring agent tasks must be paused one at a time from the Tasks or Schedules pages, and there is no documented global stop.C4 Data serves only the owner FailChatGPTOn consumer plans, 'Improve the model for everyone' is on by default, so content, including memories and data accessed through connectors, is used for training unless the owner opts out. Authorized OpenAI staff may access agent content. Changes to the Privacy Policy take effect on posting.C5 Credential custody PartialChatGPTConnectors are listed in Settings and can be removed one by one. Agent-mode site logins persist as cookies, which can only be cleared as a group from data controls. OpenAI does not document where tokens are held or whether it keeps copies after revocation.C6 Full deletion FailChatGPTThe owner can delete the account in Settings, with deletion 'within 30 days'. Data that has already been de-identified for training, data under legal hold, and banned content are exempt, so some provider-side copies survive. No completion confirmation is documented.
DevinP1 Owner-usable round trip FailDevinThe docs describe no export of a Devin agent and no importer or runtime outside Cognition's service. Skills can be read through the API and uploaded plugins downloaded as .zip files, but no restore target the provider does not operate exists. Outposts runs only tool execution locally; the agent loop stays in Devin's cloud.P2 Complete export FailDevinThere is no export bundle. Session history, schedules, automations, secrets, repository indexes and environment snapshots all stay with the provider. Only some pieces (skills, plugin zips, session messages) can be fetched one at a time through the UI or API.P3 Runs without the provider FailDevinThe agent needs a Cognition account and a running Cognition service to start or act. Inference and planning run in Devin's cloud even with Outposts. The ToS grants only a non-transferable right to access the Services 'during the Term'.P4 Model independence FailDevinThe owner can pick only from a model catalogue that Cognition curates and serves (Anthropic, OpenAI, Google, Cognition and some hosted open models). No custom endpoint or local open-weight model is documented, so model choice depends on the provider. Linking a ChatGPT plan changes billing only.P5 Exit survives termination FailDevinCognition may suspend or terminate at any time, including 'for any other reason'. On termination it 'may at our option delete any Customer Data'. No post-termination export window or shutdown plan is published, and state is held in the cloud rather than by the owner.T1 Readable without the provider FailDevinMain agent state (sessions and worklogs, memory, schedules, automations) can be read only through Cognition's web app or API, and Cognition holds the encryption keys (AWS KMS). Skills are readable SKILL.md text, but that covers only part of the state.T2 What runs is what you see FailDevinTwo shortfalls. (1) The docs describe no way to view Devin's base system prompt or the fully assembled context. (2) The default 'Devin' agent's model identity is not stated, and short model names resolve to 'the latest version'.T3 No shadow data FailDevinRetention is open-ended. Personal data is kept 'as needed'. 'Feedback Data and User Interaction Data are retained as long as needed and as determined by Cognition'. Cognition owns Usage Data outright. None of this is visible item by item.T4 No hidden steering PartialDevinNo paid placement was found. First-party defaults (the default Devin agent, the recommended Fusion mode, promoted SWE-2 and the official marketplace) are visible and can be replaced. No binding commitment against steering exists, so absence rests on non-binding evidence only.A1 Complete action record FailDevinSessions keep a worklog of shell commands and tool steps. Audit logs, which cover admin, settings and provider-side events, need enterprise permissions, so a Pro owner has no record of provider or staff actions. No alert is documented for when recording fails, and hooks are 'best effort'. That is two shortfalls.A2 Tamper-evident, owner-checkable FailDevinNo integrity data (hash chaining, signatures or an owner-held checkpoint) is documented for session worklogs or audit logs. The record holder can change the record without trace.A3 Agent cannot rewrite its record PartialDevinThe worklog is produced by the hosted platform, outside the agent's VM. But by default the agent has the built-in Devin MCP with write access to session management (child sessions, schedules, knowledge). Making it read-only needs a non-default security profile, and no document says the agent cannot archive or suppress its own record.V1 Obtainable source, matching build FailDevinRequired components (the hosted agent service and the Outposts control plane) are closed source. The ToS forbids copying, modifying or reverse engineering the Services.V2 Owner-rooted identity FailDevinThe agent acts through provider-issued identities: the Cognition account and Devin's GitHub App / Slack app. Cognition can revoke or reissue these. No identity root is held by the owner.V3 Change history and rollback PartialDevinPlugins (skills, rules, hooks, MCPs) have version history with diffs, and every save can be restored. The docs describe no change history or restore for other state: knowledge, secrets, settings, model or runtime version. Settings audit logs are enterprise-only.M1 Instructions and logic PartialDevinThe owner can add and edit skills, rules, playbooks, AGENTS.md and plugins. Devin's base system prompt and built-in agent behaviour cannot be replaced.M2 Memory PartialDevinKnowledge (now migrating to Skills) is itemised and can be edited, disabled and deleted. The repository index Devin reuses across sessions is not itemised or editable; the only option is removing a repository from indexing.M3 Tools PartialDevinThe owner can connect custom MCP servers and plugins, from any repository or by upload, without provider review. No documented setting removes built-in tools (shell, browser, editor, Devin MCP) in cloud sessions.M4 Owner has the final say PartialDevinThe owner can opt out of optional security profiles. Provider-imposed protections cannot be changed: automated safety and abuse classifiers (ToS §3.3.3) and the provider-managed session VM. Because no exit route exists, these do not fall under the hosted-policy carve-out.M5 No unilateral change FailDevinThe Services are 'subject to modification and change, in Cognition's sole discretion'. Model aliases resolve to the latest version, and new agent behaviours roll out server-side (release notes). There is no pinning of the cloud runtime, agent or model.C1 Boundaries FailDevinSecurity profiles offer per-domain network allowlists, per-server MCP allowlists, git read-only/full access and session spending limits. Two shortfalls remain. (1) Tool scope is coarse: built-in shell and browser cannot be scoped, and MCP is per server, not per tool. (2) By default the agent has write access to its own configuration through the Devin MCP, and no document shows that self-edits cannot widen its limits.C2 Approval gates PartialDevinOnly a fixed set of actions can be gated: network-destination requests, plugin/MCP/skill changes and installs, and the git read-only level. Cloud hooks are 'best effort', so the owner cannot gate arbitrary action classes.C3 Immediate halt PartialDevinIndividual sessions can be stopped or terminated, and terminating with archive sleeps descendant sessions. Schedules and automations have to be paused separately, and there is no documented single global stop.C4 Data serves only the owner FailDevin'By default, we may use your data for model training.' Opting out is available only on paid tiers. Data-term changes apply through continued use after 30 days' notice, and flagged content may be kept for safety review.C5 Credential custody PartialDevinSecrets can be listed, viewed by admins and deleted one by one. The ChatGPT link and GitHub App grants can be revoked. No document says the provider keeps no copy after revocation, or limits provider use of credentials to owner-directed actions, so a pass is withheld (G12).C6 Full deletion PartialDevinNo one-step, owner-initiated account deletion is documented. Erasure goes by request to privacy@cognition.ai, with no stated completion limit or confirmation. The DPA gives 30 days for subprocessor deletion after termination.
LettaP1 Owner-usable round trip FailLettaAgentFile (.af) export/import has been removed from Letta Code, and no command moves a Cloud agent to local mode or a self-hosted runtime. Letta's docs say the legacy API export cannot be imported by the local backend or a self-hosted App Server, so restore works only on provider-operated systems.P2 Complete export FailLettaOnly memory (a MemFS directory copy) and a plain-text conversation transcript can be exported; the transcript is marked truncated past 200 pages. The system prompt, model, tool and permission config, Cloud schedules (held in Letta's cloud-api database), server-held secrets and the agent identity stay with the provider.P3 Runs without the provider FailLettaIn the default offering the agent's state lives on Letta Cloud, and the CLI needs a Letta account or token to start a Cloud agent. There is no exported form that runs without the provider. Local mode is Apache-2.0 and needs no account, but a Cloud agent cannot be moved into it.P4 Model independence PartialLettaOwners can bring their own keys, and /connect offers Ollama, LM Studio, llama.cpp and any OpenAI-compatible endpoint. However, no minimum model contract is documented. Defaults point to hosted routes: the compaction model is letta/auto and the title model is openai/gpt-6-luna. Full capability with a local open-weight model is therefore not established.P5 Exit survives termination FailLettaThe terms reserve the right to 'suspend or end the Services at any time at our discretion and without notice'. They include no post-termination export window and no shutdown plan. Apart from a locally synced copy of memory, termination ends access to the Cloud-held state, and mirroring memory to an owner-held repository is off by default.T1 Readable without the provider FailLettaIn the default Cloud mode, message history, agent configuration, schedules and secrets can be read only through Letta's hosted API. Memory is readable as Markdown in a local git clone, and local mode stores readable JSON/JSONL, but no schema is published.T2 What runs is what you see FailLettaTwo shortfalls. (1) On Cloud, context is assembled server-side: default prompts become Cloud-managed (system set to null), and /context shows only token counts, so the base prompt and compaction summaries in effect are not shown. (2) The default model 'auto' resolves to the hosted letta/auto router, and which downstream model runs is not documented.T3 No shadow data FailLettaThe privacy policy discloses no retention periods ('We generally discard information about you when it's no longer needed'). It also describes holding data the owner cannot see item by item, such as inferences ('likelihood of retention or attrition'), log data and hashed email used for ad matching.T4 No hidden steering PartialLettaThe provider's own letta/auto router is the default model, and server-supplied 'featured' models sort first in the selector. Both are visible and replaceable via /model. No paid placement was found, but that absence rests on no binding commitment, and letta/auto's routing criteria are undisclosed.A1 Complete action record FailLettaTwo or more shortfalls. Hook commands are only printed to the console and not stored durably. Tool returns in the record are truncated at about 30K characters by default, and the overflow file that holds the full output is skipped silently if it cannot be written.A2 Tamper-evident, owner-checkable FailLettaThe action record has no integrity data: no hash chain, signatures or checkpoints, locally or on Letta Cloud. MemFS commits are deliberately unsigned.A3 Agent cannot rewrite its record FailLettaThe default permission mode is 'unrestricted', which auto-allows every tool, and the filesystem sandbox is off by default. The agent's shell environment receives LETTA_API_KEY, so the agent can call the Letta API, which includes a reset of agent messages. It can also edit local transcripts and logs.V1 Obtainable source, matching build FailLettaThe client is Apache-2.0, and npm shows SLSA provenance for 0.34.0. But the default mode requires Letta Cloud, and its schedule service is documented in code as 'served by the Letta API (cloud-api), not the OSS core server', which is a closed required component.V2 Owner-rooted identity FailLettaThe agent's identity is a Letta-issued ID (agent-<uuid>) created by the Cloud API, and memory commits use <agentId>@letta.com. No owner-held key, DID or domain identity exists.V3 Change history and rollback PartialLettaMemFS git versions memory, the prompt and skills, and memory backup/restore exists. But owner edits are committed under the agent's own identity, so authors cannot be told apart. Settings, permissions, tools and model changes are not versioned, and Cloud-held state is not covered.M1 Instructions and logic PartialLettaThe base prompt can be replaced with --system-custom, and custom prompts are never overwritten. Bundled skills can be overridden or disabled. The text of the injected system reminders is hard-coded and can only be switched on or off, not rewritten.M2 Memory PartialLettaMemFS memory files can be itemised, edited and deleted via /memory, /palace and plain files. Conversation history, which the agent searches across sessions, can only be deleted whole (whole conversation or /clear-messages), not edited per item.M3 Tools PassLettaOwners can add MCP servers, skills, mods and custom subagents without provider review. Built-in tools can be removed with --tools, --disallowedTools, /toolset or deny rules, and server-side base tools are chosen with --base-tools.M4 Owner has the final say PartialLettaMost protections can be switched off: sandbox (LETTA_FS_SANDBOX), memory guard (--disable-memory-guard), overflow and channel gating. Some are locked: in subagents the cross-agent guard is 'unbypassable by any mode, rule, or flag', and Windows deletes against protected paths are always refused.M5 No unilateral change FailLettaAuto-update is on by default and installs patch releases silently. On resume, default Cloud prompts become server-managed and can change without notice. The default model is a hosted alias (letta/auto), and the terms say 'we may automatically upgrade our Services'. Env-var opt-outs exist for the client, but server-side behaviour cannot be pinned.C1 Boundaries FailLettaPer-tool and per-path allow/deny rules exist. Nothing stops the agent from editing its own settings and permission files, and memory subagents may write all of ~/.letta. There are no network egress or domain limits for the shell. Group channels default to open.C2 Approval gates PartialLettaAny client tool can be gated with ask/alwaysAsk rules or strict mode, and headless mode denies unapproved actions. Server-side tools (web_search, fetch_webpage) run on Letta Cloud outside client gates.C3 Immediate halt PartialLettaEsc aborts the foreground turn and sends a server cancel request whose errors are ignored. Background subagents run as detached processes and keep running. Crons and Cloud schedules need separate commands to stop.C4 Data serves only the owner FailLettaThe privacy policy says hosted-service message requests and responses may be used 'to train new models' and offers no opt-out. Telemetry is on by default and sends a device ID, the agent ID, startup arguments and error output to api.letta.com, including in local mode. The privacy policy changes on continued use.C5 Credential custody PartialLettaCloud secrets and provider keys can be listed and removed individually (/secret, /connect), and /logout revokes the token. There is no single list of all credentials, and no commitment that provider-side copies are deleted on revocation.C6 Full deletion FailLettaAgent deletion calls a single API delete and leaves the local MemFS clone, crons and logs in place. No deletion limit, backup coverage or confirmation is documented. Account deletion is only by a privacy-rights email request, which Letta 'may deny'.
GrokP1 Owner-usable round trip FailGrokThe owner can download their data from Settings > Data Controls, but there is no documented importer or runtime outside the provider that restores a working agent from that download.P2 Complete export FailGrokNo primary source documents what the data download contains. Memory, skills, automations, project configuration, files and connector links are not shown to be included, so under G12 two or more classes stay with the provider.P3 Runs without the provider FailGrokThe grok.com agent runs only on provider servers under a SpaceXAI account. The Terms let SpaceXAI suspend or discontinue the service 'for any reason … or no reason'.P4 Model independence FailGrokgrok.com offers only xAI models (Auto, Fast, Expert and Heavy, based on Grok 4.6). There is no documented way to use another vendor's model or a local model in the grok.com agent.P5 Exit survives termination FailGrokThe Terms allow termination or account deletion 'at any time without notice', including after 120 days of inactivity. There is no post-termination export window, and no copy of the state is held by the owner by default.T1 Readable without the provider FailGrokAgent state (memory, skills, automations, history) lives on provider servers and is readable through the provider's app. The format of the data download is not publicly documented.T2 What runs is what you see FailGrokxAI publishes system prompts on GitHub, but the last commit is 2025-11-17 and there are no prompts for Grok 4.20 through 4.6, skills, connectors or automations. The model behind the default 'Fast' mode and the server-side feature flags cannot be determined by the owner. That is two shortfalls.T3 No shadow data FailGrokData is kept 'where we have an ongoing legitimate business need' with no fixed period. De-identified copies of deleted chats may be kept, and no retention periods are given for safety logs or backups.T4 No hidden steering PartialGrokThe primary documents do not mention sponsored content or paid placement in Grok responses, but no binding commitment rules it out. First-party X search is a disclosed built-in source, and no switch to turn it off was found.A1 Complete action record FailGrokActions show up only inside chat threads (tool-call accordions) and automation run histories. The client config hides raw tool arguments (hide_tool_json_args: true), the owner can delete chats, and nothing documents alerts when recording fails. That is two or more shortfalls.A2 Tamper-evident, owner-checkable FailGrokNo integrity data or verifier is documented for chat or automation history; the provider-held record can be changed without trace.A3 Agent cannot rewrite its record PartialGrokThe record is kept by the provider's chat service, outside the agent's code sandbox, but no documentation states that the agent's tools cannot alter or suppress it. Protection rests on architecture that is not documented.V1 Obtainable source, matching build FailGrokThe grok.com app and backend are closed source. The released Grok-1 (Apache-2.0) and Grok-2 (revocable community licence) weights and the Grok Build CLI are not the components that run this agent.V2 Owner-rooted identity FailGrokThe agent's only identity is the owner's SpaceXAI account, which the provider can suspend or delete. Outbound actions go through the owner's third-party accounts via provider-held connectors.V3 Change history and rollback PartialGrokMemory update cards show some memory changes during conversations, but there is no change history across instructions, skills, settings or model, no record of who made each change, and no restore of an earlier state.M1 Instructions and logic PartialGrokThe owner can set preset or custom response styles, project instructions and override built-in skills. The published prompt template only inserts owner preferences into the provider's base prompt, which cannot be replaced.M2 Memory PartialGrokSaved memories can be viewed, edited and deleted in Settings. Personalisation from conversation history and the imported X profile and post history are not shown as individually editable items.M3 Tools PartialGrokThe owner can add any remote MCP server without review and choose which connector tools are exposed. Locally hosted MCP servers are rejected, and no documented setting removes built-in tools such as web or X search.M4 Owner has the final say PartialGrokSome content settings (mature voices, the Spicy Imagine mode) are adjustable behind an 18+ gate, but content filters and the sandbox are provider-controlled and locked.M5 No unilateral change FailGrokMode names such as Fast and Expert resolve to models the provider changes without pinning. The Grok 4.5 and 4.6 switches have no release-note entry, and xAI has acknowledged past prompt changes to Grok on X made without notice.C1 Boundaries FailGrokThe owner can choose which tools a connector exposes. There are no documented limits on whom the agent takes instructions from (for example email triggers) or whom it contacts via Gmail send. Spend was treated as met under G11, because Grok has no payment capability.C2 Approval gates PartialGrokRelease notes mention tool permission buttons for connectors, but there is no documentation that the owner can gate any action class or that every path, including scheduled automations, honours the gate.C3 Immediate halt PartialGrokThe owner can stop the current response, stop individual background tasks from the tray, and pause or delete individual automations. No single control stops all activity.C4 Data serves only the owner FailGrokConversations are used to train models by default (opt-out under 'Improve the Model'), and feedback is used even after opting out. Staff may review content, data may be shared with affiliated companies, legal disclosure comes with no notice commitment, and continued use counts as acceptance of changed terms.C5 Credential custody PartialGrokConnectors are listed at grok.com/connectors and each can be disconnected. Where OAuth tokens are held, whether revocation is immediate, and whether provider-side copies are deleted are not disclosed.C6 Full deletion FailGrokAccount deletion is one action and takes up to 30 days, but data may be kept for 'legal, compliance, or safety purposes', de-identified copies may survive, backups are not addressed, and no confirmation is documented.
LovableP1 Owner-usable round trip FailLovableThe owner can download the project code as a zip (paid plans) or sync it to Git, and export Cloud data as an SQL dump that is emailed when ready, but there is no export of the agent itself and no importer: the Lovable agent runs only on Lovable's hosted service, Git import into Lovable is not supported ('You can only export from Lovable to GitHub'), and running the code elsewhere needs manual hosting, dependency and backend setup.P2 Complete export FailLovableExports cover code (including Supabase migrations and edge functions) and a Cloud database SQL dump; project chat and activity history, project and workspace knowledge, agent permission settings and connector credentials have no export (secrets are write-only and gateway connectors authenticate through Lovable's OAuth client), so two or more classes stay with the provider.P3 Runs without the provider FailLovableThe agent is a hosted service that requires a Lovable account; the Terms grant only a 'limited ... revocable license' that 'terminates immediately if ... your account is terminated for any reason'. Exported app code can run elsewhere, but the agent cannot run without Lovable.P4 Model independence FailLovable'Can I choose which model Lovable uses in Build mode or Plan mode? No. Lovable manages the underlying model used by the agent'; there is no setting to switch models and no local-model option. (Model choice for the owner's app AI features is separate and not the agent's model.)P5 Exit survives termination FailLovableThe Terms state 'On termination, your right to access the Services ends', allow immediate suspension or termination for suspected breach, and give no post-termination export window or shutdown plan; by default no current copy is held by the owner (Git sync and code download are manual/optional).T1 Readable without the provider FailLovableCode is readable as ordinary source files, but the agent's own state (chat and activity record, knowledge, settings, provider-supplied skills) is readable only through Lovable's hosted UI; Lovable-built skills 'cannot be edit[ed], delete[d], or download[ed]', and data is encrypted at rest 'with managed keys' held by the provider.T2 What runs is what you see FailLovableThe owner can see knowledge, skills and activity cards, but not Lovable's base system prompt or the fully assembled context; the model identity is not determinable ('The agent runs on current frontier models' with no setting or disclosure of which). Two shortfalls.T3 No shadow data FailLovableThe Privacy Policy keeps operational and security logs 'for as long as needed to run and secure the Services' and retains de-identified or aggregated data derived from Customer Data 'on a perpetual basis for any lawful business purpose' (Terms); these holdings are not visible item by item and have no bounded period.T4 No hidden steering PartialLovableThe agent defaults to Lovable's own services (Lovable Cloud with 'Always allow' enablement, the Lovable AI gateway with a Lovable-chosen default model, Lovable hosting); these are documented and can be replaced (Cloud set to 'Never allow', own Supabase, own API keys). No paid placement was found, but its absence rests on no binding commitment.A1 Complete action record FailLovableChat activity cards record file edits, commands, web searches, browser tests and subagents, and version history keeps every code version; workspace audit logs exist only on Enterprise (and are pruned after about 90 days). No record of provider or staff actions on the owner's projects is documented, and nothing documents that recording failures are surfaced to the owner (G12). Two shortfalls.A2 Tamper-evident, owner-checkable FailLovableNo integrity data (hashes, signatures or independent anchoring) is documented for the chat/activity record or version history; the record is held only by Lovable.A3 Agent cannot rewrite its record PartialLovableThe chat/activity record and version history are maintained by the Lovable platform, and the agent's documented tools (file edits, commands, database and connector actions) do not include editing the chat record, but Lovable documents no protection guaranteeing the agent cannot alter or suppress its record; protection rests on architecture/convention only.V1 Obtainable source, matching build FailLovableThe Lovable agent and platform are closed source; the Terms define the Platform and 'models' as Lovable Materials in which the owner gets no ownership interest.V2 Owner-rooted identity FailLovableThe agent has no owner-rooted identity: it acts through the owner's Lovable account and, on GitHub, commits under 'the Lovable GitHub app identity', both provider-issued and revocable.V3 Change history and rollback PartialLovableVersion history shows code diffs per version, links each to the chat message that produced it, and restores code in one click; it does not restore database data, very old versions 'can no longer be restored', and changes to knowledge, settings, permissions or the provider-managed model are not versioned or attributable on self-serve plans (audit logs are Enterprise-only).M1 Instructions and logic PartialLovableProject and workspace knowledge and custom workspace skills are owner-editable and persist, but Lovable's base prompt is not exposed and 'Skills built by Lovable' can be used but 'you cannot edit, delete, or download them'.M2 Memory PartialLovableKnowledge (project and workspace, 10,000-character text fields) and the project's security memory document are fully viewable and editable; the project chat history the agent reuses across sessions is not documented as itemised editable or deletable memory.M3 Tools PartialLovableOwners can connect any remote MCP server as a chat connector on all plans without provider review, and Cloud tools and connectors can be set to 'Never allow'; built-in tools such as file editing, commands, web search and browser testing have no documented removal control.M4 Owner has the final say PartialLovableSome provider protections are adjustable (connector approval defaults, Cloud tool permissions), but others are locked: Lovable-managed secrets 'cannot be deleted', secrets are write-only, and VITE_-prefixed secrets are rejected; no documented switch exists for the platform's sandboxing or dependency audit.M5 No unilateral change FailLovable'Lovable rolls out model upgrades automatically to all users, so there is no setting to switch the agent between specific models'; the hosted agent and its instructions change without an opt-in or pin.C1 Boundaries FailLovablePer-connector and per-Cloud-tool permissions (always/ask/never) and credit check-ins exist, but there is no per-domain egress control for web search or fetches and no per-path file scope (shortfall 1), and code the agent writes and deploys as edge functions uses stored secrets to call external services, which the agent can invoke directly, outside connector-level limits (shortfall 3). Two shortfalls. Payment spend was treated as met under G11 (the agent has no payment instrument).C2 Approval gates PartialLovableOwners can gate a fixed set of classes: connector write actions, Cloud tools (read/modify database, change data, configure auth), project creation and edits from Chats, and image/video generation; code edits, commands and web searches cannot be gated except by choosing Plan mode, and Cloud tools default to 'Always allow'.C3 Immediate halt PartialLovableThe stop button 'immediately halt[s] the current task' in a project, and builds started from Chats have a separate 'Stop Lovable' control; there is no single documented control that stops all activity across projects and background runs at once.C4 Data serves only the owner FailLovableOn Free and Pro, 'customer data ... may be used to train, develop, and improve Lovable's AI models ... unless you opt out'; the Privacy Policy states 'trained members of our team may review this content', and the Terms grant a perpetual licence to exploit Customer Data 'for any other lawful business purpose'. Training is on by default for the graded plan.C5 Credential custody PartialLovableProject secrets and connections are listed by name and can be deleted individually, with 'access ends immediately when you delete a connection', and Lovable states no one can read credentials back; however Lovable-managed secrets 'cannot be deleted', and no binding term limits Lovable's use of owner credentials to owner-directed actions.C6 Full deletion FailLovableAccount deletion is one flow that finalises after a 30-day grace period and confirms only that deletion is scheduled; per the docs it 'does not retract content from training datasets assembled, or models trained, before then', so provider-side copies survive.
Mistral VibeP1 Owner-usable round trip FailMistral VibeA self-serve Export button downloads the owner's Vibe data, but no importer or non-Mistral runtime is documented that restores a working agent from the export. Restore works only on Mistral's service.P2 Complete export FailMistral VibeThe terms guarantee only that the export includes 'at least your Vibe conversations (prompts and outputs)'. Other data needs an emailed Switching Request with two months' notice. Nothing documents that Skills, custom and Project instructions, scheduled tasks, Knowledge Base structure or Connector re-linking (OAuth through Mistral-registered apps) are in the export, so under G12 two or more classes stay with the provider.P3 Runs without the provider FailMistral VibeThe Work agent, with its Skills, schedules, Connectors and Knowledge Base, runs only on Mistral's hosted service and needs a Mistral account to start or act. Nothing exported can run with Mistral blocked.P4 Model independence FailMistral VibeWork mode runs on Mistral Medium 3.5, and cloud Code sessions use Mistral Medium 3.5 'regardless of the local model'. No option to use a third-party or local model in the hosted agent is documented. Medium 3.5 weights are open, but the owner cannot point the hosted agent at their own copy.P5 Exit survives termination PartialMistral VibeThe terms say 'You may not be able to export Your Data after your account is terminated'. Advance notice of suspension is given only 'to the extent possible' and is skipped for immediate or unremediable breaches. The only safeguards are the owner's manual self-serve export and that discretionary notice. There is no post-termination export window and no published shutdown plan.T1 Readable without the provider FailMistral VibeThe live agent state (conversations, Knowledge Base, schedules, harness logic) is stored on Mistral's servers under Mistral-held encryption and is readable in full only through the provider's service. The export format is not publicly documented. Skills use the open SKILL.md format.T2 What runs is what you see FailMistral VibeCustom instructions, active Skills, tool calls and Knowledge read/write markers are shown, but the Work harness's base system prompt and any provider-added context are not visible. Provider-side behaviour switches, such as the 'built-in protections' against prompt injection and the harness version, cannot be determined.T3 No shadow data FailMistral VibeAfter deletion, input and output may be kept 'for monitoring and legal retention purposes' for an unstated period. Memories and Knowledge are kept 'for the duration of your subscription' while retention periods are still being implemented, and Code session history is kept 'according to your plan'. Feedback datasets and 'Usage Data' (declared Mistral's property) are held without an owner-visible, bounded retention period.T4 No hidden steering PartialMistral VibeNews search draws from disclosed partners (AFP, AP) and web search can be switched off, and help-centre articles say conversations are not used for ads. The absence of paid placement or self-preferencing rests on non-binding help-centre statements, not the terms.A1 Complete action record FailMistral VibeTool calls, with inputs, outputs and status, and scheduled-task results appear in conversation history until the owner deletes them. Actions by provider staff or provider-side automation are not recorded for the owner (audit logs are Enterprise-only), and nothing documents that the owner is told if recording fails.A2 Tamper-evident, owner-checkable FailMistral VibeNo integrity data (hashes, signatures or checkpoints) is documented for the conversation or action record. Mistral holds the record and can change it without trace.A3 Agent cannot rewrite its record PartialMistral VibeThe transcript is written server-side by Mistral's harness, and no documented agent tool deletes or edits conversation history. No protection is documented, however, so protection rests on convention only (G12).V1 Obtainable source, matching build FailMistral VibeThe hosted Work harness and service backend are closed source. The Vibe CLI is Apache-2.0 and Mistral Medium 3.5 weights are open, but the evaluated hosted agent needs closed components.V2 Owner-rooted identity FailMistral VibeThe agent has no identity separate from the owner's Mistral account. Integrations act through Mistral-registered OAuth apps (for example the Mistral GitHub App), which the provider can revoke.V3 Change history and rollback PartialMistral VibeA 'write pill' in the chat marks when the agent saves a Knowledge Base entry, so some memory changes are visible and attributable. No history exists for instructions, Skills, settings, model or harness version, and no documented feature restores a prior state.M1 Instructions and logic PartialMistral VibeOwners can write custom instructions and Project instructions, and create, edit or disable Skills (built-in Skills can be disabled). The Work harness's base prompt cannot be replaced.M2 Memory PartialMistral VibeKnowledge Base (on by default) lists its entries, but 'You can't edit the text of a knowledge entry directly'. Single entries can be changed or deleted only by asking the model. Legacy Memories are itemised, editable and deletable in the UI.M3 Tools PartialMistral VibeOwners on Free and Pro plans are administrators and can add any MCP server as a custom Connector without Mistral review, and can disconnect Connectors and disable Skills. Removal of every built-in tool is not documented (only web search has a documented toggle; code interpreter, image generation and Canvas have none), so under G12 some built-ins count as non-removable.M4 Owner has the final say FailMistral VibeProtections such as the built-in prompt-injection protections, the Code sandbox egress policy and command timeouts, and the Knowledge Base sensitive-data exclusion are set by Mistral. No owner control to change any of them is documented. Enforcement of the usage policy was not graded, per the test's scope note.M5 No unilateral change FailMistral VibeMistral changes the hosted default model, the harness and features server-side, for example making Medium 3.5 the new default and removing the Chat tab progressively. The owner cannot pin a version. Notice is promised only for updates that 'materially impair' use, and the only remedy is to terminate.C1 Boundaries FailMistral VibeLimits are per Connector and per function (allowed, restricted or blocked), not per recipient, sender or domain. Nothing limits whose instructions the agent follows: Mistral's docs say injected content can redirect it and advise the owner to review. The Stripe Connector can manage payments with no spend cap.C2 Approval gates PartialMistral VibeThe owner can gate each Connector function, and the agent asks before write actions by default. Only Connector functions can be gated, and Vibe Code Web sessions have no approvals for sensitive actions yet (listed as planned).C3 Immediate halt PartialMistral VibeA stop button interrupts the foreground Work task, 'but there's no guarantee'. Scheduled tasks must be paused separately, and Vibe Code Web sessions have no cancel or manual-stop control ('not available yet').C4 Data serves only the owner FailMistral VibeOn non-Enterprise Vibe plans, 'input and output data are used by default to train our artificial intelligence models, unless you opt out'. Feedback is always used for training. The Privacy Policy may be amended without the owner's specific consent.C5 Credential custody PartialMistral VibeConnected services are listed on the Connectors page and can be disconnected one by one ('disconnecting revokes access immediately'), and the terms limit use to requested actions. Nothing documents that Mistral deletes its stored copy of a credential (for example bearer tokens for custom MCP servers) after revocation, so the pass is withheld (G12).C6 Full deletion FailMistral VibeOne Admin-panel action deletes the account. Mistral 'may retain your input and output data to enforce our Usage Policy', with no stated completion limit, and keeps account data for 1 year and identity data for 5 years afterwards, so provider-side copies survive deletion.
GeminiP1 Owner-usable round trip FailGeminiGoogle Takeout exports Gemini chats, uploads and Gems as an archive delivered in hours to days, but no import exists and no runtime outside Google can restore the export into a working agent.P2 Complete export FailGeminiThe Takeout export covers Gemini Apps Activity (chats, generated media, uploads) and Gems. It does not list Instructions for Gemini, memory derived from past chats, scheduled actions, or connected and custom app configuration, so two or more classes of state stay with Google.P3 Runs without the provider FailGeminiGemini runs only as a Google-hosted service that needs a Google Account, and nothing can be exported to run on its own. With Google unreachable, no agent runs.P4 Model independence FailGeminiThe Gemini app offers only Google's Gemini models through its model picker. There is no documented way to use another vendor's model or a local model.P5 Exit survives termination PartialGeminiGoogle says users of disabled accounts 'may' be able to download their data, excluding some violation categories and valid legal requests, and its Terms promise a Takeout opportunity when a service is discontinued. There is no binding post-termination export window of at least 30 days that also covers bans.T1 Readable without the provider PartialGeminiChats, uploads and Gems can be exported via Takeout and read with general tools. Google-supplied instructions and the memory Gemini derives from past chats are not readable outside Google's service.T2 What runs is what you see FailGeminiThe base system prompt and any context Google adds are not visible to the owner, and for past-chat memory the documented check is to ask Gemini whether it used past chats. Server-side behaviour switches cannot be determined from public documentation (G12), which makes two shortfalls.T3 No shadow data FailGeminiChats selected for human review are kept 'for up to three years', are disconnected from the account and are not visible to the owner. Some data is kept 'until you delete your Google Account', and Google's retention page allows unspecified 'extended' retention for business and legal purposes.T4 No hidden steering PartialGeminiGoogle states that Gemini Apps chats are not used to show ads, but this is a help-centre statement, not a binding commitment. Gemini automatically uses Google's own Search, Maps, YouTube, Flights and Hotels; this is disclosed, but the documentation does not show that these defaults can be switched off.A1 Complete action record FailGeminiGemini Apps Activity records prompts, responses and connected-app actions, but auto-deletes after 18 months by default. Public documentation does not show that every path (for example scheduled or provider-side work) is recorded, or that failed recording is reported to the owner (G12).A2 Tamper-evident, owner-checkable FailGeminiGemini Apps Activity is held by Google and no integrity data or verifier is documented, so the owner cannot detect edits or deletions.A3 Agent cannot rewrite its record PartialGeminiThe activity record is written on Google's side, and no Gemini tool that edits Gemini Apps Activity is documented. However, no protection against the agent altering or suppressing its record is documented either, so the pass is withheld (G12).V1 Obtainable source, matching build FailGeminiGemini's service, runtime and models are closed source and hosted by Google.V2 Owner-rooted identity FailGeminiThe agent's only identity is the owner's Google Account, which Google issues and can suspend.V3 Change history and rollback FailGeminiNo change history or rollback is documented for instructions, Gems, memory, connected apps or model version. The activity log records chats, not changes to agent state.M1 Instructions and logic PartialGeminiOwners can add, edit and delete Instructions for Gemini and create and edit their own Gems. Google's base system prompt cannot be replaced, and the documentation does not say premade Gems can be edited.M2 Memory PartialGeminiInstructions for Gemini are itemised, editable and deletable. Memory of past chats is not itemised: to remove something, the owner must delete the chats containing it, or correct Gemini in a chat. This memory feature is documented for Google AI plan subscribers.M3 Tools PartialGeminiOwners who are 18+ in the US with a personal account can add custom MCP servers without Google review and remove them, and can disconnect Connected Apps. Gemini automatically uses public information from Google services such as Search, Maps and YouTube, and the documentation does not show a way to remove these or the other built-in tools.M4 Owner has the final say PartialGeminiSome protections can be changed by the owner (for example, Keep Activity logging can be turned off). Others are locked: manual confirmation of write actions by custom apps is required, and no owner switch for it is documented.M5 No unilateral change FailGeminiGoogle changes the default model server-side (for example, 3.6 Flash 'introduced as default', 2026.07.21) and announces it only in release notes. The owner cannot decline the change or pin a version.C1 Boundaries FailGeminiLimits are on/off per connected app. No per-recipient, per-domain, per-path or per-tool allow or deny lists are documented, and no limit is documented to hold against model output or injected instructions.C2 Approval gates FailGeminiGoogle imposes manual confirmation for custom-app write actions, but the documentation shows no approval gate the owner can set or choose.C3 Immediate halt PartialGeminiA reply can be stopped in the chat. Scheduled actions must be turned off one at a time in settings, and no single global stop is documented.C4 Data serves only the owner FailGeminiKeep Activity is on by default for adults, and while it is on, chats are used to 'develop, and improve its services (including training generative AI models)' and a subset is reviewed by human reviewers. Both are opt-out, not opt-in.C5 Credential custody PartialGeminiConnected Apps and custom MCP apps are listed and can be disconnected or removed one by one. The documentation does not say how quickly revocation takes effect or that Google keeps no copy afterwards, so the pass is withheld (G12).C6 Full deletion FailGeminiDeleting Gemini activity or the Google Account does not remove human-reviewed chats, which are kept for up to three years. Backups can persist for up to 6 months, and some data can be retained for extended periods.
Meta MuseP1 Owner-usable round trip FailMeta MuseThe owner can download Muse data from Settings > Data controls. However, no importer, runtime or host outside Meta's service can restore an agent from that export.P2 Complete export FailMeta MuseThe download is described as covering chats, files and 'other agent information'. Nothing public shows that it includes the action record, schedules, provider-supplied skills and system prompt, stored credentials, or a way to re-link connectors without Meta's service. Under G12 these classes count as missing.P3 Runs without the provider FailMeta MuseThe agent runs only inside a Meta-operated VM, with inference through Meta proxies. The terms say that on termination 'your right to use Muse ceases immediately', so the agent cannot start or act without Meta.P4 Model independence FailMeta MuseMuse is powered by Meta's Muse Spark model. No documentation offers the owner any other model, including a local open-weight model.P5 Exit survives termination FailMeta MuseAll state lives in the Meta-hosted VM. Meta may suspend or terminate access 'at any time, for any reason, without prior notice'. The terms give no post-termination export window and no shutdown plan.T1 Readable without the provider PartialMeta MuseMemory and files are in open formats (for example MEMORY.md in a Linux VM), and the owner can inspect and download them. The export schema, the action record format, the system prompt and the connector SKILLs are undocumented or not visible, and the VM is held by Meta.T2 What runs is what you see FailMeta MuseThe system prompt is not shown; Meta quotes only its first line, and nothing shows the untrusted-input labels or safety-layer insertions being made visible. Provider-controlled behaviour switches, such as the safety classifiers and Meta's reserved right to modify actions, cannot be determined by the owner. That is both shortfalls.T3 No shadow data FailMeta MuseMeta 'may access and retain logs of actions taken by Muse' for safety, service improvement and legal compliance. It also collects activity metadata and telemetry. No retention period for these is disclosed, and the owner cannot view them item by item.T4 No hidden steering PartialMeta MuseNo paid placement is documented. However, Meta says it expects to profit from transaction fees, Stripe Link is the launch wallet, and there are retail partnerships. No binding commitment rules out self-preferencing in recommendations or 'Ideas', so the pass is withheld (G12).A1 Complete action record FailMeta MuseAn activity log gives 'a chronological record of actions your Muse has taken'. Public evidence does not show that it records Meta-initiated interventions, which the terms reserve. It also does not settle whether recording can fail silently, or whether entries keep arguments without pruning. Under G12 that is two or more shortfalls.A2 Tamper-evident, owner-checkable FailMeta MuseNo hashing, signing or other integrity mechanism for the activity log is documented, so the owner has no way to detect edits.A3 Agent cannot rewrite its record PartialMeta MuseDurable application state lives in a Postgres database outside the agent's runtime cell, and Sentinel runs host-side. Nothing documents specifically that the activity log is written from outside the agent's reach, so the pass is withheld.V1 Obtainable source, matching build FailMeta MuseThe Muse runtime (the 'Hatch' daemon, Sentinel, authd), the VM and the clients are closed source, and the terms forbid reverse engineering. The planned Confidential VM source is shared only with auditors.V2 Owner-rooted identity FailMeta MuseThe agent's identity is a Meta/Muse account that Meta may suspend or disable. No owner-rooted key or domain identity is documented; an agent email address is only 'coming soon'.V3 Change history and rollback PartialMeta MuseThe activity log shows approved permissions, and 'VM data is backed up continuously so you can restore it'. No documentation shows attributed diffs of memory, instructions or settings, or an owner-initiated restore to a chosen point.M1 Instructions and logic PartialMeta MuseThe owner can shape behaviour through instructions, memory files, personality and proactivity settings. The Meta-authored system prompt and the connector SKILLs are not documented as replaceable.M2 Memory PartialMeta MuseMEMORY.md and other files can be read and edited directly. However, Muse 'may still remember information it learned from what you deleted', and the Forget skill works only 'to the best of its ability', so at least one store is not fully itemised or controllable.M3 Tools PartialMeta MuseOwners can add custom connectors, and 'Meta doesn't review custom connectors'. Muse can also build its own tools. No documented control removes built-in tools such as the browser or shell. (Listing a connector publicly for all users does need Meta review.)M4 Owner has the final say FailMeta MuseProvider protections are designed so they cannot be disabled: hatch-safety classifiers ('attackers cannot disable these protections'), the email OTP and reset-link filter, the malicious-site blocklist and the runtime sandbox. The terms also prohibit getting Muse to override its safety systems. No owner switch is documented for any of them.M5 No unilateral change FailMeta MuseMeta runs and updates the model and harness server-side (Muse Spark 1.1 to 1.3), with no pinning. The terms reserve the right to 'modify Muse's actions at any time … in Meta's discretion'.C1 Boundaries FailMeta MuseSentinel enforces owner connector policies and per-site web permissions outside the agent's reach. Two shortfalls remain. No spend cap is documented; the terms put 'transaction limits' on the owner to set with financial institutions. Contact limits are per connector and action, not per recipient or sender. The spend-cap finding is recorded despite per-purchase approvals.C2 Approval gates PartialMeta MuseThe owner can choose 'Ask for some actions' or 'Always ask', and approvals are strict capabilities enforced by Sentinel. However, a taint-based auto-allow path exists, Sentinel decides which grant types to offer, and the evidence for this hosted component is non-binding documentation (G6).C3 Immediate halt FailMeta MuseThe documented way to stop work is to 'ask your Muse to undo or stop some actions … in chat', which is a request to the model. No global stop for background, scheduled and subagent work is documented.C4 Data serves only the owner FailMeta MuseTraining on Muse interactions, including connector data, 'is on when you first use Muse' and is opt-out. Meta staff access is limited only 'through operational policies'. Data-term changes take effect on notice if the owner keeps using the service.C5 Credential custody PartialMeta MuseConnector tokens are stored in authd in the owner's VM, the agent never sees them, and each connector can be disconnected. There is no documented itemised view or revoke control for saved website logins, and no statement that credentials are deleted on disconnect.C6 Full deletion FailMeta Muse'Reset Muse' deletes chat history, files and tasks in one action. No completion limit is stated, and nothing says it reaches the action logs Meta retains under the terms or the continuous VM backups. Meta's account-deletion help also says backup copies may remain after 90 days.
ChatGPT AtlasP1 Owner-usable round trip FailChatGPT AtlasThe only self-service export is the ChatGPT account export ('can take up to 7 days to arrive', ZIP of chat history and account data) plus Atlas bookmark HTML export. No importer or non-OpenAI runtime restores the agent from it.P2 Complete export FailChatGPT AtlasOpenAI says browser history and open tabs will not transfer and gives cookie export only 'where available'. The documented export covers chats and 'other relevant account data'. It does not document inclusion of browser memories, agent-mode custom instructions, schedules or provider-supplied logic, so two or more classes of state stay with the provider.P3 Runs without the provider FailChatGPT AtlasAtlas requires signing in to a ChatGPT account, and its assistant and agent run on OpenAI's service. OpenAI states Atlas 'may no longer open, browse, or support browser-based agentic workflows' after the provider-set shutdown date.P4 Model independence FailChatGPT AtlasThe sidebar has a model picker (release notes, 28 Oct 2025), but no documentation describes using a non-OpenAI or local model. The agent is locked to OpenAI's models.P5 Exit survives termination FailChatGPT AtlasOpenAI discontinued Atlas with an 'approx. 30 day wind down period' covering only bookmark export and manual saving of pages. Browser history and tabs would not transfer and Atlas stops working afterwards. The Terms of Use allow suspension or termination for breach or risk and commit to no post-termination export window.T1 Readable without the provider FailChatGPT AtlasMemories, chats, agent history and agent instructions are held on OpenAI's servers and read through ChatGPT. Browser memory summaries are generated server-side. Provider-supplied instructions and agent logic are not published, and no schema is documented for the export.T2 What runs is what you see FailChatGPT AtlasNo documentation lets the owner view the base system prompt or the assembled context for the sidebar or agent. The model used by agent mode, the server-side safeguards (prompt-injection monitoring, site blocklist) and provider flags are not exposed as settings the owner can determine.T3 No shadow data FailChatGPT AtlasBrowser-memory summaries are kept up to 7 days and are not itemised. 'Help improve browsing & search' diagnostic logs are on by default with no retention period stated. The Privacy Policy keeps data of banned accounts and data under legal or security holds without a bounded period, and keeps de-identified content used for training.T4 No hidden steering PartialChatGPT AtlasFirst-party ChatGPT search is the default and the owner can set Google as the default search engine. OpenAI shows ads to Free and Go users, not to the graded Plus tier, and shares information with search and shopping partners. No binding commitment rules out paid or self-preferencing influence on the agent.A1 Complete action record FailChatGPT AtlasThe agent's work is visible only as chat history and screenshots, which the owner can delete. OpenAI documents no per-action record with arguments and targets and no notice when recording fails. For the one tier where documentation exists (Enterprise), it states that Atlas emits no Compliance API or audit logs and that individual agent actions are not logged.A2 Tamper-evident, owner-checkable FailChatGPT AtlasNo integrity data, signing or verifier for any agent record is documented.A3 Agent cannot rewrite its record PartialChatGPT AtlasChat history is stored server-side and is not among the agent's documented tools. The agent cannot run code or reach the file system. However, the logged-in agent operates the owner's browser session, and nothing documented stops it from reaching chatgpt.com history controls. The protection rests on convention, not a documented guarantee.V1 Obtainable source, matching build FailChatGPT AtlasAtlas is a closed-source proprietary browser. OpenAI publishes only an LGPL source bundle per build, and the ChatGPT service the agent needs is closed.V2 Owner-rooted identity FailChatGPT AtlasThe agent's identity is the owner's ChatGPT account, which OpenAI can suspend or terminate under its Terms of Use.V3 Change history and rollback FailChatGPT AtlasNo change history, attribution or rollback is documented for memories, custom instructions, settings or model changes. The owner can only view current state.M1 Instructions and logic PartialChatGPT AtlasOwners can set custom instructions for agent mode (preferred sources, required steps, approval checkpoints) and ChatGPT custom instructions. OpenAI's base system prompt and built-in agent behaviour cannot be replaced.M2 Memory PartialChatGPT AtlasBrowser memories can be viewed and archived in Settings > Personalization but not edited. They are deleted only by deleting the related browsing history. ChatGPT saved memories are managed separately.M3 Tools PartialChatGPT AtlasAtlas supports browser extensions and the agent can use ChatGPT apps from OpenAI's catalogue. The agent's own tool set (navigate, click, type) is fixed, it cannot install extensions, and no way to add owner-built agent tools or remove built-ins is documented in the sources reviewed.M4 Owner has the final say PartialChatGPT AtlasSome protections are owner-adjustable: safe search for result links, page visibility, logged-in or logged-out agent mode, and the browser-memory sensitive-data filter's effect via memories off. Watch mode on sensitive sites, the agent website blocklist and the restrictions on code and downloads are locked.M5 No unilateral change FailChatGPT AtlasAutomatic updates are on and recommended. Agent behaviour changed server-side (e.g. reduced 'laziness', Feb 2026). The Terms let non-adverse changes take effect on posting. OpenAI then discontinued the product outright. The owner cannot pin the model or instructions.C1 Boundaries FailChatGPT AtlasOwners can limit data scope per site (page visibility), choose logged-out agent mode and set site permissions. No owner limit on whose instructions the agent follows (prompt injection is handled only by provider safeguards) or whom it contacts is documented. On the graded Plus plan, website blocking is available only to workspace owners on request.C2 Approval gates PartialChatGPT AtlasThe agent asks for confirmation on high-impact actions the model identifies. Owners can write 'approval checkpoints' only as custom instructions, so the model decides what is gated.C3 Immediate halt PartialChatGPT AtlasThe owner can pause, interrupt or take over the agent in the foreground. Recurring agent tasks are managed separately at chatgpt.com/schedules, and no single global stop is documented.C4 Data serves only the owner FailChatGPT AtlasFor consumer plans, content is used to train models unless the owner opts out, and Atlas chats follow that setting. Browsed-content training is off by default. Diagnostic logs are on by default. Staff may access agent content for abuse, support, legal and model improvement. Privacy Policy changes take effect on posting.C5 Credential custody PartialChatGPT AtlasSaved passwords, passkeys, per-site cookies and site data can be viewed and deleted locally, and logged-in agent mode uses the owner's existing sessions. OpenAI does not state where session data used by the agent is held, that it is used only for owner-directed actions, or that no copy remains after revocation.C6 Full deletion FailChatGPT AtlasAccount deletion is one action with a 30-day limit. OpenAI may retain a limited set of data longer where required or permitted by law, retains data from banned accounts, and does not remove de-identified content already used for training. Atlas browser data is deleted separately, and completion is not documented as confirmed.
DoubaoP1 Owner-usable round trip FailDoubaoThe privacy policy's rights section lists viewing, copying and deleting individual conversations and works, and downloading cloud-drive files. It documents no export of the agent, and no restore on a runtime the provider does not operate.P2 Complete export FailDoubaoNo agent export is documented. Memory, custom skills, connector configuration, task history and settings stay with the provider. The account-deletion agreement tells users to back up data themselves before deletion but offers no bundle to do so.P3 Runs without the provider FailDoubaoThe agent runs only as a provider-hosted service tied to a Doubao account. The User Agreement §2.10 grants a personal, revocable, non-transferable licence, and §2.7 lets the company stop providing the service at any time.P4 Model independence FailDoubaoThe algorithm and model filing lists only Doubao's own models (Doubao LLM, text-to-image, video and music models). No owner choice of model, including local open-weight models, is documented.P5 Exit survives termination FailDoubaoThe terms let the company suspend or terminate service without notice for suspected violations and decline to restore deleted content (§5.1.7, §10.1). The privacy policy §6.6 says data is deleted or anonymised if operations stop. No post-termination export window exists.T1 Readable without the provider FailDoubaoAgent state (conversations, memory, skills, task records, cloud-drive items) is held on provider servers and is readable only through Doubao's closed apps and service. No schema or owner-held keys are documented.T2 What runs is what you see FailDoubaoTwo shortfalls. The base system prompt and injected memory summaries are not viewable. The model version in effect cannot be determined, since only the product-level 'Doubao LLM' is named. Task mode shows execution steps, but not the assembled context.T3 No shadow data FailDoubaoRetention is stated only as 'as long as necessary', with open-ended extensions for legal, financial, audit and dispute purposes. The provider builds a personalised-recommendation profile from inputs and behaviour, and device and app-list logs are collected, none of which the owner can view item by item.T4 No hidden steering PartialDoubaoShopping and group-buy results are supplied by affiliated Douyin e-commerce and local-services platforms. When a Douyin account is bound, Douyin returns personalised search results. This is disclosed, but no switch to exclude affiliate sources from answers is documented. No binding no-paid-placement commitment was found, and the third-party SDK list shows no advertising SDKs.A1 Complete action record FailDoubaoTask mode displays actions and progress, and browser screenshots can be viewed after a task. No durable action log is documented for connector calls, cloud-computer background runs or remote control from other devices. Behaviour when recording fails is not addressed, and the record is conversation history the user can delete. Under G12, at least two shortfalls count as present.A2 Tamper-evident, owner-checkable FailDoubaoNo integrity data, signing or verifier for any action record is documented. The record is held by the provider and can change without trace.A3 Agent cannot rewrite its record PartialDoubaoThe task record is stored server-side, outside the file and browser tools documented for the agent. No protection against the agent altering or suppressing it is documented, so protection rests on architecture convention only.V1 Obtainable source, matching build FailDoubaoThe hosted service and the client apps are closed. The terms forbid reverse engineering (§5.1.2(9)), and only third-party open-source notices are published.V2 Owner-rooted identity FailDoubaoThe agent's identity is the provider-issued Doubao account, which the provider can suspend or permanently close (Account Service Terms §6.3, §6.5).V3 Change history and rollback FailDoubaoNo change history, attribution or rollback is documented for memory, skills, settings or model version. The memory panel shows only current items, and auto-summarised memory refreshes on its own schedule.M1 Instructions and logic PartialDoubaoOwners can create custom skills, use local skills and stop using official skills. The base system prompt and provider instructions are not documented as replaceable.M2 Memory PartialDoubaoMemory can be toggled off. Explicitly requested memories are listed and deletable item by item, and can be changed by chat instruction. Model-summarised memory drawn from chat history is not itemised; it is only described on request, and updates are periodic rather than immediate.M3 Tools PartialDoubaoOwners can add connectors from a marketplace, create custom connectors and skills, and disable or delete them. Removing built-in tools (for example web search, browser operation, image generation) is not documented.M4 Owner has the final say PartialDoubaoThe local sandbox is on by default, and owners can choose a less restrictive authorisation mode ('allow all'). Provider content filtering (User Agreement §2.11, §5.2) and the high-risk confirmation mechanism are fixed, with no owner override documented.M5 No unilateral change FailDoubaoThe company may update, replace or change the software and service at any time, with notice only 'where feasible' (§2.6, §8.2). The model is a hosted service, and no version pinning or opt-in updates are documented.C1 Boundaries FailDoubaoLimits exist for local file scope (project folder and authorisation modes), connector enablement and Feishu permissions. Several dimensions are missing: no per-recipient or per-domain contact limits, no instruction-source limits, and no spend caps, even though shopping and a Douyin Pay wallet are offered. Remote control from other logged-in devices is on/off only.C2 Approval gates PartialDoubaoOwners choose among 'always ask', 'confirm on demand' and 'allow all' modes, and high-risk actions (funds, identity, sensitive accounts) need manual confirmation. The gated classes are fixed, and in 'confirm on demand' a safety model decides what counts as risky.C3 Immediate halt PartialDoubaoOwners can terminate or take over a task at any time. No single global stop covering cloud-computer background tasks, other running tasks and remote-control sessions is documented.C4 Data serves only the owner FailDoubaoFor the personal edition, content data (text, uploads, voice, screen-share frames) is used for analysis and model training by default, with an opt-out switch. The terms grant the company a free, worldwide, sublicensable licence to user content, including for brand promotion. Marketing SMS is on by default, and term changes bind on continued use.C5 Credential custody PartialDoubaoConnector credentials are listed in connector management, can be disabled one by one and are deleted when disabled. The notice says they are used only for the owner's tasks. Cloud-mode login cookies are saved automatically and can be cleared only all at once via a single switch.C6 Full deletion FailDoubaoAccount deletion is available in settings, but only after identity checks, and the provider may halt it during disputes. No completion time limit or confirmation is stated. Data subject to legal retention or kept for finance, audit or dispute purposes survives, stored separately.
ManusP1 Owner-usable round trip FailManusThe only documented export is the Data Backup Tool, which produces encrypted archives that are restored through Manus's own restoration portal (one restore allowed). No documented route restores the agent on a runtime or host Manus does not operate.P2 Complete export FailManusThe Task Data Backup contains tasks, generated files and 'configuration data'. Persistent memory, knowledge-base entries, stored Cloud Browser logins and credentials are not listed. Connectors come back only inside Manus and must be re-enabled or re-authorised through Manus's own OAuth clients. Under G12, two or more classes count as staying with the provider.P3 Runs without the provider FailManusManus is a hosted service that needs a Manus account and credits to start or act. The app licence is 'revocable', and the Terms let the Company suspend or terminate access and modify or terminate any part of the Services without prior notice.P4 Model independence FailManusThe documentation lets the owner pick modes and plans but documents no way to choose the model behind the agent, and no support for local open-weight models. The GPT/Claude/Gemini model list in the docs is for AI features inside websites the owner builds, not for the agent.P5 Exit survives termination FailManusUnder Terms §7, Manus may suspend or terminate immediately for cause, or without cause on notice, and 'may delete Your Content'. No post-termination export window is committed, and no shutdown plan is published. In the August 2026 service change Manus gave a discretionary 12-day backup window (11–23 Aug) before deleting affected data, and a restore that could be done once only. That window was not a binding term and did not cover bans.T1 Readable without the provider FailManusAgent state lives on Manus servers. The backup packages are encrypted archives with an undocumented format that users are told not to modify, and they are readable and restorable only through Manus. The owner holds no keys. A plaintext export exists for team members, but it cannot be used for restore.T2 What runs is what you see FailManusNo documented view shows the fully assembled context, including Manus's base system prompt and any injected content or compaction. The model identity and version behind the agent are not documented. Both shortfalls apply.T3 No shadow data FailManusThe Privacy Policy keeps data 'as long as we need' for business purposes. Data from banned accounts may be retained for an unstated period. Task execution logs are kept for replay and debugging with no stated period. De-identified data is kept to improve models. No item-by-item view of this retained data is documented.T4 No hidden steering PartialManusNo paid placement inside the agent is documented. First-party services such as Manus hosting, Data Sources, Nano Banana Pro and Cloud Computer upsells are presented openly as Manus features. No binding commitment against self-preferencing exists, and no documented switch turns these defaults off.A1 Complete action record FailManusTasks keep a replay history. Automations → Updates lists runs with status and a summary, and Browser Operator says 'all actions are logged'. Nothing documents recording of provider- or staff-initiated actions, or an alert when recording fails. Under G12 both count as shortfalls.A2 Tamper-evident, owner-checkable FailManusNo hash chaining, signing or other integrity data is documented for task history or automation logs. The record is held and served only by Manus.A3 Agent cannot rewrite its record PartialManusTask execution logs are kept server-side by Manus for replay (Privacy Policy), apart from the agent's sandbox. No documented guarantee stops the agent from altering or suppressing its record, so this rests on architecture alone.V1 Obtainable source, matching build FailManusManus is proprietary. The Terms forbid decompiling or reverse engineering the Services, and no source for the required runtime is published.V2 Owner-rooted identity FailManusAgent identities, such as Cue agents with their own email, phone number and wallet, and Mail Manus bot addresses, are issued and held by the provider. They depend on the Manus account, which Manus can revoke.V3 Change history and rollback FailManusCheckpoints exist only for website-builder projects. No change history, attribution or rollback is documented for agent state such as project instructions, skills, knowledge, memory, settings or model. Deleted tasks cannot be recovered.M1 Instructions and logic PartialManusOwners can write project master instructions, and they can create, upload, import and edit file-based Skills. Manus's base system prompt and built-in behaviour cannot be replaced.M2 Memory PartialManusKnowledge-base entries are itemised and deletable, but editing them is not documented. The Privacy Policy also describes a separate 'persistent memory' store ('where memory is enabled'), and no documented item-level view or edit exists for it.M3 Tools PartialManusOwners can add their own tools without provider review, through custom MCP servers and uploaded or GitHub-imported Skills. No documented way removes built-in tools such as the browser, shell or search.M4 Owner has the final say PartialManusSome protections are adjustable: 'Always Allow' for My Computer commands, and switching off login storage in Cloud Browser settings. Others are locked: the cloud sandbox, and an output watermark the Terms say 'cannot be removed'.M5 No unilateral change FailManusManus changes the hosted agent server-side, for example the Manus 2.0 'rebuilt internal system' (Cascade). The Terms reserve the right to modify the Services at any time without prior notice. No pinning of model, instructions or runtime is offered.C1 Boundaries FailManusSome scope controls exist: approved senders for Mail Manus, folder authorisation for My Computer, and per-connector OAuth grants. No limits on whom the agent contacts are documented, and none on spending, even though Cue agents have wallets. At least two shortfalls apply.C2 Approval gates PartialManusOn My Computer, every local command needs approval (Allow Once or Always Allow). This is a fixed class. No owner-set gates are documented for cloud actions such as sending email, posting or purchasing.C3 Immediate halt PartialManusIndividual tasks can be stopped, and each automation can be paused separately. No single control is documented that stops all in-flight, scheduled and background work at once.C4 Data serves only the owner FailManusTerms §5.3 license Your Content to Manus and its affiliates to 'develop and improve the Services'. Personal data is used to create de-identified data 'to develop and improve our models', with opt-out only by email and only in some jurisdictions. Information is shared with marketing partners for targeted advertising unless the user opts out. Policy changes take effect on posting.C5 Credential custody PartialManusConnectors and Cloud Browser logins are listed in settings and can be revoked one by one. Stored credentials can be deleted by asking in chat. Nothing commits to deleting provider-side copies on revocation, and removing a connector 'does not by itself delete content already retained'. Under G12 the pass is withheld.C6 Full deletion FailManusAccount deletion is one in-app action (with an email code), and data is removed 'within 30 days'. Data may be kept longer for legal, fraud or ban reasons, and de-identified content already used for model improvement survives. No confirmation of completion is documented.
Replit AgentP1 Owner-usable round trip FailReplit AgentThe only documented export is 'Download as zip' or GitHub sync of project files. Replit states that the model powering the Agent 'runs on Replit's infrastructure and can't be swapped', and there is no documented way to restore the agent on a runtime Replit does not operate.P2 Complete export FailReplit AgentThe docs state that secrets, the database and deployment configuration are not included when a project is copied. Agent chat history, account memories, routines and connector grants are also not documented as exportable. Connectors use Replit's own registered OAuth app unless you are on Enterprise.P3 Runs without the provider FailReplit AgentThe Agent runs only on Replit's hosted service and needs a Replit account and plan allowance or credits to act. No perpetual right to run it exists.P4 Model independence FailReplit AgentThe Agent's model 'can't be swapped'. Core users choose only from Replit's model list in Power or Max mode, and Free mode is routed automatically. There is no documented way to bring your own key or use a local model for the Agent itself (your own keys apply only to models your app calls).P5 Exit survives termination FailReplit AgentThe ToS lets Replit suspend or terminate accounts 'in its sole discretion' and gives self-serve users no export window after termination. The 30-day migration window appears only in the Enterprise commercial agreement, and agent state lives on Replit's servers.T1 Readable without the provider FailReplit AgentProject files are ordinary files, but the Agent conversation, checkpoints metadata, memories, routines and run history can be read only through Replit's hosted service. No public schema or offline format is documented for them.T2 What runs is what you see FailReplit AgentReplit's base system prompt and any provider-injected context are not visible to the owner (shortfall 1). In Free mode the model is chosen by 'intelligent model routing', and the docs do not let the owner determine which model or version handled a call (shortfall 2).T3 No shadow data FailReplit AgentThe privacy policy says data is deleted or de-identified 'when it is no longer necessary', with no retention table or bounded periods. Usage data, logs and revision history are collected, but none of it is shown to the owner item by item.T4 No hidden steering PartialReplit AgentThe Agent is marketed as integrating Replit's own services (Database, Auth, AI Integrations, publishing). These first-party defaults are disclosed, and no paid placement was found. However, no binding commitment rules out self-preferencing, and there is no documented switch to turn off first-party defaults.A1 Complete action record FailReplit AgentOn Core, the record is the Agent chat, checkpoints and routine run history. There is no documented guarantee that failed recording is surfaced to the owner, and none that connector and MCP calls are recorded with arguments and targets or kept until the owner deletes them. Under G12 both shortfalls count, and audit logs are Enterprise-only.A2 Tamper-evident, owner-checkable FailReplit AgentNo integrity data or verifier for the action record is documented. Even the Enterprise audit log docs do not address tamper evidence.A3 Agent cannot rewrite its record FailReplit AgentNo documented mechanism stops the Agent from altering or suppressing its record. Checkpoints are Git commits in a project where the Agent works with shell access, and the docs describe no protection for that history or for the chat record.V1 Obtainable source, matching build FailReplit AgentThe Agent runtime and its hosted services are proprietary and closed source, so a required component is closed.V2 Owner-rooted identity FailReplit AgentThe agent acts through the owner's Replit account and, for connectors, through Replit's registered OAuth app (custom OAuth apps are Enterprise-only). Both are provider-issued credentials Replit can revoke.V3 Change history and rollback PartialReplit AgentCheckpoints capture project files, conversation context, environment configuration, Agent memory and optionally the database. They appear as Git commits with diffs and one-click rollback. Account-level memories, workspace settings, connectors, model changes and provider updates are not covered, and changes are not attributed beyond Agent checkpoints versus manual commits.M1 Instructions and logic PartialReplit AgentOwners can edit replit.md and add skills (workspace custom instructions are not available on Core). Replit's base Agent prompt is not exposed and cannot be replaced.M2 Memory PartialReplit AgentMemories are opt-in, and user memories can be viewed, edited and disabled through a Memory file in Settings. Project memories are 'retrieved when relevant', but the docs do not show them itemised and editable.M3 Tools PartialReplit AgentOwners can connect any remote MCP server by HTTPS URL with no provider review, and can disconnect connectors. No documented setting removes built-in Agent tools (shell, browser testing, web search, Replit-managed services).M4 Owner has the final say FailReplit AgentThe Agent's sandboxed environment, rate and allowance limits, and platform logging are set by Replit. No documented owner setting changes or disables any of them.M5 No unilateral change FailReplit AgentFree mode picks models by server-side routing, Replit states 'availability can vary', and modes change at Replit's discretion (Economy Mode was discontinued and replaced by Power). No version pinning for the Agent, model or instructions is offered.C1 Boundaries FailReplit AgentNo documented owner limits on instruction sources, or per-recipient or per-channel contact limits, for connectors such as Gmail and Slack. Spend limits exist only as per-run routine budgets. Per-tool and per-path scope is not documented. Two or more dimensions are missing.C2 Approval gates PartialReplit AgentApproval exists only for a fixed set of cases: Plan mode approval before building, confirmation before paid usage, and confirmation for MCP tools that declare it. The owner cannot choose which action classes are gated.C3 Immediate halt PartialReplit Agent'Select Stop in the Agent panel to halt it immediately' stops the foreground turn. Scheduled routines are managed separately (run now or delete, with no documented pause), and there is no single global stop.C4 Data serves only the owner FailReplit AgentReplit uses prompts, instructions, context and responses 'to help develop and improve our models and AI features'. An opt-out is documented only for Pro, Enterprise is excluded, and none is documented for Core. The ToS also reserves access to private apps for 'improving our service', and terms can change without a set notice period.C5 Credential custody PartialReplit AgentSecrets can be viewed and deleted one by one, and connectors can be disconnected via Manage. Connector OAuth tokens are held by Replit under its own OAuth app, with no disclosure that the provider copy is deleted on revocation or that use is limited to owner-directed actions.C6 Full deletion PartialReplit AgentOwners can request account deletion in Settings, which 'removes all of your content'. No completion time limit, completion confirmation or handling of backups is stated, and the privacy policy allows keeping data in de-identified form.
Claude CoworkP1 Owner-usable round trip FailClaude CoworkThe Pro export (Settings > Privacy > Export data) arrives as an emailed link after an unspecified delay, and personal exports cannot be imported into another Claude account. The only documented restore on owner-run infrastructure is the Claude Desktop 3P import wizard, which first needs hand-set managed configuration (3P inference provider plus claudeAiImport.enabled, off by default). Both P1 shortfalls are present.P2 Complete export FailClaude CoworkA personal export never includes project knowledge files or conversation attachments. It covers conversations, projects and memory, and the docs do not say it includes scheduled tasks, installed plugins/skills or connector configuration. Anthropic notes that projects and scheduled tasks do not carry over when moving to Claude Code. Connectors are authorised through Anthropic's server-side connector layer. At least two classes of state stay with the provider.P3 Runs without the provider FailClaude CoworkBy default the agent loop runs on Anthropic servers and needs a paid Anthropic account. Even the 3P desktop mode always downloads the VM workspace bundle and CLI binary from downloads.claude.ai at session start. The Consumer Terms let Anthropic suspend or terminate access at any time, so the right to run is revocable.P4 Model independence FailClaude CoworkThe Pro offering uses Anthropic's Claude models only. The documented gateway mode is described as serving Claude models through an Anthropic Messages API-compatible gateway, and there is no documented route to a local open-weight model.P5 Exit survives termination FailClaude CoworkDefault Cowork state lives in the Claude account on Anthropic servers. The Consumer Terms allow suspension or termination without notice and say that on termination Anthropic 'may at our option delete any Materials or other data associated with your Account'. No post-termination export window or shutdown plan is published.T1 Readable without the provider PartialClaude CoworkConversations, projects and memory can be exported as a zip and memory can be viewed verbatim, but no public schema for the export is published. The Cowork-specific system instructions and server-side components are not readable. The action record lives in Anthropic's backend, readable only via the service or its export.T2 What runs is what you see FailClaude CoworkAnthropic publishes core claude.ai system prompts, but the published list does not mention Cowork, and the Cowork agent prompt, reminders and classifier-layer context are not shown to the owner. Server-side behaviour changes (the workspace auto-updates at 'Setting up Claude's workspace') are not exposed as determinable settings. The model in use is visible in the picker.T3 No shadow data FailClaude CoworkAnthropic keeps trust-and-safety classification scores for up to 7 years on flagged sessions, and the owner cannot see them. It also reserves open-ended retention 'as required by law, to resolve disputes' and for anonymised or de-identified data, which it 'may retain ... longer'.T4 No hidden steering PartialClaude CoworkAnthropic has publicly pledged that Claude will remain ad-free and that responses will not be influenced by advertisers, but this is a blog statement Anthropic says it may revisit, not a binding term. First-party defaults (Anthropic's plugin catalog as default marketplace, built-in browser) are disclosed and replaceable.A1 Complete action record FailClaude CoworkSession transcripts, scheduled-task run history and Dispatch child-task transcripts provide an action record. The docs do not say whether plugin hook executions are recorded or whether the owner is told when recording fails. Under G12 both count as shortfalls; OTel event export is Team/Enterprise only.A2 Tamper-evident, owner-checkable FailClaude CoworkNo hash-chaining, signing or other integrity data for session transcripts is documented, and there is no owner-side verifier.A3 Agent cannot rewrite its record FailClaude CoworkIn the default cloud mode, the agent loop and code execution run in the same per-session sandbox. No mechanism is documented that keeps the agent from altering or suppressing its own record.V1 Obtainable source, matching build FailClaude CoworkRequired components (Claude Desktop, the Cowork cloud agent loop and sandbox infrastructure, the VM workspace bundle) are closed and proprietary. The Consumer Terms reserve all rights in the Services.V2 Owner-rooted identity FailClaude CoworkThe agent has no identity of its own. It acts through the owner's Anthropic account and connector authorisations held by Anthropic, which Anthropic can suspend or revoke.V3 Change history and rollback FailClaude CoworkThere is no documented change log, attribution or rollback for instructions, memory, tools, permissions, settings or runtime version. Memory changes apply immediately without history, and conversation branching is not available in the new experience.M1 Instructions and logic PartialClaude CoworkThe owner can set global, project and folder instructions, upload and toggle skills and plugins, and edit plugin files. The Cowork base system prompt and injected reminders cannot be replaced.M2 Memory PartialClaude CoworkGlobal memory is itemised by topic with edit and delete controls. Project memory and the per-project summary are documented as separate stores, but their itemisation and editing are not documented; the owner can only reset memory wholesale. Under G12 that counts as one store not settled.M3 Tools PartialClaude CoworkPro users can add custom remote MCP connectors, upload plugins and add Git marketplaces without review, and can disable plugins and skills. Some built-ins cannot be removed; for example, the new experience has no web search toggle.M4 Owner has the final say PartialClaude CoworkSome provider protections are owner-adjustable, such as opting in to sensitive-topic memory. Others are locked: deletion prompts in every mode, content classifiers on all untrusted content, the cloud sandbox, and categories never saved to memory.M5 No unilateral change FailClaude CoworkThe Consumer Terms allow manual or automatic software updates 'without advance notice', and the cloud workspace updates itself server-side. Pro owners cannot pin the runtime or decline changes; for example, local execution is being removed on 2026-10-06.C1 Boundaries FailClaude CoworkPro offers folder scoping and per-connector-tool permissions, but network egress is only an on/off toggle for Pro (domain allowlists are Team/Enterprise). There is no control over whom the agent takes instructions from. Egress rules do not apply to web fetch, web search, MCPs or Claude in Chrome, and computer use bypasses the permission checks that gate other tools.C2 Approval gates PartialClaude CoworkThe owner can set each connector tool to Needs approval and use Manual mode, and deletions always require approval. Computer use within an approved app acts 'without the permission checks that gate other Cowork tools', so one path bypasses the gate.C3 Immediate halt PartialClaude CoworkThe owner can stop a task and pause or delete scheduled tasks one by one. There is no documented single control that stops all cloud sessions, scheduled runs and Dispatch child tasks at once.C4 Data serves only the owner FailClaude CoworkThe Consumer Terms state Anthropic may use Materials for training 'unless you opt out'. Content flagged by safety systems, and feedback, is used for training even after opt-out. Term changes bind on continued use without specific consent.C5 Credential custody PartialClaude CoworkConnectors are listed under Customize > Connectors and can be disconnected one by one, and Anthropic discloses that connector tokens are held server-side. The docs do not say that no copy is kept after revocation, so the pass is withheld under G12.C6 Full deletion FailClaude CoworkPro account deletion takes several steps: cancel, wait for the period to end, then delete, and sometimes a support request. Safety-flagged content (2 years), classification scores (7 years) and data already in training runs or trained models survive deletion.
Grok BotP1 Owner-usable round trip FailGrok BotThe documentation describes no export of a Bot (profile, memory, routines, conversation) and no import into any runtime Cursor does not operate; Grok Bot runs only on Cursor-hosted cloud computers. Template sharing produces a link that can only be added to another Grok Bot account.P2 Complete export FailGrok BotNo agent export exists; memory, conversation history, routines, skills, the cloud computer's disk and connector OAuth tokens (kept on Cursor's backend) all remain with the provider. Individual result files can be saved from cards, and Bot duplication/templates omit conversation history and learned memory.P3 Runs without the provider FailGrok BotGrok Bot requires a Cursor account and plan, runs only on Cursor-hosted computers with Cursor-managed models, and the Terms of Service let Anysphere suspend or discontinue access at any time without notice.P4 Model independence FailGrok BotCursor manages model selection; there is no customer-facing model picker and the serving mix can change without a guaranteed vendor set. Owners cannot select any model, local or otherwise.P5 Exit survives termination FailGrok BotAgent state lives in Cursor's cloud; the Terms of Service allow suspension or termination without notice and deletion of the account and its information on payment failure, and advise users to keep their own copies. No post-termination export window or shutdown plan is published; the DPA's 30-day delete-or-return clause covers personal data on written request, not the agent.T1 Readable without the provider FailGrok BotBot memory, conversation history, routines and settings are held on Cursor's backend and reached only through the Grok Bot apps; no storage format or schema is published. Files in /workspace are readable through the Cursor-hosted computer's terminal.T2 What runs is what you see FailGrok BotThe documentation offers no view of the assembled model context (base prompt, injected untrusted-content markers, reviewer inputs), and provider-controlled switches such as gradual feature rollouts cannot be determined by the owner. Model identity is visible only after the fact in usage analytics; enabled plugin tools are listed.T3 No shadow data FailGrok BotThe privacy policy states personal data is kept 'as long as necessary' for business needs without bounded periods, backend retention for deleted Bots 'follows the applicable Cursor terms', and flagged content may be stored by model providers for investigation. No item-by-item view of provider-held data is documented.T4 No hidden steering PartialGrok BotNo paid placement or sponsored tools are documented, and the Marketplace presents first-party-listed plugins. The absence of self-preferencing is not backed by source or a binding commitment, and Cursor-chosen models and defaults cannot be replaced.A1 Complete action record FailGrok BotFor a Pro account the record is the conversation transcript plus routine run history, which keeps only the 20 most recent runs per routine; Action Recording (90-day internal store) is Enterprise-only and off by default. No mechanism is documented that tells the owner when an action goes unrecorded, and provider-side actions (content removal, disabling actions, computer recreation) are not described as recorded for the owner.A2 Tamper-evident, owner-checkable FailGrok BotNo integrity data, hash chaining, signatures or verifier for the action record is documented.A3 Agent cannot rewrite its record PartialGrok BotThe transcript is stored on Cursor's backend, outside the cloud computer the Bot's tools operate on, but no protection of the record is documented. Bots can create, edit and delete routines (automation writes), and deleting a routine is immediate with no undo; these writes are gated only by model-based Auto Review.V1 Obtainable source, matching build FailGrok BotGrok Bot is proprietary: the Terms of Service forbid reverse engineering and derivative works, and the hosted computer and backend are closed Cursor services.V2 Owner-rooted identity FailGrok BotA Bot has no identity of its own; it acts as the signed-in member through a Cursor account and the member's browser sessions, all of which the provider can revoke.V3 Change history and rollback FailGrok BotNo change log or version history of Bot descriptions, memory, skills, settings or model is documented. Reset restores the cloud computer from the last saved snapshot but shows no diff or attribution and does not cover backend state.M1 Instructions and logic PartialGrok BotThe owner can edit each Bot's name and description, write and edit private skills, and edit routine instructions. The provider's base instructions and Auto Review policy are not exposed and cannot be replaced.M2 Memory FailGrok BotBots retain preferences, facts and summaries across sessions, but no UI to list, edit or delete memory items is documented; correction is by telling the Bot in chat. Duplication and account deletion are the only documented bulk operations.M3 Tools PartialGrok BotOwners can install Marketplace plugins, add custom MCP servers (remote HTTPS or command) and enable or disable individual plugin tools without review. The built-in shell, browser and computer-use capabilities have no documented removal setting.M4 Owner has the final say PartialGrok BotOwners can set Auto-review rules and local-execution policy. Other provider-imposed protections have no owner override: the cloud VM sandbox, the takeover requirement for passwords, CAPTCHAs and payments, and write-only secret handling.M5 No unilateral change FailGrok BotCursor manages model selection and the serving mix can change over time without notice; stale computer images are recreated on the fresh image; the Terms of Service let Anysphere modify the service at any time without notice, and Grok Bot Terms updates that are not materially adverse take effect on posting.C1 Boundaries FailGrok BotFor a Pro account, network egress controls are Enterprise-only (default allow-all), there are no per-recipient contact limits, and scoping relies mainly on Auto-review rules evaluated by a model, which the docs say does not review memory writes or most settings changes. Per-plugin tool toggles and local-execution policy exist. Spend was treated as met (G11) because payment steps are handed to the owner.C2 Approval gates PartialGrok BotOwners can write 'Ask first' Auto-review rules and use per-command approval for local execution, but whether an action matches a rule is decided by a review model. The agent changing its own settings is graded in C1.C3 Immediate halt FailGrok BotThe documented stop is a direct 'Stop now' chat message to one Bot; routines must be paused separately per Bot, and closing the app does not stop cloud work. No single control halting all Bots, routines and background work is documented; the organization-wide computer terminate is Enterprise admin only.C4 Data serves only the owner FailGrok BotContent is sent to model providers that Cursor selects and can change, not destinations the owner chooses. Usage data is collected with no documented off switch, flagged inputs may be analysed to improve enforcement, and model providers may store flagged data. The Privacy Policy changes take effect on posting without specific consent. Privacy Mode (no training) is documented as on by default only for Enterprise teams, although the Terms of Service state content is not used for training without explicit agreement.C5 Credential custody PartialGrok BotInstalled plugins are listed and can be uninstalled, and Bot secrets are listed by name with Remove/Replace. Browser sessions and command-line credentials on the shared computer are not itemised, connector tokens are held on Cursor's backend, and deletion of the provider-side token copy on uninstall is not documented.C6 Full deletion PartialGrok BotWhole-agent deletion requires cancelling any subscription and then deleting the entire Cursor account (deleting a Bot leaves computer files and sessions); the help page states all data is removed within 30 days, but completion confirmation and the reach into daily backups and model-provider flagged data are not documented.
Alexa+P1 Owner-usable round trip FailAlexa+No documented export of the agent exists that can be restored anywhere. Alexa.com offers per-chat 'Download full discussion', and Amazon's account-wide 'Request your data' returns a download link, but no importer or non-Amazon runtime accepts either, so restore works only on Amazon-operated systems.P2 Complete export FailAlexa+The available exports cover chat transcripts, attachments and account data. Routines, personality and other settings, learned preferences, derived content (Alexa 'may still retain … text extracted … or summaries' after an attachment is deleted), and integration links are not documented as exportable. Integrations re-link only through Amazon's Alexa account-linking broker.P3 Runs without the provider FailAlexa+Alexa+ runs only as an Amazon cloud service tied to an Amazon account and a Prime or Alexa+ subscription. The Terms let Amazon 'terminate your access to Alexa at our discretion without notice' and 'change, suspend, or discontinue Alexa … at any time without notice'.P4 Model independence FailAlexa+Amazon routes each request among models it selects, 'including Amazon Nova and Anthropic Claude', accessed through Amazon Bedrock. No public source shows owner model choice, bring-your-own model or local model support.P5 Exit survives termination FailAlexa+The Terms allow termination 'at our discretion without notice' and say nothing about data access afterwards. No post-termination export window, owner-held backup or shutdown plan is documented, and state lives only in Amazon's cloud.T1 Readable without the provider FailAlexa+Agent state (history, notes, routines, preferences, action records) is held in Amazon's cloud and readable only through the Alexa app, Alexa.com or Amazon's data-request service. No public schema covers the state, and Amazon holds the keys.T2 What runs is what you see FailAlexa+Both shortfalls apply. The base prompt and any provider-layer context are not viewable. The model serving a request is chosen per request by Amazon's router and is not disclosed to the owner. 'Why did you do that?' gives only a natural-language explanation.T3 No shadow data FailAlexa+Amazon discloses retention options only for voice recordings and transcripts (3 or 18 months, or don't save, with transcripts kept 30 days). It 'may still retain … records of actions Alexa took', attachments, extracted text and summaries with no stated period, and it keeps 'systems trained on your data after your data has been deleted'. The US Privacy Notice has no retention table.T4 No hidden steering FailAlexa+Amazon Ads states that Sponsored Products campaigns are 'automatically integrated into shopping conversations' on Alexa+, giving 'a mix of organic and sponsored options'. It also sells Conversational Entertainment Ads and Sponsored Tiles inside Alexa+ answers. No first-party source documents per-answer labelling as prominent as the answer, or an off switch. Opting out of interest-based ads only stops targeting ('You will still receive ads').A1 Complete action record FailAlexa+Alexa History lists requests and responses, with booking confirmations sent by email. No public source shows a durable action record with arguments and targets across every entry path (Hunches acting automatically, Auto Buy, routines, add-ons). No source says the owner is told when recording fails. Amazon's retained 'records of actions Alexa took' are not shown to the owner.A2 Tamper-evident, owner-checkable FailAlexa+No integrity data or verifier is published for Alexa history or action records. Amazon holds the record and can change or delete it without trace.A3 Agent cannot rewrite its record PartialAlexa+History is written and stored by Amazon's service, and Amazon documents that 'voice requests can't be used to delete your voice history with Alexa+'. Nothing shows that the agent cannot suppress or alter entries, so protection rests on undocumented product convention.V1 Obtainable source, matching build FailAlexa+Alexa+ is a proprietary Amazon cloud service licensed under the Alexa Terms, and no source is published for any required component.V2 Owner-rooted identity FailAlexa+The agent acts under the owner's Amazon account. Bookings and confirmations are tied to 'the email address associated with your Amazon account', and the account can be revoked by Amazon under the Terms. No owner-held identity root is documented.V3 Change history and rollback FailAlexa+No source shows a change history of agent state (routines, settings, memory, model, version) with attribution or restore. History covers conversations, not state changes.M1 Instructions and logic PartialAlexa+Owner layers exist: routines can be created and edited, there are four personality styles, and notes can be stored with Remember This. The base prompt and Amazon's default behaviours are fixed on Amazon's side, and personality changes are limited to preset styles.M2 Memory PartialAlexa+Remember This notes are itemised and can be edited by voice, but in the app they can only be archived. Attachments can be deleted one by one, but derived text or summaries may remain. The learned-preference and shared Rufus/Alexa conversation memory has no documented item-level view. Under G1 these several partial conditions stay partial, because itemised stores exist.M3 Tools PartialAlexa+Owners can link and unlink third-party services from Amazon's catalogue. Adding new tools goes through the Alexa+ add-on and MCP Toolkit, which are 'available to select partners only' and need certification. No source shows a way to remove built-in capabilities.M4 Owner has the final say PartialAlexa+Some protections can be changed, such as the voice code, the explicit-language filter and unlock/disarm opt-ins. Amazon's general content moderation and add-on safety controls cannot be changed by the owner.M5 No unilateral change FailAlexa+The Terms state that 'We may change, suspend, or discontinue Alexa … at any time without notice'. Amazon changes the model routing server-side, and no version or model pinning is offered. Owners can revert Echo devices to the original Alexa, but they cannot pin an Alexa+ version, and the free Alexa.com experience 'can't be deactivated'.C1 Boundaries FailAlexa+At least two shortfalls. (1) Spend is on/off only (voice purchasing on, off, voice code, or recognized voices), with no amount cap. (2) There is no general limit on who may instruct the agent beyond purchases. (3) Some paths bypass the limits: voice-purchasing settings 'may not apply to' purchases paid directly to skill developers, and Hunches and Auto Buy act automatically.C2 Approval gates PartialAlexa+Amazon fixes the gated classes: confirmations for purchases, bookings and payments are required by add-on policy, and the owner can add a voice code or opt in to unlock or disarm. Owners cannot choose to gate other action classes, such as messages or agentic web actions.C3 Immediate halt PartialAlexa+'Alexa, stop' and the microphone-off button end the foreground interaction. Deal tracking, Auto Buy, routines, timers and provider-side tasks each need separate steps, and no single documented stop covers all of them.C4 Data serves only the owner FailAlexa+Alexa interactions are used 'to train Amazon's machine learning models', and 'humans review an extremely small sample'. Amazon documents a setting to manage this, but no Amazon page states it is opt-in. Interactions inform ads unless the owner opts out. The Terms change by posting, with continued use as acceptance. Law-enforcement disclosures include notice unless prohibited.C5 Credential custody PartialAlexa+Linked services and skills are listed and can be unlinked one by one. Developer docs say disabling a skill deletes the stored access token, and Alexa holds tokens rather than passwords. No binding commitment limits Amazon's use of those grants to owner-directed actions. Where each grant is held is not listed in one owner view.C6 Full deletion FailAlexa+Full deletion requires a request to close the entire Amazon account, which must be verified within five days, and no completion limit is stated. Amazon keeps order history, de-identified copies and systems trained on the data, and data already sent to skill developers is not deleted.
Gemini SparkP1 Owner-usable round trip FailGemini SparkSpark is a Google-hosted feature of the Gemini Apps; the only documented exports are per-skill .zip downloads and Google Takeout of account data. No documented route restores a working agent (tasks, schedules, skills, connections) on any runtime Google does not operate.P2 Complete export FailGemini SparkSkills can be downloaded individually as .zip and account data can be exported through Takeout, but no documentation shows schedules and triggers, remote-computer files, remote-browser state, past-chat memory or integration links leaving Google in usable form; Connected Apps and MCP links rely on Google Account linking.P3 Runs without the provider FailGemini SparkSpark runs in Google's cloud ('Spark runs in the cloud instead of on your phone or computer') and requires a signed-in personal Google Account plus a Google AI Pro or Ultra subscription; cancelling the subscription removes access to Spark, tasks and schedules.P4 Model independence FailGemini SparkSpark runs only on Google's Gemini models; the documented model options (Gemini 3 Flash-Lite, Flash, Pro) are all Google-hosted, and no documentation shows third-party or local model support.P5 Exit survives termination PartialGemini SparkAfter subscription cancellation Spark data is retained but paused, and the Terms promise 'reasonable advance notice' and a Takeout opportunity when a service is discontinued, except in urgent situations. Download from a disabled account is 'may be able to', and accounts 'may be disabled without the ability to download data for certain violations'; no 30-day guarantee exists.T1 Readable without the provider FailGemini SparkAgent state (tasks, schedules, remote-computer files, memory, provider instructions) is readable only through Google's Gemini apps and services. Skill files are plain-text SKILL.md, but the main state has no documented open format outside Google's service.T2 What runs is what you see FailGemini SparkGoogle does not publish the base system prompt or provider-injected context that Spark runs with (shortfall 1), and the Spark pages do not document which Gemini model or version a task uses, while the limits page notes 'model names, versions, and availability may change' (shortfall 2). Owners see planned steps, progress, skills, apps and files used.T3 No shadow data FailGemini SparkGoogle says it retains some data 'longer when necessary for legitimate business or legal purposes' with no stated bound, and some data 'until you delete your Google Account'. Human-reviewed chats are kept for up to 3 years, disconnected from the account and not visible item by item.T4 No hidden steering PartialGemini SparkGoogle says Gemini chats 'are not being used to show you ads. If this changes, we will clearly communicate it to you', which is a non-binding statement. Spark uses Google Search services (Flights, Hotels, Maps) and shows 'Trending' tasks and 'Recommended' skills; no documented paid or self-preferencing ranking was found, but the absence is not bound by terms.A1 Complete action record FailGemini SparkThe work panel shows steps and the files Gemini read or updated, and activity lives in Gemini Apps Activity. But Gemini Apps Activity auto-deletes after 18 months by default (pruned by default). Public docs also do not show that entries include arguments and recipients or that failed recording is surfaced (counted as not met under G12).A2 Tamper-evident, owner-checkable FailGemini SparkNo documentation describes integrity data, hash chaining or an independent verifier for Spark's task history or Gemini Apps Activity. Google holds the record and it can be changed without trace from the owner's side.A3 Agent cannot rewrite its record PartialGemini SparkNo documented Spark tool edits Gemini Apps Activity or task threads, and skills cannot be deleted from a chat or task thread. But in the local Chrome browser Spark 'has access to all the same sites that you do, including sites you're signed into', and that path is limited only by model-applied 'site and action restrictions' (policy-level protection).V1 Obtainable source, matching build FailGemini SparkSpark's runtime, the Gemini apps and the Gemini models are closed and Google-hosted. No source is published for any required component.V2 Owner-rooted identity FailGemini SparkThe agent has no identity of its own. It acts as the owner's Google Account, with 'Sign in with Google', Google Account-linked Connected Apps and Password Manager logins, and Google can disable that account.V3 Change history and rollback FailGemini SparkSkills and schedules can be edited and saved, but no documented version history, diff, attribution or rollback exists for skills, schedules, instructions, memory, tools or model changes.M1 Instructions and logic PartialGemini SparkOwners can create, edit, replace and deactivate skills, edit schedule instructions and set 'Instructions for Gemini'. Google's base instructions and built-in behaviour cannot be replaced, and Google notes that owner instructions to avoid topics 'doesn't always work perfectly'.M2 Memory PartialGemini SparkInstructions for Gemini are itemised and editable. Past-chat memory can be managed only by deleting the source chats or correcting Gemini in conversation, and what Gemini has inferred is not itemised ('just ask Gemini'). Remote-computer files and remote-browser data can only be deleted as a whole.M3 Tools PartialGemini SparkOwners can add any MCP server as a custom app in the web app without Google review, and can disconnect Connected Apps. Some built-ins cannot be removed: Gemini 'can always use public information from Google Search', and no documented setting removes the remote browser or remote computer from Spark.M4 Owner has the final say FailGemini SparkProvider protections such as 'prohibited task recognition', site and action restrictions, mandatory take-control for passwords and payments, required confirmation for custom-app write actions, and the no-network rule for skill scripts are set by Google. No documented control lets the owner change them.M5 No unilateral change FailGemini SparkGoogle changes Spark server-side through dated rollouts (new connected apps, monitoring behaviour, notifications). Model names and versions 'may change', Gems are being removed and auto-converted to skills, and no pinning or decline option is documented.C1 Boundaries FailGemini SparkLimits are mostly on/off: per-app connections, per-device autobrowse and per-folder Mac access. No documented control limits whom the agent takes instructions from (content such as emails and websites can inject instructions), whom it contacts, or how much it spends; its web purchases can use saved Password Manager logins. Site scope is chosen by the model.C2 Approval gates FailGemini SparkConfirmation prompts cover a fixed set of classes that Google defines, and Gemini is 'designed to ask' for them. Owners cannot choose which classes need approval, and Google Tasks bulk actions proceed without confirmation. Google also warns that offline schedules may complete unintended actions.C3 Immediate halt UnverifiedGemini SparkPer-task Stop and 'Turn off Gemini Spark' exist, and turning Spark off pauses all schedules and tasks until the owner turns it back on. No documented guarantee covers how fast in-flight provider-side work stops. Stopping a local-browser run may lead Gemini to continue via the remote browser, and after subscription cancellation 'in-progress Spark tasks will finish'. Per the C3 note this is Unverified.C4 Data serves only the owner FailGemini SparkSpark requires Keep Activity to be on, and with it on 'Google uses your activity to provide, develop, and improve its services (including training generative AI models)' with human reviewers. So content use for training and staff review is on by default and cannot be turned off while using Spark. Data-term changes follow Terms updates on notice, not specific consent.C5 Credential custody PartialGemini SparkConnected Apps and custom MCP servers are listed and can be disconnected or unlinked one by one. Remote-browser cookies holding website sign-ins are neither itemised nor revocable per site ('delete all saved browser data'), and Google says this data is used 'consistently with the Gemini Apps Privacy Notice', which does not limit use to owner-directed actions.C6 Full deletion FailGemini SparkDeleting the agent takes several steps: task threads, schedules, skills, remote browser and remote computer data are deleted separately, and turning Spark off deletes remote data only. Human-reviewed chats 'are not deleted when you delete your activity' and are kept for up to 3 years. Google-wide deletion 'generally takes around 2 months', and backups can last up to 6 months.
Perplexity CometP1 Owner-usable round trip FailPerplexity CometPerplexity documents no export and import of the agent. Memory, threads and connectors live in the Perplexity account, and no documented route restores them on a runtime Perplexity does not operate. Local deletion and account deletion are documented; export is not.P2 Complete export FailPerplexity CometThere is no documented agent export. Memories, search history and threads, and connector grants stay in the Perplexity account (at least three classes of state). Only local browser data sits on the device.P3 Runs without the provider FailPerplexity CometComet Assistant sends requests to Perplexity servers to act, so it needs Perplexity's service to work. The Comet licence 'automatically terminates if we terminate your right to use Comet', and the General Terms let Perplexity suspend or delete accounts 'for any or no reason'.P4 Model independence FailPerplexity CometThe documented model choice is limited to models Perplexity hosts and selects for Comet. No documentation covers bring-your-own-key or local open-weight models for Comet Assistant. Detail on the Comet model picker came only from third-party reviews, which were not used for grading.P5 Exit survives termination FailPerplexity CometPerplexity 'may suspend, disable, or delete your Account ... with or without notice, for any or no reason' and 'may, but is not obligated to, delete any of Your Content'. No post-termination export window or shutdown plan is published. Only local browsing data stays on the device.T1 Readable without the provider FailPerplexity CometThe main agent state (memories, threads, temporary Assistant threads, connector grants) sits in the Perplexity account and can only be read through Perplexity's service. No schema or local copy is documented. Perplexity does not document the format of local browser data either.T2 What runs is what you see FailPerplexity CometThe base system prompt and the context Comet assembles (which page content and history items it sends) cannot be viewed. Nothing documents which model the default (free) Assistant uses, and provider-side switches such as the 'personal query' classification are not visible. Both shortfalls apply.T3 No shadow data FailPerplexity CometSome retention periods are bounded: personal Assistant context up to 30 days, logs of cleared memories up to 30 days, files 30 days. The Privacy Notice's general rule, though, is to keep data 'for only as long as necessary ... unless a longer retention period is required or permitted by law', which leaves retention undisclosed or unbounded for other data classes, including device data, usage tracking and inferences.T4 No hidden steering PartialPerplexity CometPerplexity is Comet's default search engine, and this is disclosed. A help article titled 'Can I change the default Search Engine on Comet?' is listed but could not be opened. The Privacy Notice keeps purposes such as 'Market the products or services of our business partners' and advertising-partner sharing, and no binding commitment rules out paid or self-preferencing influence on Assistant answers or shopping. No primary source shows undisclosed paid steering.A1 Complete action record FailPerplexity CometThe Assistant sidecar shows steps live, but no durable action record is documented. 'Agent actions ... are not stored in memory'. Assistant threads that use page, history or tab context are 'Temporary threads' deleted after 30 days by default. Recording of background or email assistants, and what happens when recording fails, are not documented.A2 Tamper-evident, owner-checkable FailPerplexity CometNo integrity data, hash chain, signature or verifier is documented for any Assistant activity record.A3 Agent cannot rewrite its record FailPerplexity CometNo protection is documented. The Assistant can navigate and act on websites, including perplexity.ai where threads are deleted, unless the owner blocks that site. Nothing states that the record is out of the agent's reach.V1 Obtainable source, matching build FailPerplexity CometComet is proprietary: the licence forbids decompiling, modifying or creating derivative works, apart from open-source components listed at comet://credits. The Assistant's server side is closed. The perplexityai GitHub organisation lists no Comet source.V2 Owner-rooted identity FailPerplexity CometThe Assistant has no identity of its own. It acts through the owner's Perplexity account and the owner's browser sessions, and Perplexity can suspend or delete that account.V3 Change history and rollback FailPerplexity CometNo change history or rollback is documented for memory, settings, permissions, model or instructions. Memories can only be viewed and deleted, and referenced memories are cited in answers, but changes are neither logged nor restorable.M1 Instructions and logic FailPerplexity CometThe base instructions are set on Perplexity's side, and no documentation offers a way to replace them or any default workflow. The sources opened do not document owner instruction layers for Comet Assistant, so none were credited (G12).M2 Memory PartialPerplexity CometSaved memories are itemised, searchable and deletable one by one, and both memory and search-history referencing can be switched off. Memory items are not documented as editable (only deletion is), and search-history threads can only be deleted.M3 Tools PartialPerplexity CometThe owner can turn the Assistant off or block it on specific websites, and can connect services from Perplexity's connector catalogue. Custom remote MCP connectors are documented only in the Enterprise help collection and are not shown to be available to a one-person consumer account. Individual built-in Assistant actions cannot be removed.M4 Owner has the final say PartialPerplexity CometSome protections can be changed: ad and tracker blocking has per-site exclusions, and the first-run agent prompt offers 'Always allow'. The pause-for-permission safeguard before logins and purchases, and the 'personal query' handling, are not documented as switchable, so they are treated as locked.M5 No unilateral change FailPerplexity CometUnder the Comet Terms, 'The Software may periodically check for updates and download and install them for you ... without any additional notice'. Models, prompts and Assistant behaviour are served by Perplexity and change server-side, with no documented pinning.C1 Boundaries FailPerplexity CometPer-site blocking limits where the Assistant acts or reads context (data and site scope). No control documents whom the Assistant takes instructions from (for example page content), and there are no per-recipient contact limits. At least two dimensions are missing. Spend was not graded as a separate dimension because Perplexity says Comet never accesses stored payment data.C2 Approval gates PartialPerplexity CometThe Assistant pauses for permission 'when [it] recognizes that a task is important', such as logins or purchases, and asks before first running an agent. The model decides what counts as gated, and the owner cannot gate arbitrary action classes.C3 Immediate halt PartialPerplexity CometThe Assistant sidecar has 'clear buttons to stop the Assistant', which halts the foreground task. No single control is documented that also stops Background Assistants, Email Assistant or other provider-side work.C4 Data serves only the owner FailPerplexity CometSearch data and memories are used to improve Perplexity's models by default until the owner turns off 'AI Data Retention/Usage'. The Privacy Notice also lists sharing browsing and commercial data with advertising partners, and legal disclosure without a notification commitment.C5 Credential custody PartialPerplexity CometPasswords stay in the OS vault and can be viewed and deleted locally. Connector grants such as Gmail are opt-in and 'may [be] revoke[d] or modif[ied] at any time within Perplexity settings'. No documentation says Perplexity keeps no copy of connector tokens after revocation, so the pass is withheld.C6 Full deletion FailPerplexity CometAccount self-deletion is available and completes after 30 days, with local browsing data deleted separately. The Privacy Notice says Perplexity 'must keep your personal data to comply with legal obligations, resolve disputes, enforce agreements, or for other business purposes' and may keep historical data in backups, so some provider-side copies can survive. Completion is not documented as confirmed.
ClaudeP1 Owner-usable round trip FailClaudeSelf-service export is delivered as an emailed download link after processing, with the link expiring after 24 hours. No importer or non-Anthropic runtime is documented that restores the export as a working agent.P2 Complete export FailClaudeThe export is documented as 'conversation data and the user data for your account', and all memory data is included. It is not documented whether project instructions and knowledge files, styles, preferences or connector configuration are included, so under G12 two or more classes are treated as staying with the provider. Connectors re-link through Anthropic's cloud.P3 Runs without the provider FailClaudeThe agent runs only on Anthropic's hosted service and needs an Anthropic account. The Consumer Terms reserve the right to 'modify, suspend, or discontinue the Services or your access ... at any time without notice'.P4 Model independence FailClaudeModel choice is limited to the Anthropic models in the selector next to the send button. No documented way to use a third-party or local open-weight model.P5 Exit survives termination FailClaudeThe Consumer Terms allow Anthropic to suspend or terminate access 'at any time without notice' for suspected breach and 'at our option delete any Materials or other data associated with your Account'. No post-termination export window or shutdown plan is published, and state is not held locally by default.T1 Readable without the provider FailClaudeAgent state lives on Anthropic's hosted service and is read through claude.ai. The provider holds any encryption keys. The export file format and schema are not publicly documented.T2 What runs is what you see FailClaudeAnthropic publishes the core claude.ai system prompt per model, and the selected model is shown. However, the published prompt lists classifier-triggered reminders (image_reminder, cyber_warning, system_warning, ethics_reminder, ip_reminder, long_conversation_reminder) that the owner cannot see when injected, and tool definitions are not published. Whether a provider switch such as a classifier is active cannot be determined. Two shortfalls.T3 No shadow data FailClaudeActive chats are kept 'for as long as reasonably necessary', which is not a bounded period. Safety-flagged inputs and outputs are kept up to 2 years and trust and safety classification scores up to 7 years, and neither is visible to the owner item by item. De-identified training data is kept up to 5 years.T4 No hidden steering PartialClaudeAnthropic has stated 'Claude will remain ad-free' and that responses will not be 'influenced by advertisers or include third-party product placements'. This is a news post, not binding terms, so absence of steering rests on a non-binding statement.A1 Complete action record FailClaudeConversation history is the only record of actions, and it is kept until the owner deletes it. No documentation shows that provider-side or staff actions are recorded, or that the owner is told when recording fails, so under G12 two shortfalls are counted.A2 Tamper-evident, owner-checkable FailClaudeNo integrity data, signing or verifier is documented for the conversation or action record. The record holder can change it without trace.A3 Agent cannot rewrite its record PartialClaudeNo documented tool lets Claude delete or edit chat transcripts, but Anthropic documents no protection that stops the agent from altering or suppressing its record. Under G12 the pass is withheld.V1 Obtainable source, matching build FailClaudeThe claude.ai service and apps are closed source and hosted by Anthropic. A required component is closed.V2 Owner-rooted identity FailClaudeThe agent's only identity is the owner's Anthropic account, which the provider can suspend or terminate under the Consumer Terms.V3 Change history and rollback FailClaudeMemory can be edited, but no change history, attribution or restore to a prior state is documented for memory, instructions, settings or model.M1 Instructions and logic PartialClaudeThe owner can add user preferences, styles and project instructions. The base system prompt and injected reminders are set by Anthropic and 'periodically updated', and the owner cannot replace them.M2 Memory PartialClaudeMemory topics are listed under Settings > Memory with edit and delete controls. Past chats used through chat search can be deleted but not edited, and editing of per-project summaries is not documented.M3 Tools PartialClaudePro and Max users can add custom remote MCP connectors without Anthropic review and remove them. The docs do not establish that every built-in tool can be removed.M4 Owner has the final say FailClaudeProvider protections such as safety classifiers, classifier-triggered reminders and usage limits are applied by Anthropic. No owner setting documents a way to change or disable them. The hosted-policy exemption does not apply because no exit route exists (P1, P3).M5 No unilateral change FailClaudeThe claude.ai system prompt 'is periodically updated' without owner consent, and the Terms reserve the right to modify the Services 'at any time without notice'. The documented 60-day model retirement notice applies to Anthropic's API platforms, not to the claude.ai model selector.C1 Boundaries FailClaudeThe owner can turn connectors on or off per conversation and set a conversation-wide tool-loading mode. There are no per-recipient, per-domain or instruction-source limits for individual plans (org-level permissions are Team/Enterprise only). Spend is treated as met under G11 because no payment capability exists. Two shortfalls: on/off-only limits and a missing instruction-source dimension.C2 Approval gates FailClaudeOwner-set 'Always allow / Needs approval / Blocked' permissions are documented only for Team and Enterprise owners. The opened docs describe no owner-set approval gates for individual plans.C3 Immediate halt UnverifiedClaudeNo opened primary source documents a global stop covering in-flight, background (for example research) and provider-side work.C4 Data serves only the owner FailClaudeThe Privacy Policy states 'We may use your Inputs and Outputs to train and improve Anthropic AI models, unless you opt out', so training is on by default. Flagged conversations and feedback are used for training even after opt-out. Legal disclosure carries no owner-notice commitment, and policy changes need only notification.C5 Credential custody PartialClaudeConnected services are listed under Customize > Connectors and can be removed one at a time. Where OAuth tokens are held, and whether provider copies are deleted on revocation, is not documented, so under G12 the pass is withheld.C6 Full deletion FailClaudePro users must cancel and wait for the billing period to end before deleting the account, and some cases need contact with support. Safety-flagged data (up to 2 years), classification scores (up to 7 years) and de-identified training data (up to 5 years) are retained, so provider-side copies survive deletion.
Microsoft CopilotP1 Owner-usable round trip FailMicrosoft CopilotThe privacy dashboard offers 'Export all activity history' as a CSV download of prompts and responses, but there is no documented import or restore, in Copilot or in any runtime Microsoft does not operate.P2 Complete export FailMicrosoft CopilotThe documented export is conversation text only: a CSV of 'the things you type in Copilot and the AI-assisted responses you receive'. No export is documented for memories, uploaded files, connector links or settings.P3 Runs without the provider FailMicrosoft CopilotCopilot runs only as a Microsoft-hosted service and needs a Microsoft account. The Services Agreement lets Microsoft remove features or stop providing a service, and the Terms allow access to be revoked 'in our sole discretion, at any time and without notice'.P4 Model independence FailMicrosoft CopilotThe owner picks only preset modes (Quick response, Think Deeper, Study and learn, Smart, Search), backed by Microsoft and OpenAI models. Users cannot select specific model versions, and no other or local models are supported.P5 Exit survives termination FailMicrosoft CopilotMicrosoft may 'limit, suspend, or permanently revoke' access at its sole discretion without notice. When an account is closed 'by you or us', access stops immediately, and Microsoft 'won't be able to retrieve Your Content or Data once your account is closed'. No post-termination export window or shutdown plan is documented.T1 Readable without the provider FailMicrosoft CopilotAgent state (conversations, memories, files) is held on Microsoft servers and read through Copilot or the Microsoft privacy dashboard, both operated by Microsoft. There is no public schema, and the owner holds no keys.T2 What runs is what you see FailMicrosoft CopilotBoth shortfalls. (1) The base system prompt and other provider-injected context are not shown to the owner. (2) The model in effect cannot be determined, because Smart mode uses 'a real-time router' to pick models and specific versions cannot be selected.T3 No shadow data FailMicrosoft CopilotConversation history has a disclosed 18-month retention. However, the Privacy Statement says Copilot data is also used for advertising and product improvement, gives no Copilot-specific retention, and applies only general criteria (purpose, sensitivity, legal obligations). Advertising data derived from conversations is not visible item by item.T4 No hidden steering PartialMicrosoft CopilotAdvertiser content is placed in the conversation below Copilot's answer, introduced by an 'ad voice' lead-in and chosen from 'the context of the entire conversation'. Sponsored links are stated to be clearly identified. On the free plan, however, ads cannot be switched off (the Terms say Copilot 'may include advertising' without a paid Microsoft 365 subscription), and the statement that commissions do not influence rankings is on a support page, not in binding terms.A1 Complete action record FailMicrosoft CopilotThe only record is the conversation history, which is pruned after 18 months by default. No record is documented for provider-side or staff actions (for example human review), or for alerting the owner when recording fails. That is at least two shortfalls.A2 Tamper-evident, owner-checkable FailMicrosoft CopilotNo integrity data (hashes, signatures or checkpoints) is documented for the conversation or action record, so Microsoft as the record holder could change it without trace.A3 Agent cannot rewrite its record PartialMicrosoft CopilotThe history is written by Microsoft's service, and no documented Copilot capability lets the assistant delete or edit it. However, no protection against the agent altering its record is documented, so the pass is withheld under G12.V1 Obtainable source, matching build FailMicrosoft CopilotEvery required component (the Copilot service, its orchestration and models) is closed and hosted by Microsoft. The Services Agreement also forbids using the AI services 'to discover any underlying components of the models, algorithms, and systems'.V2 Owner-rooted identity FailMicrosoft CopilotThe agent has no identity of its own. It acts under the owner's Microsoft account, which Microsoft issues and can suspend or close.V3 Change history and rollback FailMicrosoft CopilotNo change history, attribution or rollback is documented for memories, settings, model or instructions. Memories can only be viewed, edited or deleted in their current state.M1 Instructions and logic FailMicrosoft CopilotThe consumer Copilot base instructions are fixed by Microsoft and cannot be viewed or replaced. The documented custom-instructions feature is for work/school Microsoft 365 Copilot Chat, not the consumer service.M2 Memory PartialMicrosoft CopilotRemembered items can be viewed, edited and deleted one by one, and 'Delete all Memory' is available. A second persistent store, 'Microsoft usage data' from Bing, MSN, Edge and other products (on by default), is only an on/off switch and is not itemised.M3 Tools FailMicrosoft CopilotTools are fixed. Connectors are limited to Microsoft's pre-integrated services (OneDrive, Outlook.com, Google Drive, Gmail, Calendar, Contacts), with no documented way to add custom tools or MCP servers.M4 Owner has the final say FailMicrosoft CopilotMicrosoft sets which protections apply, such as content safety and the built-in Copilot Tasks safeguards, which users 'cannot opt out of'. No owner-adjustable protection is documented. Because no exit route exists (criterion P), hosted-policy enforcement is not excluded from this test.M5 No unilateral change FailMicrosoft CopilotModels are routed and changed server-side (for example, Smart mode's real-time router), with no pinning or opt-in. Microsoft reserves the right to 'remove or change features or functionality' of the service.C1 Boundaries FailMicrosoft CopilotTwo or more shortfalls. Connector scope is on/off per service only, not per folder, recipient or sender. The spend dimension has no documented owner limit even though Copilot Checkout lets U.S. users buy on Copilot.com. No limit on instruction sources, such as injected web content, is documented.C2 Approval gates FailMicrosoft CopilotThe only documented approvals are fixed provider-chosen safeguards (purchases, submitting personal data, messages, account changes) in Copilot Tasks, a Microsoft 365 Premium feature. The owner cannot choose which action classes are gated.C3 Immediate halt PartialMicrosoft CopilotScheduled and recurring Copilot Tasks can be paused or stopped task by task in the Tasks view (Premium). No single global stop covering in-flight, scheduled and provider-side work is documented, and there is no documented guarantee that nothing resumes. Pass withheld under G12.C4 Data serves only the owner FailMicrosoft CopilotTraining on text and voice conversations is on by default and is opt-out only. Personalized ads are on by default. Opting out of training does not exclude conversations from 'advertising, digital safety, security, and compliance' use. 'Opt-out of human review is not available.'C5 Credential custody PartialMicrosoft CopilotConnected services are listed and can be switched on or off one by one, and connected data is stated not to be used for training. Where the authorization tokens are held, how quickly revocation takes effect, and whether Microsoft keeps a copy afterwards are not disclosed, so the pass is withheld (G12).C6 Full deletion PartialMicrosoft CopilotDeletion takes several steps: 'Delete all activity history' and 'Delete all Memory' are separate controls, and full removal requires closing the Microsoft account. Closure deletes data only after a 30- or 60-day reopen window, and completion confirmation and backup deletion are not documented.
GensparkP1 Owner-usable round trip FailGensparkNo documented export of the agent (projects, custom agents, workflows, skills, SecondBrain) exists, and nothing can restore it outside Genspark. The help centre offers only per-file downloads and advises exporting conversation history by hand before deleting a Custom Agent.P2 Complete export FailGensparkWith no agent export, instructions, memory (SecondBrain), workflow definitions and triggers, action history and connector grants all stay with the provider. Only individual files and a privacy-law access request by email are available.P3 Runs without the provider FailGensparkThe agent runs only on Genspark's servers and needs a Genspark account and credits. The terms grant a 'non-exclusive, non-transferable, non-sublicensable, revocable license' and reserve the right to terminate access at any time for any reason.P4 Model independence FailGensparkOwners choose only among models Genspark hosts or proxies (a 40+ model picker in AI Chat; tier selection in Super Agent). No documented way exists to connect an owner-chosen endpoint or a local open-weight model.P5 Exit survives termination FailGensparkThe provider may suspend or terminate access 'with or without notice', and no post-termination export window or shutdown plan is published. State is not held locally by the owner.T1 Readable without the provider FailGensparkMain agent state (projects, SecondBrain memory, workflows, custom-agent configuration) is readable only through Genspark's web service, and no storage format or schema is published. Only owner-uploaded skill files (.md/.zip) are in an open format.T2 What runs is what you see FailGensparkTwo shortfalls. The Super Agent base prompt and injected SecondBrain/Hub context are not shown to the owner. Material settings are not fully determinable: the Super Agent 'chooses the right tools' automatically, and model selection is by tier or 'auto-mix'. Execution step cards show operations, not the assembled context.T3 No shadow data FailGensparkThe Privacy Policy keeps account data 'as long as your account remains active' and other personal data 'for other legitimate business purposes' with no bound. Usage data goes to analytics vendors and is not visible item by item. The terms state 'we may retain Content'.T4 No hidden steering PartialGensparkNo evidence of paid placement was found. Genspark's first-party tools and a curated skills library are visibly defaults, but no binding commitment rules out paid or self-preferencing influence on tool or model choice (G12).A1 Complete action record FailGensparkProject history shows execution step cards, and workflows have a run history. Nothing documents that recording failures are surfaced to the owner, or that entries capture full arguments and targets on every path, including background SecondBrain sync. Under G12 that is two or more shortfalls.A2 Tamper-evident, owner-checkable FailGensparkNo integrity data (hashing, signing or external anchoring) for the action record is documented. The provider holds the record and could change it without trace.A3 Agent cannot rewrite its record UnverifiedGensparkNo public documentation covers whether the agent can alter or suppress its own action record.V1 Obtainable source, matching build FailGensparkThe runtime is a closed, provider-hosted service with no published source.V2 Owner-rooted identity FailGensparkThe agent has no identity apart from the owner's Genspark account, which the provider can suspend or terminate. Outbound actions use the owner's connected third-party accounts through Genspark-held OAuth grants.V3 Change history and rollback FailGensparkNo change history or rollback exists for instructions, custom agents, skills, settings or memory. Skills cannot be edited online and must be deleted and re-uploaded. Only AI Drive files have a trash restore.M1 Instructions and logic PartialGensparkOwners can write Hub custom instructions, Custom Agent instructions and their own skill files. The Super Agent base prompt cannot be replaced, and Community Skills cannot be edited.M2 Memory FailGensparkSecondBrain builds a 'personal context layer' automatically from connected sources, and GenMail builds a profile from email history. Neither is documented as itemised, editable memory: GenMail inferences can only be corrected, and sources can only be disconnected.M3 Tools PartialGensparkOwners can add tools through custom MCP servers, API-key connectors and uploaded skills without provider review; only Team-published skills go through admin review. No documented way exists to remove built-in Genspark tools such as web search.M4 Owner has the final say FailGensparkProvider-imposed protections (safety/NSFW filtering, usage rate windows) are applied by Genspark, and the owner cannot change any of them. The hosted-policy exemption does not apply because no exit exists (P1–P5).M5 No unilateral change FailGensparkGenspark retires models at its discretion ('Older models are retired...') and rolls out product changes server-side. It may amend the terms 'with or without prior notice'. There is no version pinning.C1 Boundaries FailGensparkConnector scopes are mostly 'fixed request' with no scope checkboxes, and read-only narrowing is an organisation feature. No per-recipient or per-domain contact limits exist for the Super Agent. The help centre warns that injected content may redirect behaviour. That is two or more shortfalls.C2 Approval gates PartialGensparkWorkflow runs can pause at 'Pending Confirmation' steps. There is no documented owner setting for choosing which action classes need approval across the Super Agent and workflows.C3 Immediate halt UnverifiedGensparkWorkflows can be disabled one by one. No documented single control stops all in-flight, scheduled and background activity, including SecondBrain sync, within seconds.C4 Data serves only the owner FailGensparkBy default, the terms license user content for 'operating, promoting, and improving our Service, and to develop new ones'. The privacy policy uses data for analysis and marketing, with tracking cookies and Google Analytics. Team/Enterprise accounts, not individual ones, are 'automatically opted out of model training'. Policy changes take effect by posting.C5 Credential custody PartialGensparkConnectors are listed and can be disconnected individually, including per mailbox. Disconnecting does not always revoke the provider grant (MCP credentials 'may need to be revoked separately'), and deletion of Genspark-held copies after revocation is not stated.C6 Full deletion PartialGenspark'Delete User' in Settings deletes the account after a popup and email confirmation, and account data is deleted within 30 days. Completion is not confirmed: the help centre says to check that login fails. The terms reserve that 'we may retain Content'.
Meta AIP1 Owner-usable round trip FailMeta AIMeta offers a data export (Settings > Data & privacy > Download your information) that may take up to 30 days. There is no documented import or restore of the agent on any runtime or host that Meta does not operate.P2 Complete export FailMeta AIThe export covers Meta AI app, meta.ai, Vibes and glasses data, excluding deleted content. Several classes of state stay with Meta: base instructions, model and logic, connector grants (Google Workspace connects through Meta's OAuth client), scheduled-task definitions, and the personalization signals Meta draws from other Meta products.P3 Runs without the provider FailMeta AIMeta AI is a hosted service that needs a Meta account and Meta's servers. Under the Terms, Meta may suspend or permanently disable access or delete the account.P4 Model independence FailMeta AIMeta AI runs on Meta's own Muse Spark model; the announcement offers Instant and Thinking modes but no choice of a third-party or local model.P5 Exit survives termination FailMeta AIThe Terms allow Meta to permanently disable access or delete the account for breach. No post-termination export window or shutdown plan is documented, and state is not kept on the owner's device by default.T1 Readable without the provider FailMeta AIAgent state lives on Meta's servers and can be read only through Meta's app or its export service. Base instructions and logic are not available to the owner in any form.T2 What runs is what you see FailMeta AIMeta does not document any view of the system prompt or the assembled context. Material settings are not exposed either: the exact model version in use and provider-side behaviour switches, including personalization from other Meta products, cannot be determined.T3 No shadow data FailMeta AIThe Terms say AIs 'will sometimes retain and use' shared information, and that deleting messages, threads or even the account 'may not delete our copy'; no bounded retention period for AI interactions was found. Interests inferred from AI chats feed ad personalization, and the owner cannot see them item by item.T4 No hidden steering PartialMeta AIMeta-first defaults are disclosed but cannot be switched off: personalization from Meta products and search partners chosen by Meta. No binding commitment against sponsored or self-preferencing recommendations was found.A1 Complete action record FailMeta AIMeta AI keeps a chat History and runs scheduled or recurring tasks, but no durable record of consequential actions is documented, with or without arguments and targets. Meta does not describe how the owner learns about unrecorded actions or recording failures.A2 Tamper-evident, owner-checkable FailMeta AINo integrity data or tamper-evidence mechanism is documented for any Meta AI record.A3 Agent cannot rewrite its record FailMeta AINo protection is documented that separates the record from the agent. Search excerpts of Meta help content (not rendered when opened) indicate that prompts like 'Delete full history' make Meta AI delete stored history.V1 Obtainable source, matching build FailMeta AIThe service, runtime and Muse Spark model are closed. Meta says it 'hopes' to open-source future versions of the model.V2 Owner-rooted identity FailMeta AIThe agent's identity is the owner's Meta account and Meta AI profile in Accounts Center. Meta issues these and can disable them under the Terms.V3 Change history and rollback PartialMeta AIPer Meta help content seen only as search excerpts, a 'Memory updated' marker shows when a memory is saved, with a list of saved memories. No history of instructions, settings or model changes is documented, and there is no restore.M1 Instructions and logic FailMeta AIBase instructions are set by Meta on its servers. No documented means lets the owner replace or override them.M2 Memory PartialMeta AISaved memories are listed and can be deleted one by one or all at once (per Meta help excerpts), but editing is not documented. The Terms say AIs also personalize using information such as 'interests, location and profile' from Meta products, a store that is not itemised in Meta AI.M3 Tools PartialMeta AIConnectors are a Meta-curated set (for example Google Workspace and Meta business assets) that can be connected and disconnected. No documented way lets an owner add an arbitrary tool without Meta review, or remove built-ins such as web search.M4 Owner has the final say FailMeta AIMeta applies protections such as content restrictions, including memory items that 'can't be saved' when deemed unsafe. No owner setting to change them is documented.M5 No unilateral change FailMeta AIMeta changes the model server-side without offering pinning or opt-in (Muse Spark, then Muse Spark 1.1). The Terms promise notice only for material changes to the Terms themselves.C1 Boundaries FailMeta AIThe only documented limit is connecting or disconnecting whole services. No limits are documented for instruction sources, contacts, spend or per-tool scope, or for enforcement against model output.C2 Approval gates FailMeta AINo owner-configurable approval gates are documented for Meta AI. The July 2026 post describes steering tasks in real time, not gating action classes.C3 Immediate halt UnverifiedMeta AIMeta AI runs scheduled and recurring tasks, but no global stop for in-flight, queued or scheduled work is documented. Per the C3 note, this is recorded as Unverified.C4 Data serves only the owner FailMeta AISince 2025-12-16, Meta uses interactions with Meta AI to personalize ads and content in most regions, and the announcement offers no opt-out (only Ads Preferences). The Terms state that interactions may be reviewed by humans and shared with 'select partners, like search engines'.C5 Credential custody PartialMeta AIConnected Google services are listed under Settings > Connected apps and can be disconnected individually. Revoking access to business assets is done by asking Meta AI in chat. Meta does not disclose where tokens are held, whether copies are deleted on revocation, or limits on its use of connected data.C6 Full deletion FailMeta AIDeleting chats removes them from History. The Terms state that deleting messages, threads 'or even your account may not delete our copy of your personal information'.
SiriP1 Owner-usable round trip FailSiriNo export of the Siri AI agent (conversations, personalization, settings) is documented, and Siri AI runs only on Apple devices using Apple Foundation Models on-device and on Apple-operated Private Cloud Compute; there is no restore route on infrastructure Apple does not operate.P2 Complete export FailSiriApple's data-copy service covers Apple Account and iCloud data but excludes data stored end-to-end encrypted or not stored by Apple; Siri conversation history, Siri personalization and on-device personal context are not listed as exportable, so several classes of agent state stay inside Apple's software.P3 Runs without the provider FailSiriSiri AI needs Apple Foundation Models on Apple-operated Private Cloud Compute and an Apple Account; the Usage Terms let Apple 'change, suspend, remove, or disable access to any part of Apple Intelligence Features at any time without notice'. iOS is licensed under Apple's software licence, not an irrevocable right to run the agent independently.P4 Model independence FailSiriThe agent is built on Apple Foundation Models (developed with Google Gemini); the only other model route documented is the Apple-integrated ChatGPT extension. The owner cannot select a local open-weight or other third-party model.P5 Exit survives termination FailSiriThe iCloud terms state that on termination the owner 'may lose all access' to data stored in iCloud (where Siri conversations sync), and Apple may terminate without prior notice for listed causes; the Usage Terms allow suspension or termination of Apple Intelligence access. No post-termination export window or shutdown plan for Siri state is published.T1 Readable without the provider FailSiriSiri conversation history, personalization and personal-context indexes are stored in undocumented formats readable only through Apple's closed Siri app and OS; under default Standard Data Protection the keys for several iCloud categories are held by Apple.T2 What runs is what you see FailSiriNeither the base instructions nor the assembled context sent to the model are viewable; the Apple Intelligence & PCC Report (off by default) logs requests sent to PCC but not system prompts. The exact model identity/version serving a request and server-side behaviour switches are not documented. Both shortfalls apply.T3 No shadow data FailSiriBy default Apple stores Siri transcripts and related request data for up to two years, and a reviewed subset 'may be kept beyond two years' with no stated bound; the owner cannot view these server-side items one by one (they are keyed to a rotating identifier).T4 No hidden steering PartialSiriNo evidence of paid placement was found, and Apple states Siri data is not used to build a marketing profile, but there is no binding commitment against self-preferencing in Siri's recommendations or tool choices, and the statements are policy text Apple may change.A1 Complete action record FailSiriThere is no durable action record: the Siri app keeps conversations (prunable via 'Keep Conversations'), and the Apple Intelligence & PCC Report is off by default and covers only off-device requests. Actions in apps (e.g. messages sent with 'Automatically Send Messages'), Shortcuts automations and proactive suggestions are not documented as recorded.A2 Tamper-evident, owner-checkable FailSiriNo integrity data or owner-side verifier exists for any action record. Apple's PCC transparency log attests PCC software images, not the owner's action record.A3 Agent cannot rewrite its record FailSiriNo documented mechanism stops the agent from altering or deleting its own conversation record; Siri AI can take actions across apps, and Apple documents no separation between the agent's capabilities and its record. Under G12 the protection is not established.V1 Obtainable source, matching build FailSiriiOS, the Siri app and Apple Foundation Models are closed. For Private Cloud Compute Apple publishes production binary images and a transparency log but only 'a subset of the security-critical PCC source code', so required components are closed.V2 Owner-rooted identity FailSiriThe agent has no identity separate from the owner's Apple Account, which Apple issues and may suspend or terminate.V3 Change history and rollback FailSiriNo change history exists for Siri settings, personalization or memory, and no rollback; iCloud sync of Siri settings keeps only the current state.M1 Instructions and logic FailSiriSiri AI's base instructions and orchestration logic are set by Apple on-device and on PCC and cannot be viewed or replaced; owner customisation is limited to voice, responses, app access and personal information.M2 Memory FailSiriSiri conversations are itemised but can only be deleted, not edited; Siri & Dictation history can only be deleted as a whole; Siri personalization learned from apps ('Learn from this App') and the personal-context index are not itemised or editable. More than one store falls short.M3 Tools PartialSiriThe owner can add tools without review by building Shortcuts, and third-party apps expose actions via App Intents, but app-based tools reach users through App Store distribution; built-ins can be limited (per-app 'Use with Siri Requests', Screen Time capability blocks) but core capabilities such as world-knowledge search are not documented as removable.M4 Owner has the final say PartialSiriSome protections are owner-adjustable (Screen Time Allow/Block for Sensitive Topics, Explicit Language, Image Creation; ChatGPT confirmation), but others are locked: server-side usage limits, Siri Recap's omission of sensitive content, and PCC-side filtering.M5 No unilateral change FailSiriThe Usage Terms let Apple change, suspend or disable any part of Apple Intelligence at any time without notice and modify usage limits at any time; server-side models are updated by Apple with no pinning, and continued use after term changes counts as acceptance.C1 Boundaries FailSiriOwner limits exist per app (Use with Siri Requests, Learn from this App) and via Screen Time, but there is no spend cap for Siri-initiated Apple Cash payments, instruction-source control is on/off (Hey Siri, lock-screen access), and Apple does not document that limits hold against prompt injection from onscreen content. Two or more shortfalls.C2 Approval gates PartialSiriSiri asks for confirmation for some fixed classes (sending messages unless 'Automatically Send Messages' is on, sending to ChatGPT, Apple Pay authentication), but the owner cannot gate arbitrary action classes.C3 Immediate halt PartialSiriThe owner can dismiss Siri or turn it off, and Siri Recap can be toggled from Control Center, but no single control documented to stop in-flight PCC work, scheduled Shortcuts automations and background features together.C4 Data serves only the owner FailSiriSiri, Dictation & Privacy describes the default: Apple stores Siri transcripts, may review a subset, and may use them to fine-tune Siri and other models for up to two years; this applies to anyone who uses Siri, with 'Improve Siri and Dictation' adding audio review only as an opt-in. Training use and staff review of content are therefore on by default.C5 Credential custody PartialSiriSiri uses the owner's existing app sessions; per-app Siri access and the optional ChatGPT account link are visible and individually revocable. Apple does not document that no provider-side copy of grants or linked-account tokens remains after revocation, so pass is withheld (G12).C6 Full deletion FailSiriDeleting Siri & Dictation History deletes transcripts 'stored on your device'; server-side transcripts kept for up to two years (and reviewed items beyond) are not covered, and no single action deletes the whole agent with confirmation.
OpenAI DotsP1 Owner-usable round trip FailOpenAI DotsNo documented export of a dot exists; the ChatGPT account export covers chat history and account data, excludes scheduled tasks, can take up to 7 days, and there is no importer or runtime outside OpenAI that can restore a dot.P2 Complete export FailOpenAI DotsScheduled tasks are excluded from the personal export, the dot's own memories cannot be viewed or exported individually, and nothing states that the dot's notes, cloud-computer files, custom rules or activity are included; at most conversation text leaves.P3 Runs without the provider FailOpenAI DotsA dot runs only on OpenAI's cloud computer inside a ChatGPT account and subscription; it cannot start or act with OpenAI unreachable.P4 Model independence FailOpenAI DotsDots are powered by GPT-6 Astra; no documented option lets the owner select another vendor's or a local open-weight model, and enterprise model controls do not apply to dots.P5 Exit survives termination FailOpenAI DotsThe Terms of Use let OpenAI suspend or terminate access for breach or risk and contain no post-termination export window; dot state lives only on OpenAI systems.T1 Readable without the provider FailOpenAI DotsDot state (notes, conversations, activity, schedules) is readable only through ChatGPT, a provider-operated service; no public schema or owner-held storage is documented.T2 What runs is what you see FailOpenAI DotsThe base system prompt is not visible (OpenAI's Model Spec keeps system instructions private by default), and provider-controlled switches such as Autoreview, safety monitoring and core safety requirements are not exposed as determinable settings; model identity (GPT-6 Astra) is disclosed.T3 No shadow data FailOpenAI DotsDeleted data is removed within 30 days, but data from banned accounts and data held for legal obligations may be retained without a stated limit; held data is not viewable item by item.T4 No hidden steering UnverifiedOpenAI DotsNo opened public source addresses paid placement, sponsored plugins or self-preferencing in dots' recommendations or tool choices.A1 Complete action record FailOpenAI DotsActivity View shows tasks, delegated tasks and steps, but no source says recording failures are surfaced, that provider-side actions (safety monitoring pauses, Autoreview) are recorded, or how long the record is kept; the admin guide warns to confirm Compliance API coverage before relying on it for audit.A2 Tamper-evident, owner-checkable FailOpenAI DotsNo integrity data (hashing, signing, independent anchoring) for the dot's action record is documented.A3 Agent cannot rewrite its record UnverifiedOpenAI DotsNo public source states whether the dot can alter, delete or suppress entries in its Activity View or other records.V1 Obtainable source, matching build FailOpenAI DotsDots, ChatGPT and GPT-6 Astra are closed, provider-hosted components; no source is available.V2 Owner-rooted identity FailOpenAI DotsA personal dot's identity is a provider-issued handle (e.g. @yourname-dot) inside the owner's OpenAI account, which OpenAI can suspend; dedicated identities are described only for enterprise specialist dots.V3 Change history and rollback FailOpenAI DotsNo change history, attribution or rollback of dot state is documented; individual dot memories cannot be viewed, and reset only deletes the dot.M1 Instructions and logic PartialOpenAI DotsOwners can set custom rules and instructions, but OpenAI's base instructions, core safety requirements and Autoreview cannot be replaced.M2 Memory FailOpenAI DotsThe dot's own memories cannot be viewed, deleted or modified individually; the only documented removal is deleting (resetting) the whole dot, and ChatGPT memory settings do not necessarily change the dot's notes.M3 Tools PartialOpenAI DotsDots use plugins installed for the account from OpenAI's plugin directory; Pro users can add custom MCP apps only with read/fetch permissions in developer mode, whether dots use them is not documented, and built-in cloud browser and computer tools cannot be removed.M4 Owner has the final say PartialOpenAI DotsOwners can loosen approval rules and choose whether to connect a local computer, but custom rules cannot turn off core safety requirements, Autoreview or proactive-research restrictions.M5 No unilateral change FailOpenAI DotsDots run on a hosted GPT-6 Astra with no documented version pinning, and OpenAI has retired ChatGPT models and moved users automatically to successors; server-side safety systems can change without owner consent.C1 Boundaries FailOpenAI DotsCustom rules are described as 'instructions your dot tries to follow, and it can make mistakes', so limits can be overridden by model output; per-recipient, per-sender and per-domain lists and instruction-source limits are not documented.C2 Approval gates PartialOpenAI DotsOwners can set actions to require approval via custom rules, and some sensitive actions always require takeover, but the rules are model-followed instructions and the classes are fixed to rule options, with the model and Autoreview deciding what matches.C3 Immediate halt PartialOpenAI DotsPause stops the dot's current main task but not every delegated task or future scheduled runs, which need separate steps in Scheduled.C4 Data serves only the owner FailOpenAI DotsOn personal plans such as Pro, OpenAI may use content, dots' conversations and work, and app data to train models unless the owner opts out; human review can occur even with model improvement off; privacy-policy changes apply on posting.C5 Credential custody PartialOpenAI DotsConnected apps are listed and can be disconnected, and saved passwords go to the cloud browser without reaching the model, but OpenAI does not disclose where connector OAuth tokens are held or that copies are deleted on revocation.C6 Full deletion FailOpenAI DotsDeleting a dot is one action, but no completion limit or reach to backups is stated for it; account deletion completes within 30 days except data retained where required or permitted by law and content already de-identified for training, which survive.
DeepSeekP1 Owner-usable round trip FailDeepSeekNo documented export of the agent that can be restored on a runtime or host DeepSeek does not operate; the Privacy Policy mentions only copying or deleting chat history via settings, and portability requests go by email to privacy@deepseek.com.P2 Complete export FailDeepSeekAny export documented covers chat history (conversation text) only; account settings, model and behaviour configuration remain with the provider.P3 Runs without the provider FailDeepSeekThe agent runs only on DeepSeek's hosted service with a registered account, and Terms 3.1 grant only a 'revocable, non-transferable, and non-exclusive right' to use it.P4 Model independence FailDeepSeekThe chat agent is served only with DeepSeek's own models; no documented way exists to select another model. Open MIT weights allow self-hosting the model, not this agent.P5 Exit survives termination FailDeepSeekTerms 8.2 allow DeepSeek to close accounts and delete content at its own judgement, with no post-termination export window; Terms 11.1 and 1.3 allow the service to be suspended or terminated. State lives only on the provider's servers.T1 Readable without the provider FailDeepSeekAgent state (chat history, settings) is readable only through DeepSeek's hosted service; no public schema or owner-held keys are documented.T2 What runs is what you see FailDeepSeekTwo shortfalls: the base/system prompt placed in context is not shown to the owner, and provider-controlled behaviour switches such as the 'risk filtering mechanisms' of Terms 3.3 cannot be determined. The served model is determinable only through changelog announcements.T3 No shadow data FailDeepSeekRetention is 'as long as necessary', including for 'legitimate business interest', with no bounded periods; device IDs, user IDs, logs and cross-device activity are collected but are not visible item by item.T4 No hidden steering PartialDeepSeekThe Privacy Policy states DeepSeek does not do targeted advertising or sell data, and no paid placement is described. This rests on a policy that can change on posting (Terms 11.1), so it is non-binding under G6. Which third-party search APIs are used is not disclosed.A1 Complete action record FailDeepSeekThe chat history is the only record. Provider-side actions such as content deletion or account restriction under Terms 8.2 are not recorded for the owner, and nothing documents that failed recording is surfaced (G12). Searches send keywords to third parties with no documented argument-level record.A2 Tamper-evident, owner-checkable FailDeepSeekNo integrity data, signatures or verifier for the chat or action record are documented; the provider can alter or delete it without trace.A3 Agent cannot rewrite its record PartialDeepSeekThe documented tools (search, reasoning, file reading) give the model no evident path to alter its chat history, but no protection is documented, so it rests on architecture by convention only (G12).V1 Obtainable source, matching build FailDeepSeekThe hosted chat application and serving stack are closed, and Terms 3.5(3) forbid attempting to discover the source. Only model weights and inference code are MIT-licensed.V2 Owner-rooted identity FailDeepSeekThe only identity is a DeepSeek account that the provider can restrict, close or bar from re-registration (Terms 2.2, 8.2).V3 Change history and rollback FailDeepSeekNo documented history of changes to settings, model or behaviour per account, and no restore of a prior state.M1 Instructions and logic FailDeepSeekNo owner-editable system prompt or custom instructions is documented by DeepSeek; a third-party guide reports there is no custom-instructions field. Base instructions are fixed provider-side.M2 Memory PartialDeepSeekPrimary sources document no cross-session memory. Chat history persists, can be deleted but not edited. That no other memory is reused is established only by third-party sources, which cannot support a pass.M3 Tools FailDeepSeekThe tool set (search, deep thinking, file upload) is fixed; no documented way to add owner-built tools to the chat agent.M4 Owner has the final say FailDeepSeekTerms 3.3 give DeepSeek the right to apply risk filtering and content-review mechanisms; no owner control over any provider-imposed protection is documented.M5 No unilateral change FailDeepSeekThe changelog records DeepSeek-V4-Pro GA 'rolled out on the APP, Web, and API' with no opt-in, and retired models' names silently re-routed to V4.1-Flash. Terms 1.3 allow modifications with notice only when changes 'may greatly affect your rights'.C1 Boundaries FailDeepSeekContact and spend are treated as met (G11: the agent cannot message others or pay). Two shortfalls remain: tool/data scope is on/off only (search toggle, no per-domain limits), and there is no limit on instruction sources such as injected web content.C2 Approval gates FailDeepSeekNo owner-set approval gates are documented; searches and outbound keyword sharing proceed without approval once search is on.C3 Immediate halt UnverifiedDeepSeekNo primary documentation of a stop control or its guarantees; the chat UI could not be observed (HTTP 403). No scheduled or background work is documented (G11).C4 Data serves only the owner FailDeepSeekInputs and outputs are used to improve and train models unless the user turns off 'Improve the model for everyone' (opt-out), data is shared with group entities for 'foundation model training', and policy changes apply on posting without specific consent.C5 Credential custody PartialDeepSeekThe agent holds no integration credentials beyond optional Apple/Google sign-in, for which DeepSeek collects an access token. Such grants are revocable at the identity provider, but nothing in-product lists them or documents deletion of the token copy.C6 Full deletion FailDeepSeekAccount deletion exists, but Terms 2.5 reserve the right to retain data after deletion, no completion limit or confirmation is stated, and de-identified training copies are not covered.
Google CCP1 Owner-usable round trip FailGoogle CCNo documented export of a CC agent exists, and CC runs only on Google-operated infrastructure, so it cannot be restored elsewhere.P2 Complete export FailGoogle CCNo export format or bundle is documented. Memory, instructions, schedules, CC's own account identity and history all stay with Google.P3 Runs without the provider FailGoogle CCCC runs on Google's cloud instances under a Google account and needs Google's servers to start or act. The Google Terms licence is personal and non-assignable, and Google may suspend or terminate access.P4 Model independence FailGoogle CCCC is built on 'the latest Gemini models' and offers no documented way to choose another model or a local one.P5 Exit survives termination FailGoogle CCNothing binding keeps CC state available after a ban, suspension or the Labs experiment ending. Google Terms promise Takeout export on discontinuation, but nothing shows Takeout covers CC state, and there is no post-ban export window.T1 Readable without the provider FailGoogle CCAgent state (memory, instructions, logic, history) is held in Google's hosted service. No documented format or schema allows reading it outside that service.T2 What runs is what you see FailGoogle CCThe base prompt and assembled context are not visible to the owner, and Google does not document the exact model in effect ('latest Gemini models'; a specific model name appears only in press reports). That is both shortfalls.T3 No shadow data FailGoogle CCNo CC-specific retention schedule exists. The general Google retention policy allows keeping some data for the life of the account or longer for security, legal and other purposes, and does not itemise per service.T4 No hidden steering PartialGoogle CCCC openly works only through Google services (Gmail, Calendar, Drive, reportedly the Maps API), and these cannot be replaced. No paid placement is documented, but no binding no-steering commitment exists either.A1 Complete action record FailGoogle CCNo owner-visible, durable record of CC's actions is documented. Drafts, events and emails show up in Google apps, but there is no action log.A2 Tamper-evident, owner-checkable FailGoogle CCNo action record or integrity data is documented, so tampering cannot be detected.A3 Agent cannot rewrite its record FailGoogle CCNo action record and no protection of one against the agent is documented.V1 Obtainable source, matching build FailGoogle CCRequired components (CC service, Antigravity-hosted runtime, Gemini) are closed. Google Terms forbid copying or modifying Google software.V2 Owner-rooted identity FailGoogle CCCC's identity is its own Google account, issued and controllable by Google, which can suspend or terminate it.V3 Change history and rollback FailGoogle CCNo way is documented to see, attribute or roll back changes to CC's memory, instructions, model or settings.M1 Instructions and logic FailGoogle CCBase instructions are set by Google. The owner can only 'teach' CC preferences by email, with no documented, viewable or editable instruction layer. Under G12 that doesn't count as an editable owner layer.M2 Memory FailGoogle CCCC keeps personal and shared household memory, but no itemised view, edit or delete control is documented. Memory can only be changed by asking the agent, or cleared by deleting or disconnecting the account; deleting a source email does not remove it from memory (third party).M3 Tools FailGoogle CCCC has a fixed set of Google-provided capabilities, with no documented way to add owner tools or remove built-ins.M4 Owner has the final say FailGoogle CCGoogle-imposed protections (for example, CC originally could only email the account owner) are fixed. No owner setting to change them is documented.M5 No unilateral change FailGoogle CCCC uses 'the latest Gemini models' on Google's service with no version pinning or opt-in updates. Google changes the model and behaviour server-side.C1 Boundaries FailGoogle CCData scope is per source (members choose emails, folders, calendars), and CC responds only to group members. But contact outside the group is a permission prompt, not a per-recipient allow list. Public evidence also doesn't settle whether limits hold against model output or the agent changing its own settings, which counts as a second shortfall under G12. Spend is treated as met (G11): CC has no payment capability.C2 Approval gates PartialGoogle CCThere is a fixed gate: CC reportedly needs permission before acting or sharing outside the group, and it prepares drafts rather than sending them. The owner cannot choose which action classes are gated.C3 Immediate halt UnverifiedGoogle CCA disconnect route is reported (Google Account connections page), but no documented guarantee says it stops in-flight, scheduled (daily brief) or background work within seconds.C4 Data serves only the owner FailGoogle CCThe disclaimer excludes training of foundational models. Otherwise CC data falls under the general Google Privacy Policy, which allows using data to improve and develop services and legal disclosure without a notice commitment. Staff or human review of CC data is not disclosed, and no CC-specific opt-out exists.C5 Credential custody PartialGoogle CCMembers choose and can change what CC may access, and access can reportedly be revoked via Google Account connections. Primary sources don't establish a per-grant list with immediate effect, or that no copy is kept after revocation.C6 Full deletion PartialGoogle CCThird parties report an owner-initiated account delete or disconnect that removes CC data. Google's general policy gives about 2 months for deletion and up to 6 months for backups. No completion confirmation is documented, and nothing states whether deletion covers CC's own account and shared memory.
InstinctP1 Owner-usable round trip FailInstinctThe terms, privacy policy and web-app settings document no export of the agent and no import or restore path. The agent runs only on Instinct-operated infrastructure.P2 Complete export FailInstinctWith no export, every class of agent state (memories, conversations, Vault items, integrations, agent mailbox and phone identity) stays with the provider. The settings page mentions 'memories' and 'conversations' but offers no download.P3 Runs without the provider FailInstinctThe agent runs on Instinct's servers and needs an Instinct account. The licence is personal, non-transferable and ends automatically on breach, and the company 'may suspend, disable, or delete your Account and/or the Services ... with or without notice, for any or no reason.'P4 Model independence FailInstinctThe homepage describes 'Instinct's core model', which Instinct trains itself. No public source documents a way for the owner to choose another model or a local open-weight model.P5 Exit survives termination FailInstinctThe company may delete the account 'with or without notice, for any or no reason', and on deletion 'may, but is not obligated to, delete any of your Materials'. No post-termination export window, owner-held backup or shutdown plan is published.T1 Readable without the provider FailInstinctMemory, conversations, Vault items and indexed external data are stored on the provider's servers. They can be read only through Instinct's apps and service, and no format or schema is published.T2 What runs is what you see FailInstinctBoth shortfalls apply. The base prompt and assembled context are not exposed to the owner. The model identity and version are not disclosed ('Instinct's core model', plus unnamed 'third party large language models'). The web chat client handles tool-invocation records, but no documentation says the full context or effective settings can be viewed.T3 No shadow data FailInstinctThe privacy policy discloses no retention period for any class of data. Instinct collects clickstream data including keystrokes and cursor positions. Indexed data from a disconnected service is kept and may still be used until the owner separately requests deletion. Derived data and de-identified data may be used 'for any purpose'.T4 No hidden steering UnverifiedInstinctThe agent books rides and services and makes purchases through built-in commerce integrations (Shop Pay, Shop Wallet, Stripe Link). No public source says whether vendor choices involve paid placement or self-preference, and there is no binding commitment either way.A1 Complete action record FailInstinctNo documented, durable action log exists. The terms state that 'Records of Actions available through the Services may not always be accurate', so recording can fail silently. By G12, the undocumented coverage of proactive, provider-side actions and the undocumented retention count as further shortfalls.A2 Tamper-evident, owner-checkable FailInstinctNo integrity data, signing or verifier is documented for any action record. Records are held by the provider.A3 Agent cannot rewrite its record UnverifiedInstinctNo public source says whether the agent can alter, delete or suppress records of its own actions.V1 Obtainable source, matching build FailInstinctThe runtime, model and service are closed and provider-hosted. The terms forbid decompiling or reverse engineering the Services.V2 Owner-rooted identity FailInstinctThe agent's identities are provider-issued: an Instinct-hosted agent mailbox address ('Pick a new name for your Instinct inbox'), Instinct's messaging number, and trusted-people links issued by the service. The provider can revoke all of them.V3 Change history and rollback FailInstinctNo change history, attribution or rollback is documented for memory, instructions, settings or the model. The terms note that some changes (for example Beta Offerings) may not be reversible.M1 Instructions and logic FailInstinctThe base instructions and behaviour live in Instinct's hosted service and trained model. No public source documents a way to view or replace the base prompt or default skills. The owner can only give instructions in conversation.M2 Memory FailInstinctThe service keeps 'memories' and uses prior interactions for personalisation, but no public source or settings screen lists memory items with edit or delete controls. Only bulk deletion of indexed external data and location history is offered.M3 Tools PartialInstinctThe owner can connect and disconnect integrations from a provider catalogue (Google Workspace, Outlook, Slack, Notion, Linear, GitHub, Granola, WHOOP, Shop Pay, Link and others). No interface for owner-built tools is documented, and built-in capabilities (computer and phone use, messaging) cannot be removed.M4 Owner has the final say FailInstinctThe provider sets safeguards and confirmation requirements on Actions at its discretion. The Acceptable Use Policy forbids circumventing 'usage restrictions or other safety measures' or enabling 'functionality disabled by the Company'. No owner override is documented.M5 No unilateral change FailInstinctThe terms reserve the right to change 'the content, features, and other services' at any time, with no advance notice or opt-in. No version pinning of model, instructions or runtime is offered.C1 Boundaries FailInstinctAt least two shortfalls apply. Spend limits are missing, even though the agent can pay through stored cards and Shop Pay/Link. No documented per-recipient limit on who the agent contacts by email or phone. No documented control over whose instructions it follows. Agent-to-agent messaging does have per-person trust and block lists and a pause switch.C2 Approval gates FailInstinctThe owner authorises the Services to take Actions 'as the Services may deem are responsive'. Any confirmation requirements are set by the provider ('We may implement safeguards, confirmation requirements...'). No owner-configurable approval gates are documented. Third-party reporting, which does not affect the grade, describes an email sent without prior approval.C3 Immediate halt UnverifiedInstinctNo public source documents an owner control that halts all agent activity, including proactive and provider-side work. The only documented pause covers agent-to-agent connections. Under the C3 note, a desk review without a documented guarantee records this test as Unverified.C4 Data serves only the owner FailInstinctBy default Instinct uses Materials and Usage Data, including keystroke-level clickstream, to train its AI models. The settings toggle 'Improve Instinct for everyone' stays on unless the owner opts out, and safety-flagged content is used for training even after opting out. Data may go to 'third-party business partners who may use the information for their own purposes', aggregated or anonymised. Contact details are used for marketing. Changes to data terms apply on notice, without specific consent.C5 Credential custody PartialInstinctVault items (logins, cards, authenticators, including items the agent created) and connected integrations are listed and can be removed one by one. The provider does not say where credentials are held, and does not commit to deleting provider-side copies on revocation. Disconnecting an integration 'does not automatically delete data collected from that integration'.C6 Full deletion FailInstinctSettings offer one-step 'Delete account' ('Deletion usually completes within an hour'). However, the terms state that on account deletion the Company 'may, but is not obligated to, delete any of your Materials'. Backups, third-party model providers and previously trained models are not covered, so provider-side copies can survive.