DeepSeek Harness
DeepSeek (Hangzhou DeepSeek Artificial Intelligence) · github.com/deepseek-ai/deepseek-harness
DeepSeek Harness is an MIT-licensed, local-first agent harness. All agent state is kept as documented files under $DSH_HOME on the owner's machine, and it can run against any OpenAI- or Anthropic-compatible model endpoint. By default it uploads the canonical session log, including user content, tool arguments and results, to DeepSeek with each official-API request, under a setting labelled as helping improve DeepSeek models and products. It offers no whole-agent export or import, no tamper-evident action record, no owner-rooted identity and no session deletion.
- MIT source; the runtime has no licence check, login gate or remote configuration
- State is local-first under $DSH_HOME, so it survives provider termination
- Supports third-party and self-hosted OpenAI- or Anthropic-compatible models
- Owner can add any plugin, MCP server or skill without review
- Session-log upload to DeepSeek is on by default, framed as model and product improvement
- No whole-agent export or import; only a per-session ZIP download
- Action record has no integrity data, and sessions cannot be deleted
- No per-domain egress limits, reads are unconfined, and no global stop
All 26 findings
v0.2.0-rc.2 (npm latest, published 2026-09-29), commit 639ed015397290b3745d163aafe02ffee4aa3f84 (tag dsh-v0.2.0-rc.2). Default `web` profile (dsh-base + dsh-web-app bundles, `standard` preset, `workspace-write` sandbox, approval `ask`). Scope and assumptions: the platform is not fixed and the review is desk-only. Schedules and webhooks are off by default and are outside the default path. Spend is treated as met under G11 because there is no payment capability. Evaluated 2026-09-30.
Portable · Can you leave, take the whole agent with you, and keep it alive?
60%- P1Owner-usable round trip
The only documented export is a per-session ZIP (session log, sub-sessions, attachments) downloaded through the browser. There is no import, restore or backup command, and no documented procedure for copying $DSH_HOME to another host. Session folders are keyed by absolute working-directory paths.
› How it could pass
Suggestions. Any change that meets the test counts.
- Ship a whole-agent export plus a one-command import or restore
- Document a supported 'copy $DSH_HOME' migration that survives path changes
Fail - P2Complete export
All state classes live in owner-held files under $DSH_HOME and nothing stays only with the provider: - sessions, KV storage, profile patches, .credentials.yaml, attachments, skills and AGENTS.md - integrations use API keys or owner-generated webhook secrets, not provider OAuth clients The documented export omits settings, credentials, skills, instructions and workspaces, and there is no restore route. The fail definition (classes held by the provider, or text-only export) is not met, so the test stays partial under G1.
› How it could pass
Suggestions. Any change that meets the test counts.
- Include settings, credentials, skills, instructions and workspace registry in the export, with a manifest
Partial - P3Runs without the provider
The runtime is MIT-licensed, with no licence key, login, remote-config fetch or update check in the CLI/Web path; the official build profile only embeds version strings. DeepSeek endpoints are all configurable or replaceable: - model API - default web_search provider (Exa and Perplexity plugins exist) - OTel collector (DSH_TELEMETRY_OTLP_URL, or disable) This is a desk grade; the provider-blocked run is hands-on.
Pass - P4Model independence
Other providers are supported, and custom OpenAI Chat, OpenAI Responses and Anthropic Messages endpoints work at any base URL, including self-hosted ones. Capabilities are model-neutral: titles and compaction use the session route, and vision works for any model declared image-capable. However, there is no documented minimum model contract, no text-based tool-calling fallback, and no local open-weight model is named or tested (G12).
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish the minimum model contract and test it against a local open-weight model
- Offer text-based tool calling for models without native tools
Partial - P5Exit survives termination
All agent state is kept locally under $DSH_HOME on the owner's machine, with no hosted account or sync. Termination of a DeepSeek account ends only model access and does not remove the owner's copy.
Pass
Transparent · Can you see everything the agent is, and everything shaping it?
38%- T1Readable without the provider
State is kept in open formats: - session logs are JSONL (zstd-compressed by default) with a public event catalog and JSON schema - KV storage is JSON - credentials and config are YAML - no encryption The catalog marks some fields as opaque: replayState, ReplayEnvelope.response, tool-private meta, and provider thinking signatures. Storage-domain schemas exist only as zod definitions in source.
› How it could pass
Suggestions. Any change that meets the test counts.
- Document or remove opaque replay and meta fields
- Publish JSON schemas for storage domains
Partial - T2What runs is what you see
The following are visible to the owner: - system prompt, dynamic context and compaction summaries are logged as session events - the per-request header with config and full tool schemas is logged, and the Trajectory view shows per-request system prompt and tools - `dsh --dump-config` shows every layer - DeepSeek's docs document what the `deepseek-flash` alias resolves to One shortfall is counted under G12: no public evidence settles whether the hosted API adds server-side context. Unevaluated `!!js` config expressions are recorded but not counted.
- System prompt subsystem
- ui-trajectory README
- CLI reference (--dump-config)
- DeepSeek API models & pricing
› How it could pass
Suggestions. Any change that meets the test counts.
- Obtain and publish a statement on server-side context added by the hosted API
- Offer a raw request log option
Partial - T3No shadow data
DeepSeek receives and holds several kinds of data about the owner's use: - by default, the uploaded session log: working directory, prompts, user and assistant content, tool arguments and results, feedback - request headers carrying a stable anonymous installation UUID and the plugin inventory - feedback-triggered OTel uploads of the session prefix DeepSeek's privacy policy gives only an 'as long as necessary' retention, and nothing lets the owner view that data item by item.
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish bounded retention for uploaded session logs, headers and OTel data
- Let the owner view what DeepSeek holds
Fail - T4No hidden steering
No paid placement or sponsored tools were found in source. The first-party defaults (DeepSeek model, DeepSeek-hosted web search) are disclosed in config and replaceable, and the plugin registry is plain npm with no ranking. The absence of steering in the hosted search and model results rests on no binding commitment.
› How it could pass
Suggestions. Any change that meets the test counts.
- Put a no-steering commitment in binding terms
- Expose alternative search providers in the Settings UI
Partial
Auditable · Can you reconstruct exactly what the agent did, and trust the record?
33%- A1Complete action record
The append-only session log records the following, and a tool does not run until its checkpoint write succeeds: - tool calls with full arguments - PTC sub-calls, subagents as child sessions, workflows - approvals and hooks One shortfall: by default, tool results over 12,500 tokens are replaced in the log by a head/tail preview. The full output goes to a temp spill file that is deleted after 30 days.
- persistence catalog (tool/call)
- session-checkpoint-policy README
- spill-policy README
- spill-local README
› How it could pass
Suggestions. Any change that meets the test counts.
- Keep full tool results in the durable record, or never auto-delete spill files
Partial - A2Tamper-evident, owner-checkable
The session log has no hash chain, signatures or anchoring. The only integrity data is zstd's per-frame content checksum, which detects corruption but not deliberate edits. The SHA-256 values in the docs are schema-type fingerprints, not record integrity.
› How it could pass
Suggestions. Any change that meets the test counts.
- Hash-chain or sign log entries and publish an open verifier
- Anchor checkpoints with the owner
Fail - A3Agent cannot rewrite its record
In the default workspace-write sandbox, writable roots are the workspace and temp directories, so ~/.dsh/sessions is normally out of reach. This protection is incidental rather than an explicit deny. It fails if the workspace contains $DSH_HOME, for example a workspace at the home directory, and the linked spill files sit in writable temp.
› How it could pass
Suggestions. Any change that meets the test counts.
- Explicitly deny agent writes to $DSH_HOME and spill storage in every mode
- Document the protection and how to check it
Partial
Verifiable · Can you prove what it runs, who it is, and what changed?
33%- V1Obtainable source, matching build
Source is available: - all required components are MIT source, with vendored Cordis - the native Landlock addon source is BSD-3, though it ships as prebuilt optional packages - the only closed dependency (Claude Agent SDK) sits in an optional, disabled subagent The npm publish job uses a token without `--provenance`, SLSA or sigstore, so a third party cannot tie the published build to the source.
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish with npm provenance or SLSA attestations
- Document how to verify the installed package against source
Partial - V2Owner-rooted identity
The only identity is a random anonymous installation UUID used to correlate telemetry and DeepSeek requests. It has no keypair, DID or signing, and actions are not attributable to it. The only other identity is the provider's DeepSeek platform account.
› How it could pass
Suggestions. Any change that meets the test counts.
- Create an owner-held keypair identity, sign outbound actions, and support owner rotation
Fail - V3Change history and rollback
Per-session changes are recorded as attributed events: model selection, permission and sandbox mode, preset, system-prompt snapshots, tool addition and removal. A session can be forked at any event. There is no history or rollback for global state (profile config, credentials, skills, AGENTS.md), and workspace file diffs cannot be reverted.
› How it could pass
Suggestions. Any change that meets the test counts.
- Version global config, skills and instructions with attribution
- Offer one-step restore to an earlier state
Partial
Modifiable · Can you change anything, and can only you change it?
60%- M1Instructions and logic
The owner can replace the whole system prompt through a preset persona with `complete: true`, adjust prompt prefix and suffix and plan-mode text in YAML, and edit AGENTS.md files; no default skills ship. Built-in tool descriptions for bash, read, write, edit and grep are hard-coded with no config field.
› How it could pass
Suggestions. Any change that meets the test counts.
- Make every built-in tool description and reminder text owner-replaceable in config
Partial - M2Memory
The default Web offering has no built-in memory store the agent reuses across sessions; goals, todos and compaction are session-scoped. What does persist across sessions is plain files the owner can view, edit and delete: AGENTS.md instruction files and skill directories. Memory MCP servers are default-off examples.
Pass - M3Tools
Owners can add any npm, git, tarball or local plugin, or MCP server, through `dsh plugin add` or the Plugins dialog without provider review. Built-in tools can be removed with documented profile-patch edits (`disabled: true`, or overriding a preset's plugin list).
Pass - M4Owner has the final say
The owner can turn off the sandbox and approvals (danger-full-access preset), and can tune or disable the read-before-write check, the repeat-tool reminder and job caps. web_fetch's block on private and non-public destinations has no config switch.
› How it could pass
Suggestions. Any change that meets the test counts.
- Make every built-in protection, including web_fetch destination filtering, owner-configurable with a warning
Partial - M5No unilateral change
The default model is the hosted alias `deepseek-flash`, whose target DeepSeek changes server-side; its docs show retired model names silently served by DeepSeek-V4.1-Flash. The documented launch `npx @deepseek-ai/dsh web` is unpinned, and no pin or pre-update notice is documented. Plugins do not auto-update.
› How it could pass
Suggestions. Any change that meets the test counts.
- Offer pinned dated model snapshots with retirement notice
- Document runtime version pinning and show changelogs before upgrading
Fail
Controllable · Is your word final?
25%- C1Boundaries
Instruction sources are limited: the server binds to loopback with token auth, and schedules and webhooks are off; spend is treated as met under G11. Two limit dimensions are missing: - **Contact and egress:** web_fetch runs in every mode, and there is no per-domain allow or deny list. - **Data scope:** reads are not confined in any preset, so the agent can read .credentials.yaml.
› How it could pass
Suggestions. Any change that meets the test counts.
- Add per-domain egress allow and deny lists that apply to every tool
- Allow read-scope limits and deny $DSH_HOME secrets to the agent
Fail - C2Approval gates
Built-in gates cover a fixed set of classes: sandbox escalation, plugin_manager actions, and the ask/never mode. Gating an arbitrary tool needs a PreToolUse hook command (G3). In-process subagents run with approval `never`, so their asks are rejected rather than bypassed.
› How it could pass
Suggestions. Any change that meets the test counts.
- Let the owner gate any tool or action class from settings
Partial - C3Immediate halt
Stop cancels the foreground turn and keeps queued input. Background jobs and continuable subagents need separate stops. Archiving a session stops all of its work, and the only global stop is killing the process.
› How it could pass
Suggestions. Any change that meets the test counts.
- Provide one global stop that cancels turns, subagents, jobs and queued input
Partial - C4Data serves only the owner
By default (`enabled: true`) every official-API request carries the canonical session log. The only control is an opt-out switch labelled 'Help improve DeepSeek models and products'; DeepSeek's privacy policy lists training as a use, with opt-out only. Other flows: - one feedback click uploads the session prefix to a DeepSeek OTel collector; this can only be disabled by environment variable - data-term changes apply on notice
- session-log-deepseek src/index.ts (default true)
- ui-settings-session-log locales
- base bundle patch (OTel)
- DeepSeek Privacy Policy
- DeepSeek Open Platform Terms
› How it could pass
Suggestions. Any change that meets the test counts.
- Make session-log upload opt-in with a prominent first-run choice
- Add a UI switch for OTel feedback upload and disclose it on every rating
- Commit in binding terms that training and staff review are opt-in
Fail - C5Credential custody
Keys are stored in owner-held $DSH_HOME/.credentials.yaml. The UI is write-only for secrets and lets each provider credential be deleted. There is no single list of every credential: environment and .env credentials and the session-signing secret are not shown. Signing out of the DeepSeek account does not revoke the grant remotely.
› How it could pass
Suggestions. Any change that meets the test counts.
- List every credential and grant in one place with per-item revoke, including remote revocation
Partial - C6Full deletion
Sessions can be archived but never deleted, and there is no documented whole-agent deletion. No deletion route exists for session logs or OTel exports already uploaded to DeepSeek; withdrawing feedback exports a deletion event, not remote erasure.
› How it could pass
Suggestions. Any change that meets the test counts.
- Offer one-step deletion of the whole agent
- Provide deletion of provider-held uploads within a stated limit, with confirmation
Fail
How this assessment was produced
One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of DeepSeek Harness (https://github.com/deepseek-ai/deepseek-harness).
During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.
- Model
- claude-opus-5-5
- Harness
- Claude Code 2.1.285
- Ran
- 2026-09-30 14:26:08 UTC · 5 turns
- Criteria hash
- a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
- Full transcript
- Download (JSONL, gzip) · every tool call, page read and reply
- Run protocol
- 2026-09-30-v1
Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.
- Human review
- pending
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/deepseek-harness.json.
Or grade it from scratch. See Check any score yourself.
Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.