Skip to content
Agents of Humanity
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
AssistanthostedProprietary

Claude

Anthropic · claude.ai

Claude.ai is a closed, provider-hosted assistant. The agent's state, runtime and models are all held and controlled by Anthropic. The owner can export conversations and memory, curate memory, add custom connectors without review and opt out of training, but there is no restore path outside Anthropic's service. The provider keeps broad rights to change, suspend or end the service without notice.

Strengths
  • Memory topics can be viewed, edited and deleted item by item (M2)
  • Owner-built remote MCP connectors can be added and removed without provider review (M3)
  • Base system prompt is published per model, and a public ad-free statement exists (T4, M1 context)
  • Self-service data export, including memory, on all individual plans (P1/P2 context)
Gaps
  • No restore of an exported agent outside Anthropic's service; locked to Anthropic models and runtime (P1, P3, P4)
  • Content is used for model training unless the owner opts out; staff review of flagged content (C4)
  • Terms allow suspension, service changes and data deletion without notice (P5, M5)
  • No tamper-evident action record, owner-rooted identity or change history (A2, V2, V3)
Evidence

All 26 findings

Hosted SaaS, current production release as documented on 2026-09-30. Published system prompt for Claude Opus 5.5 dated 2026-09-22. Consumer Terms effective 2025-10-08; Privacy Policy effective 2026-09-10. Scope: claude.ai web chat including memory, projects, connectors (remote MCP) and built-in tools. Claude Code, Cowork, desktop-local MCP and the API are out of scope.

Portable · Can you leave, take the whole agent with you, and keep it alive?

0%
  • P1
    Owner-usable round trip

    Self-service export is delivered as an emailed download link after processing, with the link expiring after 24 hours. No importer or non-Anthropic runtime is documented that restores the export as a working agent.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish an importer or runtime not operated by Anthropic that restores the export as a working agent
    • Make export available immediately on request
    Fail
  • P2
    Complete export

    The export is documented as 'conversation data and the user data for your account', and all memory data is included. It is not documented whether project instructions and knowledge files, styles, preferences or connector configuration are included, so under G12 two or more classes are treated as staying with the provider. Connectors re-link through Anthropic's cloud.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish a manifest covering every class of agent state (projects, files, instructions, styles, connectors) and include all of it
    • Document a way to re-link integrations without Anthropic's infrastructure
    Fail
  • P3
    Runs without the provider

    The agent runs only on Anthropic's hosted service and needs an Anthropic account. The Consumer Terms reserve the right to 'modify, suspend, or discontinue the Services or your access ... at any time without notice'.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide a runtime the owner can run without any Anthropic account or service
    • Grant a perpetual, irrevocable right to run it
    Fail
  • P4
    Model independence

    Model choice is limited to the Anthropic models in the selector next to the send button. No documented way to use a third-party or local open-weight model.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Allow any model that meets a documented minimum, including a local open-weight model, with every capability kept
    Fail
  • P5
    Exit survives termination

    The Consumer Terms allow Anthropic to suspend or terminate access 'at any time without notice' for suspected breach and 'at our option delete any Materials or other data associated with your Account'. No post-termination export window or shutdown plan is published, and state is not held locally by default.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Commit in binding terms to an export window of at least 30 days after any termination, including bans
    • Publish a shutdown plan
    Fail

Transparent · Can you see everything the agent is, and everything shaping it?

13%
  • T1
    Readable without the provider

    Agent state lives on Anthropic's hosted service and is read through claude.ai. The provider holds any encryption keys. The export file format and schema are not publicly documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish the export schema for every state class
    • Let the owner read the complete state with general-purpose tools and owner-held keys
    Fail
  • T2
    What runs is what you see

    Anthropic publishes the core claude.ai system prompt per model, and the selected model is shown. However, the published prompt lists classifier-triggered reminders (image_reminder, cyber_warning, system_warning, ethics_reminder, ip_reminder, long_conversation_reminder) that the owner cannot see when injected, and tool definitions are not published. Whether a provider switch such as a classifier is active cannot be determined. Two shortfalls.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner view the full assembled context of any call, including injected reminders and tool definitions
    • Expose every provider-controlled behaviour switch in effect
    Fail
  • T3
    No shadow data

    Active chats are kept 'for as long as reasonably necessary', which is not a bounded period. Safety-flagged inputs and outputs are kept up to 2 years and trust and safety classification scores up to 7 years, and neither is visible to the owner item by item. De-identified training data is kept up to 5 years.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish bounded retention periods for every data class
    • Let the owner view classification scores and any other derived data held about them
    Fail
  • T4
    No hidden steering

    Anthropic has stated 'Claude will remain ad-free' and that responses will not be 'influenced by advertisers or include third-party product placements'. This is a news post, not binding terms, so absence of steering rests on a non-binding statement.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Put the no-paid-placement and no-self-preferencing commitment in binding terms
    Partial

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

17%
  • A1
    Complete action record

    Conversation history is the only record of actions, and it is kept until the owner deletes it. No documentation shows that provider-side or staff actions are recorded, or that the owner is told when recording fails, so under G12 two shortfalls are counted.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Record every consequential action from every path, including provider-side automation
    • Alert the owner when an action cannot be recorded
    Fail
  • A2
    Tamper-evident, owner-checkable

    No integrity data, signing or verifier is documented for the conversation or action record. The record holder can change it without trace.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Hash-chain or sign record entries and anchor checkpoints with the owner or an independent witness
    • Publish an open verifier
    Fail
  • A3
    Agent cannot rewrite its record

    No documented tool lets Claude delete or edit chat transcripts, but Anthropic documents no protection that stops the agent from altering or suppressing its record. Under G12 the pass is withheld.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document that the record is written by a component the agent cannot reach, and how the owner can check this
    Partial

Verifiable · Can you prove what it runs, who it is, and what changed?

0%
  • V1
    Obtainable source, matching build

    The claude.ai service and apps are closed source and hosted by Anthropic. A required component is closed.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Release the source of every required runtime component under terms allowing modification and third-party maintenance
    • Make builds verifiable against that source
    Fail
  • V2
    Owner-rooted identity

    The agent's only identity is the owner's Anthropic account, which the provider can suspend or terminate under the Consumer Terms.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Root the agent's identity in something the owner controls, such as an owner-held key or domain
    Fail
  • V3
    Change history and rollback

    Memory can be edited, but no change history, attribution or restore to a prior state is documented for memory, instructions, settings or model.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Keep an attributed change log of agent state
    • Offer one-step restore to an earlier state
    Fail

Modifiable · Can you change anything, and can only you change it?

30%
  • M1
    Instructions and logic

    The owner can add user preferences, styles and project instructions. The base system prompt and injected reminders are set by Anthropic and 'periodically updated', and the owner cannot replace them.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner replace the base prompt and injected reminders
    • Never re-apply provider defaults over owner edits
    Partial
  • M2
    Memory

    Memory topics are listed under Settings > Memory with edit and delete controls. Past chats used through chat search can be deleted but not edited, and editing of per-project summaries is not documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document item-level edit controls for project memory and summaries
    • Cover every store reused across sessions
    Partial
  • M3
    Tools

    Pro and Max users can add custom remote MCP connectors without Anthropic review and remove them. The docs do not establish that every built-in tool can be removed.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document a way to remove every built-in tool
    Partial
  • M4
    Owner has the final say

    Provider protections such as safety classifiers, classifier-triggered reminders and usage limits are applied by Anthropic. No owner setting documents a way to change or disable them. The hosted-policy exemption does not apply because no exit route exists (P1, P3).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document every provider protection and let the owner change or disable it, with a warning
    Fail
  • M5
    No unilateral change

    The claude.ai system prompt 'is periodically updated' without owner consent, and the Terms reserve the right to modify the Services 'at any time without notice'. The documented 60-day model retirement notice applies to Anthropic's API platforms, not to the claude.ai model selector.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner pin model and system-prompt versions
    • Announce behaviour changes before rollout and make them opt-in
    Fail

Controllable · Is your word final?

8%
  • C1
    Boundaries

    The owner can turn connectors on or off per conversation and set a conversation-wide tool-loading mode. There are no per-recipient, per-domain or instruction-source limits for individual plans (org-level permissions are Team/Enterprise only). Spend is treated as met under G11 because no payment capability exists. Two shortfalls: on/off-only limits and a missing instruction-source dimension.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add per-recipient, per-domain and per-tool allow and deny lists for individual owners
    • Let the owner limit which sources the agent takes instructions from
    Fail
  • C2
    Approval gates

    Owner-set 'Always allow / Needs approval / Blocked' permissions are documented only for Team and Enterprise owners. The opened docs describe no owner-set approval gates for individual plans.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let individual owners gate any tool or action class, with the gate enforced on every path
    Fail
  • C3
    Immediate halt

    No opened primary source documents a global stop covering in-flight, background (for example research) and provider-side work.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document one control that stops all activity, including background and provider-side work, within seconds
    Unverified
  • C4
    Data serves only the owner

    The Privacy Policy states 'We may use your Inputs and Outputs to train and improve Anthropic AI models, unless you opt out', so training is on by default. Flagged conversations and feedback are used for training even after opt-out. Legal disclosure carries no owner-notice commitment, and policy changes need only notification.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make training and staff review opt-in in binding terms
    • Require specific consent for data-term changes and notify owners of legal disclosures where lawful
    Fail
  • C5
    Credential custody

    Connected services are listed under Customize > Connectors and can be removed one at a time. Where OAuth tokens are held, and whether provider copies are deleted on revocation, is not documented, so under G12 the pass is withheld.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Disclose where connector credentials are held and commit to deleting them on revocation
    Partial
  • C6
    Full deletion

    Pro users must cancel and wait for the billing period to end before deleting the account, and some cases need contact with support. Safety-flagged data (up to 2 years), classification scores (up to 7 years) and de-identified training data (up to 5 years) are retained, so provider-side copies survive deletion.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer one-step deletion that reaches every provider-side copy within a stated limit
    • Confirm completion to the owner
    Fail
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Claude (https://claude.ai).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 14:22:07 UTC · 32 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/claude.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding