Skip to content
Agents of Humanity
About
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
AssistanthostedProprietary

Grok

xAI · grok.com

Grok is a closed, provider-hosted assistant with memory, skills, connectors (including custom remote MCP servers) and scheduled automations. The owner can download their data, edit memory, override skills and disconnect connectors individually. There is no restore path outside the provider, no choice of model outside xAI, no pinning of model versions, and conversations are used for training by default.

Portable0
Transparent13
Auditable17
Verifiable17
Modifiable40
Controllable25
Strengths
  • Memory can be viewed, edited and deleted in Settings
  • Custom remote MCP connectors can be added without provider review
  • Built-in skills can be overridden by owner versions
  • Connectors are listed and can be disconnected one by one
Gaps
  • No export-and-restore route off provider infrastructure; runs only on xAI servers with xAI models
  • Content used for training by default; retention justified by 'ongoing legitimate business need' with no fixed period
  • Models behind the Fast/Expert names change without pinning, and the published system prompts stop at November 2025
  • No tamper-evident action record and no global stop for background automations
Evidence

All 26 findings

Hosted service grok.com (web), Grok 4.6 behind the Expert/Heavy modes and 'Fast' as the default mode, per the grok.com client config and release notes dated up to 2026-09-12. Terms of Service last updated 2026-09-11; Privacy Policy effective 2026-08-24. Grok on X and the open-source Grok Build CLI are separate products and were not graded. Desk review, 2026-09-30.

Portable · Can you leave, take the whole agent with you, and keep it alive?

0%
  • P1
    Owner-usable round trip

    The owner can download their data from Settings > Data Controls, but there is no documented importer or runtime outside the provider that restores a working agent from that download.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer an import that restores the exported agent on a runtime xAI does not operate
    • Document the round trip for non-developers
    Fail
  • P2
    Complete export

    No primary source documents what the data download contains. Memory, skills, automations, project configuration, files and connector links are not shown to be included, so under G12 two or more classes stay with the provider.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish a manifest of the export covering memory, skills, automations, projects, files and connectors
    • Allow connectors to be re-linked without the provider's OAuth client
    Fail
  • P3
    Runs without the provider

    The grok.com agent runs only on provider servers under a SpaceXAI account. The Terms let SpaceXAI suspend or discontinue the service 'for any reason … or no reason'.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide a runtime the owner can run without SpaceXAI services or accounts
    • Grant a perpetual, irrevocable right to run it
    Fail
  • P4
    Model independence

    grok.com offers only xAI models (Auto, Fast, Expert and Heavy, based on Grok 4.6). There is no documented way to use another vendor's model or a local model in the grok.com agent.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner configure any model that meets a documented minimum, including local open-weight models
    Fail
  • P5
    Exit survives termination

    The Terms allow termination or account deletion 'at any time without notice', including after 120 days of inactivity. There is no post-termination export window, and no copy of the state is held by the owner by default.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Commit in binding terms to at least a 30-day export window after any termination, including bans
    • Publish a shutdown plan
    Fail

Transparent · Can you see everything the agent is, and everything shaping it?

13%
  • T1
    Readable without the provider

    Agent state (memory, skills, automations, history) lives on provider servers and is readable through the provider's app. The format of the data download is not publicly documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish the schema of the export covering all agent state
    • Include provider-supplied skills and prompts in readable form
    Fail
  • T2
    What runs is what you see

    xAI publishes system prompts on GitHub, but the last commit is 2025-11-17 and there are no prompts for Grok 4.20 through 4.6, skills, connectors or automations. The model behind the default 'Fast' mode and the server-side feature flags cannot be determined by the owner. That is two shortfalls.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner view the full assembled context for any call
    • Show the exact model version and every provider-controlled switch in effect
    Fail
  • T3
    No shadow data

    Data is kept 'where we have an ongoing legitimate business need' with no fixed period. De-identified copies of deleted chats may be kept, and no retention periods are given for safety logs or backups.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish a bounded retention period for every data class, including safety logs, backups and de-identified copies
    • Let the owner see everything retained
    Fail
  • T4
    No hidden steering

    The primary documents do not mention sponsored content or paid placement in Grok responses, but no binding commitment rules it out. First-party X search is a disclosed built-in source, and no switch to turn it off was found.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Commit in binding terms to no paid or self-preferencing influence on responses
    • Let the owner switch off first-party sources such as X search
    Partial

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

17%
  • A1
    Complete action record

    Actions show up only inside chat threads (tool-call accordions) and automation run histories. The client config hides raw tool arguments (hide_tool_json_args: true), the owner can delete chats, and nothing documents alerts when recording fails. That is two or more shortfalls.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Keep a durable, account-wide action log with arguments and targets for every trigger path
    • Tell the owner when an action cannot be recorded
    Fail
  • A2
    Tamper-evident, owner-checkable

    No integrity data or verifier is documented for chat or automation history; the provider-held record can be changed without trace.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Hash-chain or sign action records
    • Anchor checkpoints with the owner or an independent witness and publish an open verifier
    Fail
  • A3
    Agent cannot rewrite its record

    The record is kept by the provider's chat service, outside the agent's code sandbox, but no documentation states that the agent's tools cannot alter or suppress it. Protection rests on architecture that is not documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document that the agent has read-only access to its record and how the owner can check this
    Partial

Verifiable · Can you prove what it runs, who it is, and what changed?

17%
  • V1
    Obtainable source, matching build

    The grok.com app and backend are closed source. The released Grok-1 (Apache-2.0) and Grok-2 (revocable community licence) weights and the Grok Build CLI are not the components that run this agent.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish source for every required runtime component with the right to modify and have others maintain it
    • Make the deployed build verifiable against source
    Fail
  • V2
    Owner-rooted identity

    The agent's only identity is the owner's SpaceXAI account, which the provider can suspend or delete. Outbound actions go through the owner's third-party accounts via provider-held connectors.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Root the agent's identity in an owner-held key or domain
    • Make actions attributable without provider infrastructure
    Fail
  • V3
    Change history and rollback

    Memory update cards show some memory changes during conversations, but there is no change history across instructions, skills, settings or model, no record of who made each change, and no restore of an earlier state.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Log every state change with who made it
    • Offer a one-step restore to an earlier state
    Partial

Modifiable · Can you change anything, and can only you change it?

40%
  • M1
    Instructions and logic

    The owner can set preset or custom response styles, project instructions and override built-in skills. The published prompt template only inserts owner preferences into the provider's base prompt, which cannot be replaced.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner replace the base system prompt
    • Guarantee owner edits persist through updates
    Partial
  • M2
    Memory

    Saved memories can be viewed, edited and deleted in Settings. Personalisation from conversation history and the imported X profile and post history are not shown as individually editable items.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Itemise every persistent store the agent draws on, including history-based personalisation and imported X data, with edit and delete
    Partial
  • M3
    Tools

    The owner can add any remote MCP server without review and choose which connector tools are exposed. Locally hosted MCP servers are rejected, and no documented setting removes built-in tools such as web or X search.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner remove every built-in tool
    • Support owner-hosted tools without restriction
    Partial
  • M4
    Owner has the final say

    Some content settings (mature voices, the Spicy Imagine mode) are adjustable behind an 18+ gate, but content filters and the sandbox are provider-controlled and locked.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document every provider-imposed protection and make each owner-configurable with a warning
    Partial
  • M5
    No unilateral change

    Mode names such as Fast and Expert resolve to models the provider changes without pinning. The Grok 4.5 and 4.6 switches have no release-note entry, and xAI has acknowledged past prompt changes to Grok on X made without notice.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner pin model, prompt and runtime versions
    • Announce behaviour-affecting changes before rollout and make them opt-in
    Fail

Controllable · Is your word final?

25%
  • C1
    Boundaries

    The owner can choose which tools a connector exposes. There are no documented limits on whom the agent takes instructions from (for example email triggers) or whom it contacts via Gmail send. Spend was treated as met under G11, because Grok has no payment capability.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add per-sender and per-recipient allow and deny lists
    • Enforce limits on every path, including automations and email triggers
    Fail
  • C2
    Approval gates

    Release notes mention tool permission buttons for connectors, but there is no documentation that the owner can gate any action class or that every path, including scheduled automations, honours the gate.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner gate any tool or action class
    • Document that automations honour approval gates
    Partial
  • C3
    Immediate halt

    The owner can stop the current response, stop individual background tasks from the tray, and pause or delete individual automations. No single control stops all activity.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide one global stop for in-flight, background and scheduled work that requires the owner to resume
    Partial
  • C4
    Data serves only the owner

    Conversations are used to train models by default (opt-out under 'Improve the Model'), and feedback is used even after opting out. Staff may review content, data may be shared with affiliated companies, legal disclosure comes with no notice commitment, and continued use counts as acceptance of changed terms.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make training and staff review opt-in in binding terms
    • Ask for specific consent before changing data terms
    • Notify owners of legal disclosure where lawful
    Fail
  • C5
    Credential custody

    Connectors are listed at grok.com/connectors and each can be disconnected. Where OAuth tokens are held, whether revocation is immediate, and whether provider-side copies are deleted are not disclosed.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Disclose where credentials are held
    • Commit to immediate revocation and deletion of provider-side token copies
    Partial
  • C6
    Full deletion

    Account deletion is one action and takes up to 30 days, but data may be kept for 'legal, compliance, or safety purposes', de-identified copies may survive, backups are not addressed, and no confirmation is documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make deletion reach every copy, including de-identified data and backups, within a stated limit
    • Confirm completion to the owner
    Fail
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Grok (https://grok.com).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 15:18:46 UTC · 2 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/grok.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding