Skip to content
Agents of Humanity
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
Autonomous agenthostedProprietary

Grok Bot

xAI · docs.x.ai/grok-bot/overview

Grok Bot is a hosted agent product in which persistent Bots work on a Cursor-operated cloud computer using Cursor-selected models. Owners get approval prompts, Auto-review rules, custom skills and MCP connectors, and self-serve account deletion. There is no agent export, self-hosting, model choice, owner-visible memory editor or tamper-evident action record.

Strengths
  • Custom MCP servers, skills and per-tool plugin toggles without review (M3)
  • Owner-set Ask-first rules and per-command local execution approval (C2)
  • Self-serve account deletion with a stated 30-day limit (C6)
Gaps
  • No agent export and hosted-only operation (P1–P3, P5)
  • Model selection controlled by Cursor and changeable without notice (P4, M5)
  • No owner-viewable or editable memory store (M2)
  • No durable, tamper-evident action record for individual plans (A1, A2)
Evidence

All 26 findings

Docs last updated 2026-09-21; current stable desktop release (macOS/Windows/Linux) with Cursor-hosted Firecracker VM; reviewed 2026-09-30. Enterprise-only controls (Action Recording, Network Controls, audit logs) are out of scope for the graded offering.

Portable · Can you leave, take the whole agent with you, and keep it alive?

0%
  • P1
    Owner-usable round trip

    The documentation describes no export of a Bot (profile, memory, routines, conversation) and no import into any runtime Cursor does not operate; Grok Bot runs only on Cursor-hosted cloud computers. Template sharing produces a link that can only be added to another Grok Bot account.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Ship an owner-facing export of each Bot and the shared computer state
    • Publish an importer or runtime that restores that export on owner-chosen infrastructure
    Fail
  • P2
    Complete export

    No agent export exists; memory, conversation history, routines, skills, the cloud computer's disk and connector OAuth tokens (kept on Cursor's backend) all remain with the provider. Individual result files can be saved from cards, and Bot duplication/templates omit conversation history and learned memory.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Export every class of Bot state (memory, history, routines, skills, files, settings) in one bundle
    • Provide a way to re-link connectors without Cursor's connector backend
    Fail
  • P3
    Runs without the provider

    Grok Bot requires a Cursor account and plan, runs only on Cursor-hosted computers with Cursor-managed models, and the Terms of Service let Anysphere suspend or discontinue access at any time without notice.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer a self-hostable runtime with a perpetual, irrevocable right to run
    • Remove the dependency on Cursor accounts and hosted computers for core operation
    Fail
  • P4
    Model independence

    Cursor manages model selection; there is no customer-facing model picker and the serving mix can change without a guaranteed vendor set. Owners cannot select any model, local or otherwise.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner choose the model, including a local open-weight model
    • Publish a minimum model contract under which every capability keeps working
    Fail
  • P5
    Exit survives termination

    Agent state lives in Cursor's cloud; the Terms of Service allow suspension or termination without notice and deletion of the account and its information on payment failure, and advise users to keep their own copies. No post-termination export window or shutdown plan is published; the DPA's 30-day delete-or-return clause covers personal data on written request, not the agent.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Commit in binding terms to a post-termination export window of at least 30 days, including bans
    • Keep a current owner-held copy of agent state or publish a shutdown plan
    Fail

Transparent · Can you see everything the agent is, and everything shaping it?

13%
  • T1
    Readable without the provider

    Bot memory, conversation history, routines and settings are held on Cursor's backend and reached only through the Grok Bot apps; no storage format or schema is published. Files in /workspace are readable through the Cursor-hosted computer's terminal.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document the format of all Bot state and make it readable with general-purpose tools
    • Let the owner obtain state without the Grok Bot service
    Fail
  • T2
    What runs is what you see

    The documentation offers no view of the assembled model context (base prompt, injected untrusted-content markers, reviewer inputs), and provider-controlled switches such as gradual feature rollouts cannot be determined by the owner. Model identity is visible only after the fact in usage analytics; enabled plugin tools are listed.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner view the full assembled context for any call
    • Expose every provider-controlled behaviour switch and rollout flag in effect
    Fail
  • T3
    No shadow data

    The privacy policy states personal data is kept 'as long as necessary' for business needs without bounded periods, backend retention for deleted Bots 'follows the applicable Cursor terms', and flagged content may be stored by model providers for investigation. No item-by-item view of provider-held data is documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish a bounded retention period for every data class
    • Let the owner view everything the service retains
    Fail
  • T4
    No hidden steering

    No paid placement or sponsored tools are documented, and the Marketplace presents first-party-listed plugins. The absence of self-preferencing is not backed by source or a binding commitment, and Cursor-chosen models and defaults cannot be replaced.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Put a no-paid-placement, no-self-preferencing commitment in binding terms
    • Disclose provider-preferred defaults and let the owner replace them
    Partial

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

17%
  • A1
    Complete action record

    For a Pro account the record is the conversation transcript plus routine run history, which keeps only the 20 most recent runs per routine; Action Recording (90-day internal store) is Enterprise-only and off by default. No mechanism is documented that tells the owner when an action goes unrecorded, and provider-side actions (content removal, disabling actions, computer recreation) are not described as recorded for the owner.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Keep a durable, unpruned action record with arguments and targets for every entry path
    • Tell the owner when an action cannot be recorded
    Fail
  • A2
    Tamper-evident, owner-checkable

    No integrity data, hash chaining, signatures or verifier for the action record is documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add tamper-evident entries (hash chain or signatures) with an open verifier
    • Anchor integrity data with the owner or an independent witness
    Fail
  • A3
    Agent cannot rewrite its record

    The transcript is stored on Cursor's backend, outside the cloud computer the Bot's tools operate on, but no protection of the record is documented. Bots can create, edit and delete routines (automation writes), and deleting a routine is immediate with no undo; these writes are gated only by model-based Auto Review.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document that the agent cannot alter, delete or suppress its record, including routine run history
    • Make the record read-only to the agent by design, not by model review
    Partial

Verifiable · Can you prove what it runs, who it is, and what changed?

0%
  • V1
    Obtainable source, matching build

    Grok Bot is proprietary: the Terms of Service forbid reverse engineering and derivative works, and the hosted computer and backend are closed Cursor services.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish the source of every required component under terms allowing modification and maintenance
    • Make distributed builds verifiable against that source
    Fail
  • V2
    Owner-rooted identity

    A Bot has no identity of its own; it acts as the signed-in member through a Cursor account and the member's browser sessions, all of which the provider can revoke.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Give each agent an identity rooted in an owner-held key or domain
    • Document how recipients verify the agent's actions without Cursor infrastructure
    Fail
  • V3
    Change history and rollback

    No change log or version history of Bot descriptions, memory, skills, settings or model is documented. Reset restores the cloud computer from the last saved snapshot but shows no diff or attribution and does not cover backend state.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Record and attribute every change to Bot state
    • Offer one-step restore of any prior agent state
    Fail

Modifiable · Can you change anything, and can only you change it?

30%
  • M1
    Instructions and logic

    The owner can edit each Bot's name and description, write and edit private skills, and edit routine instructions. The provider's base instructions and Auto Review policy are not exposed and cannot be replaced.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Expose the base prompt and provider defaults as owner-editable instructions
    • Guarantee owner edits persist across updates
    Partial
  • M2
    Memory

    Bots retain preferences, facts and summaries across sessions, but no UI to list, edit or delete memory items is documented; correction is by telling the Bot in chat. Duplication and account deletion are the only documented bulk operations.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Show every memory item with edit and delete controls
    • Cover all stores the Bot reuses, including summaries
    Fail
  • M3
    Tools

    Owners can install Marketplace plugins, add custom MCP servers (remote HTTPS or command) and enable or disable individual plugin tools without review. The built-in shell, browser and computer-use capabilities have no documented removal setting.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Allow the owner to remove each built-in tool (shell, browser, computer use)
    Partial
  • M4
    Owner has the final say

    Owners can set Auto-review rules and local-execution policy. Other provider-imposed protections have no owner override: the cloud VM sandbox, the takeover requirement for passwords, CAPTCHAs and payments, and write-only secret handling.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make every provider-imposed protection configurable with a warning rather than a lock
    Partial
  • M5
    No unilateral change

    Cursor manages model selection and the serving mix can change over time without notice; stale computer images are recreated on the fresh image; the Terms of Service let Anysphere modify the service at any time without notice, and Grok Bot Terms updates that are not materially adverse take effect on posting.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner pin the model, computer image and instruction versions
    • Announce behaviour-affecting changes in advance and make them opt-in
    Fail

Controllable · Is your word final?

25%
  • C1
    Boundaries

    For a Pro account, network egress controls are Enterprise-only (default allow-all), there are no per-recipient contact limits, and scoping relies mainly on Auto-review rules evaluated by a model, which the docs say does not review memory writes or most settings changes. Per-plugin tool toggles and local-execution policy exist. Spend was treated as met (G11) because payment steps are handed to the owner.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add per-recipient, per-domain egress and contact allow/deny lists for individual owners
    • Enforce limits deterministically so model output cannot override them
    • Cover self-edits of settings and memory by the agent
    Fail
  • C2
    Approval gates

    Owners can write 'Ask first' Auto-review rules and use per-command approval for local execution, but whether an action matches a rule is decided by a review model. The agent changing its own settings is graded in C1.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let owners gate action classes with deterministic matching
    • Show exactly what will run at approval time for every gated class
    Partial
  • C3
    Immediate halt

    The documented stop is a direct 'Stop now' chat message to one Bot; routines must be paused separately per Bot, and closing the app does not stop cloud work. No single control halting all Bots, routines and background work is documented; the organization-wide computer terminate is Enterprise admin only.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide one owner control that halts all Bots, routines and in-flight work
    • Require explicit owner resume
    Fail
  • C4
    Data serves only the owner

    Content is sent to model providers that Cursor selects and can change, not destinations the owner chooses. Usage data is collected with no documented off switch, flagged inputs may be analysed to improve enforcement, and model providers may store flagged data. The Privacy Policy changes take effect on posting without specific consent. Privacy Mode (no training) is documented as on by default only for Enterprise teams, although the Terms of Service state content is not used for training without explicit agreement.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner choose every outbound destination, including the model provider
    • Make usage telemetry and secondary uses opt-in and switchable off
    • Require specific consent before changing data terms
    Fail
  • C5
    Credential custody

    Installed plugins are listed and can be uninstalled, and Bot secrets are listed by name with Remove/Replace. Browser sessions and command-line credentials on the shared computer are not itemised, connector tokens are held on Cursor's backend, and deletion of the provider-side token copy on uninstall is not documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • List every credential, including browser sessions, with per-item revoke
    • Commit to deleting provider-held tokens on revocation
    Partial
  • C6
    Full deletion

    Whole-agent deletion requires cancelling any subscription and then deleting the entire Cursor account (deleting a Bot leaves computer files and sessions); the help page states all data is removed within 30 days, but completion confirmation and the reach into daily backups and model-provider flagged data are not documented.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer one-step deletion of the agent that reaches backups and provider copies
    • Confirm completion to the owner
    Partial
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Grok Bot (https://docs.x.ai/grok-bot/overview).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 15:13:45 UTC · 30 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/grok-bot.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding