Skip to content
Agents of Humanity
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
Autonomous agenthybridApache-2.0 (CLI); app, IDE extension and cloud proprietary

OpenAI Codex

OpenAI · github.com/openai/codex

Codex CLI is an Apache-2.0 local agent. Its configuration, instructions, session transcripts and memories are kept as files under CODEX_HOME on the owner's machine, and it can run against local models without an OpenAI account. On the default ChatGPT sign-in path, OpenAI's backend supplies model catalogue data and hosted models, receives usage analytics by default, and under the consumer terms may use content for training unless the owner opts out. The action record has no tamper evidence and the agent has no owner-rooted identity.

Strengths
  • Local-first state in documented files under CODEX_HOME (P5)
  • Apache-2.0 runtime that runs with local models and no OpenAI account (P3)
  • Owner can switch off the sandbox, approvals and analytics through config (M4)
Gaps
  • Content may be used for training by default on consumer plans; analytics on by default (C4)
  • Action record has no integrity data; hook runs are not recorded; write failures only go to a log (A1, A2)
  • No owner-rooted agent identity (V2)
  • Provider retention is open-ended and account deletion has retention carve-outs (T3, C6)
Evidence

All 26 findings

Release rust-v0.159.2, commit ff6aec96948b70d94983af2641a6b67c94faeff5 (2026-09-29), Linux/macOS terminal client, default config after first run with the folder trusted (workspace-write sandbox, on-request approvals, reviewer = user). Codex cloud, IDE/desktop apps and Business/Enterprise plans are out of scope. Evaluated 2026-09-30.

Portable · Can you leave, take the whole agent with you, and keep it alive?

70%
  • P1
    Owner-usable round trip

    All agent state is always on disk under the documented CODEX_HOME (~/.codex), and the docs describe pointing Codex at another home directory, so export is immediate. Restoring elsewhere means installing Codex, copying the directory by hand and signing in again; there is no single import action.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Ship a `codex export` / `codex import` pair, or a documented one-step restore of a CODEX_HOME bundle
    Partial
  • P2
    Complete export

    CODEX_HOME holds config.toml, AGENTS.md, skills, session transcripts (JSONL, optionally zstd-compressed), memories, the state DB and local credentials. ChatGPT Apps connectors enabled by default (feature `apps`) are linked through OpenAI's hosted service, so those integrations can only be re-linked through a provider-registered client.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let every integration be re-linked with owner-held OAuth clients or MCP servers
    • Publish a manifest of every state class in CODEX_HOME
    Partial
  • P3
    Runs without the provider

    The runtime is Apache-2.0, and OSS providers (Ollama, LM Studio) are defined with requires_openai_auth: false, so the login screen is skipped and no OpenAI account, licence key or server is needed to start or act. The update check goes to GitHub or Homebrew and can be switched off (check_for_update_on_startup). Hosted-only tool loss is graded in P4.

    Pass
  • P4
    Model independence

    Local open-weight models are supported through the documented `--oss` / `--local-provider` flags and custom providers. Some capabilities work only with OpenAI's hosted models: web search (supports_standalone_web_search is false for OSS providers), image generation, realtime voice, and remote compaction (only enabled for the OpenAI provider).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer runtime-executed fallbacks (for example MCP-based search, local compaction) for every hosted-only capability
    • Publish and test a minimum model contract against a local open-weight model
    Partial
  • P5
    Exit survives termination

    Agent state lives locally under CODEX_HOME by default, so a current copy stays with the owner through account suspension, a lapsed plan or provider shutdown.

    Pass

Transparent · Can you see everything the agent is, and everything shaping it?

38%
  • T1
    Readable without the provider

    Transcripts, config, memories and instructions are JSONL, TOML or Markdown, with types in open source. With the default OpenAI provider, compaction output (ContextCompaction) and reasoning items are stored as `encrypted_content` blobs that only OpenAI can decrypt.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Store a plaintext form of every compaction summary alongside or instead of provider-encrypted blobs
    • Publish schemas for rollout and state-DB files
    Partial
  • T2
    What runs is what you see

    Base instructions, developer messages and settings are recorded in the session transcript (SessionMeta.base_instructions, ThreadSettingsApplied), and /status and /debug-config show the effective configuration. With remote compaction the compaction summary placed in context is an encrypted blob the owner cannot read (one shortfall). Server-delivered catalogue settings are cached locally but not shown in the UI, which is recorded here but not counted.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Expose the plaintext of compaction summaries placed in context
    • Show server-delivered model catalogue settings (instruction templates, tool mode, guardian policy) in /status
    Partial
  • T3
    No shadow data

    Requests use store:false, but analytics events go to OpenAI by default, and the Privacy Policy keeps Personal Data 'as long as we need… or for other legitimate business purposes' with further retention after deletion for safety, legal or fraud reasons. Retention is therefore not bounded, and held items are not visible to the owner one by one.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish a bounded retention table for every data class held for Codex users
    • Let the owner view analytics and abuse-monitoring data held about them
    Fail
  • T4
    No hidden steering

    Tool suggestions (feature `tool_suggest`) and the remote plugin catalogue (`remote_plugin`) are on by default, surface OpenAI-catalogued plugins and connectors, and can be switched off in config. No evidence of paid placement was found, but no binding commitment rules it out.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Put a no-paid-placement commitment for Codex suggestions and catalogues into binding terms
    Partial

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

17%
  • A1
    Complete action record

    Transcripts record tool calls with arguments and outputs, but HookStarted and HookCompleted events are classed as non-durable and not persisted (hook entry path unrecorded). When a transcript write fails, persist_rollout_items only writes an error to the tracing log while the session continues, so the owner is not told (silent recording failure).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Persist hook executions with their commands and results
    • Tell the owner in the UI (or stop acting) when the action record cannot be written
    Fail
  • A2
    Tamper-evident, owner-checkable

    Session transcripts are plain JSONL or zstd files with no hash chain, signatures or external checkpoints, so edits or deletions leave no trace.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Hash-chain or sign transcript entries and publish an open verifier
    • Anchor checkpoints with the owner
    Fail
  • A3
    Agent cannot rewrite its record

    Transcripts are written by the Codex process to ~/.codex/sessions, which is outside the default workspace-write roots, and `.codex` stays read-only inside writable roots. MCP servers are launched with `sandbox: None`, so a tool path outside the sandbox could still alter the record.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make the record unreachable from every agent-driven process, including MCP servers and escalated commands
    Partial

Verifiable · Can you prove what it runs, who it is, and what changed?

33%
  • V1
    Obtainable source, matching build

    All runtime source is Apache-2.0. The release workflow signs Linux artifacts with Sigstore cosign (keyless, GitHub OIDC), but no documentation was found telling owners how to check the default npm or Homebrew binary against the source, and macOS/Windows builds rely on platform code signing.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document how an owner verifies the installed binary against source (reproducible builds or SLSA provenance) on every platform
    Partial
  • V2
    Owner-rooted identity

    On the default path the agent acts under the owner's ChatGPT account. The agent-identity module registers agent keys with OpenAI's backend and trusts JWTs issued by chatgpt.com/codex-backend/agent-identity, so the identity is provider-issued and revocable.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Root the agent identity in an owner-held key or owner-controlled domain
    • Document how recipients verify the agent's actions without OpenAI infrastructure
    Fail
  • V3
    Change history and rollback

    Per-turn settings changes are recorded in transcripts (ThreadSettingsApplied), conversations can be rolled back or forked, and the memory workspace keeps a git baseline with a diff file. There is no attributed history or restore for config.toml, AGENTS.md, skills or permissions.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Keep an attributed change log with one-step restore for config, instructions, skills and permissions
    Partial

Modifiable · Can you change anything, and can only you change it?

60%
  • M1
    Instructions and logic

    The base prompt can be replaced with model_instructions_file, developer instructions and the compaction prompt are configurable, and several injected blocks can be toggled. Other behaviour-defining templates (memory pipeline prompts, review and guardian prompts, tool descriptions) are compiled into the binary and can only be changed by editing source (G3).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Expose every injected template and default skill as an owner-editable file or setting
    Partial
  • M2
    Memory

    Memories are off by default. When enabled, consolidated memories are editable Markdown under ~/.codex/memories, but per-thread stage-1 raw memories that feed consolidation are kept in the SQLite state DB with no itemised view or editor in the normal interface.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Itemise and allow editing of stage-1 memory records, or keep them only in the Markdown workspace
    Partial
  • M3
    Tools

    Owners can add any MCP server or tool without review and scope tools with enabled_tools/disabled_tools. Web search, update_plan, shell and multi-agent can be switched off, but no documented setting was found to remove apply_patch or view_image.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide a documented switch to remove every built-in tool
    Partial
  • M4
    Owner has the final say

    The owner can switch off the sandbox (danger-full-access), approvals (never), analytics, feedback and history persistence through documented config or flags. As the owner is also the administrator, requirements.toml constraints are owner-editable.

    Pass
  • M5
    No unilateral change

    With no model pinned, the default model and per-model instruction templates, tool mode and guardian policy come from the backend /models catalogue, which is refreshed by ETag. Owners can pin these with `model`, model_catalog_json and model_instructions_file, and runtime updates are opt-in prompts. The hosted model behind a slug cannot be pinned: the server can reroute requests to another model (a warning is shown when this happens), and retirements for ChatGPT sign-in are announced with no stated notice period.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer dated, pinnable model versions for ChatGPT sign-in with at least 6 months' retirement notice
    • Make server-side reroutes opt-in or announced before they apply
    Partial

Controllable · Is your word final?

25%
  • C1
    Boundaries

    File scope (writable roots, protected paths) and per-tool scope are supported. Network access in the supported configuration is on/off only; per-domain allow/deny needs the network proxy, which is an Experimental feature (G10). MCP servers run outside the sandbox (`sandbox: None`), so file and network limits do not bind that path. Spend is treated as met under G11 (no payment capability).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Ship per-domain egress allow/deny lists as a supported feature
    • Apply sandbox limits to MCP servers and hooks
    Fail
  • C2
    Approval gates

    Owners can gate commands (execpolicy prefix rules with `prompt`), MCP and app tools per tool (approval_mode), and all escalations (approval_policy), with approvals going to the user by default. Hosted web search, on by default in cached mode, can be switched off but not gated per call.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Allow approval gating of hosted web search and every other built-in tool
    Partial
  • C3
    Immediate halt

    Esc/Ctrl+C interrupts the foreground turn, but background terminals need a separate `/stop` ('stop all background terminals'), and the local background daemon is managed separately (/daemon).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide one control that stops the turn, subagents, background terminals and the daemon together
    Partial
  • C4
    Data serves only the owner

    The consumer Terms and Privacy Policy allow OpenAI to use Content to train models unless the user opts out, and Codex docs say chat content 'may be used to improve OpenAI models if allowed by your ChatGPT data controls'. Analytics events go to OpenAI by default ([analytics] enabled defaults to true), and the Terms let non-material changes take effect on posting.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make training use opt-in for Codex content
    • Turn analytics off by default
    • Require specific consent for data-term changes
    Fail
  • C5
    Credential custody

    Local credentials are in auth.json or the OS keyring, disclosed in the auth docs and revocable with `codex logout` and per-server MCP logout. Grants for ChatGPT Apps connectors are held by OpenAI, and the public evidence I could open does not establish per-item visibility, immediate revocation or deletion of provider copies.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • List and revoke connector grants from Codex, and commit to deleting provider-side copies on revocation
    Partial
  • C6
    Full deletion

    There is no single action that deletes the whole agent: local state must be removed by hand and the ChatGPT account deleted separately. The Privacy Policy removes deleted data within 30 days 'unless we need to retain it for longer' for legal, fraud, security or financial reasons, so provider-side copies may survive.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer one-step deletion covering local state and every provider-side copy within a stated limit, with confirmation
    Fail
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of OpenAI Codex (https://github.com/openai/codex).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 17:01:56 UTC · 65 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/openai-codex.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding