Skip to content
Agents of Humanity
About
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
AssistanthybridProprietary

Siri

Apple · apple.com/newsroom/2026/09/siri-ai-a-profoundly-more-capable-and-personal-assistant-is-here

Siri AI is Apple's closed, hosted assistant that runs on Apple devices, Apple Foundation Models and Apple-operated Private Cloud Compute. Public documentation shows owner controls over conversation retention, per-app access and some content restrictions, but no export, no model choice, no action record, and default storage and review of Siri transcripts by Apple.

Portable0
Transparent13
Auditable0
Verifiable0
Modifiable20
Controllable25
Strengths
  • Owner can add tools without review via Shortcuts and limit per-app access (M3 partial)
  • Screen Time lets the owner allow or block some Siri AI capabilities (M4 partial)
  • Confirmation prompts for messages, ChatGPT hand-off and payments (C2 partial)
  • Per-app Siri access and ChatGPT link are individually revocable (C5 partial)
Gaps
  • No export or provider-independent runtime; locked to Apple models (P1–P5 fail)
  • No durable, tamper-evident action record (A1–A3 fail)
  • Transcripts stored, reviewed and used for model training by default (C4, T3 fail)
  • Apple can change or disable features without notice; no pinning (M5 fail)
Evidence

All 26 findings

iOS 27 release (Siri AI beta launched 2026-09-14), iPhone 15 Pro / iPhone 16 or later, US region. Desk review of public Apple documentation, legal pages and support guides only; no device, account or service was used. Optional ChatGPT extension (off by default) and Audio Intelligence features (Apple Watch, coming later) considered only where they bear on defaults.

Portable · Can you leave, take the whole agent with you, and keep it alive?

0%
  • P1
    Owner-usable round trip

    No export of the Siri AI agent (conversations, personalization, settings) is documented, and Siri AI runs only on Apple devices using Apple Foundation Models on-device and on Apple-operated Private Cloud Compute; there is no restore route on infrastructure Apple does not operate.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer an owner-usable export of the whole Siri agent
    • Publish an importer or a runtime not operated by Apple that accepts it
    Fail
  • P2
    Complete export

    Apple's data-copy service covers Apple Account and iCloud data but excludes data stored end-to-end encrypted or not stored by Apple; Siri conversation history, Siri personalization and on-device personal context are not listed as exportable, so several classes of agent state stay inside Apple's software.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Include conversations, personalization, memory, settings and app grants in one export
    • Export source data alongside any derived index (e.g. Spotlight/personal context)
    Fail
  • P3
    Runs without the provider

    Siri AI needs Apple Foundation Models on Apple-operated Private Cloud Compute and an Apple Account; the Usage Terms let Apple 'change, suspend, remove, or disable access to any part of Apple Intelligence Features at any time without notice'. iOS is licensed under Apple's software licence, not an irrevocable right to run the agent independently.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Grant a perpetual, irrevocable right to run the agent without Apple services
    • Make server-side model endpoints replaceable by the owner
    Fail
  • P4
    Model independence

    The agent is built on Apple Foundation Models (developed with Google Gemini); the only other model route documented is the Apple-integrated ChatGPT extension. The owner cannot select a local open-weight or other third-party model.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document a model interface that accepts any model meeting a published minimum, including local open-weight models
    Fail
  • P5
    Exit survives termination

    The iCloud terms state that on termination the owner 'may lose all access' to data stored in iCloud (where Siri conversations sync), and Apple may terminate without prior notice for listed causes; the Usage Terms allow suspension or termination of Apple Intelligence access. No post-termination export window or shutdown plan for Siri state is published.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Commit in binding terms to a ≥30-day export window after any termination, including bans
    • Keep a complete, owner-usable copy of agent state on the owner's side
    Fail

Transparent · Can you see everything the agent is, and everything shaping it?

13%
  • T1
    Readable without the provider

    Siri conversation history, personalization and personal-context indexes are stored in undocumented formats readable only through Apple's closed Siri app and OS; under default Standard Data Protection the keys for several iCloud categories are held by Apple.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish schemas for Siri state and make it readable with general-purpose tools
    • Let the owner hold all keys
    Fail
  • T2
    What runs is what you see

    Neither the base instructions nor the assembled context sent to the model are viewable; the Apple Intelligence & PCC Report (off by default) logs requests sent to PCC but not system prompts. The exact model identity/version serving a request and server-side behaviour switches are not documented. Both shortfalls apply.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner view the full assembled context, including Apple's base prompt
    • Show the exact model version and every provider-side switch in effect
    Fail
  • T3
    No shadow data

    By default Apple stores Siri transcripts and related request data for up to two years, and a reviewed subset 'may be kept beyond two years' with no stated bound; the owner cannot view these server-side items one by one (they are keyed to a rotating identifier).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Bound retention of every data class, including reviewed transcripts
    • Let the owner see each item Apple retains
    Fail
  • T4
    No hidden steering

    No evidence of paid placement was found, and Apple states Siri data is not used to build a marketing profile, but there is no binding commitment against self-preferencing in Siri's recommendations or tool choices, and the statements are policy text Apple may change.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Put a no-paid-placement / no-self-preferencing commitment in binding terms
    • Disclose and let the owner replace Apple-preferred defaults
    Partial

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

0%
  • A1
    Complete action record

    There is no durable action record: the Siri app keeps conversations (prunable via 'Keep Conversations'), and the Apple Intelligence & PCC Report is off by default and covers only off-device requests. Actions in apps (e.g. messages sent with 'Automatically Send Messages'), Shortcuts automations and proactive suggestions are not documented as recorded.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Record every consequential action from every trigger path, with arguments and targets
    • Alert the owner when an action cannot be recorded
    Fail
  • A2
    Tamper-evident, owner-checkable

    No integrity data or owner-side verifier exists for any action record. Apple's PCC transparency log attests PCC software images, not the owner's action record.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Hash-chain or sign action-record entries and publish an open verifier anchored with the owner
    Fail
  • A3
    Agent cannot rewrite its record

    No documented mechanism stops the agent from altering or deleting its own conversation record; Siri AI can take actions across apps, and Apple documents no separation between the agent's capabilities and its record. Under G12 the protection is not established.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Write the record from a component the agent cannot reach, with read-only agent access
    Fail

Verifiable · Can you prove what it runs, who it is, and what changed?

0%
  • V1
    Obtainable source, matching build

    iOS, the Siri app and Apple Foundation Models are closed. For Private Cloud Compute Apple publishes production binary images and a transparency log but only 'a subset of the security-critical PCC source code', so required components are closed.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make the source of every required component obtainable with the right to modify and maintain
    • Tie distributed builds to that source verifiably
    Fail
  • V2
    Owner-rooted identity

    The agent has no identity separate from the owner's Apple Account, which Apple issues and may suspend or terminate.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Root the agent's identity in an owner-held key Apple cannot revoke
    Fail
  • V3
    Change history and rollback

    No change history exists for Siri settings, personalization or memory, and no rollback; iCloud sync of Siri settings keeps only the current state.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Log attributed changes to settings, memory, tools and model version
    • Offer restore to an earlier state
    Fail

Modifiable · Can you change anything, and can only you change it?

20%

Controllable · Is your word final?

25%
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Siri (https://www.apple.com/newsroom/2026/09/siri-ai-a-profoundly-more-capable-and-personal-assistant-is-here/).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 14:07:32 UTC · 29 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/apple-siri-ai.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding