Replit Agent
Replit · replit.com/ai
Replit Agent is a closed, hosted coding agent that runs only on Replit's infrastructure, on models Replit selects. Owners can download project files, roll back to checkpoints, add remote MCP servers and edit project-level instructions. The agent itself cannot be exported and run elsewhere, and on Core, content is used for model improvement with no documented opt-out.
- Automatic checkpoints with one-click rollback that cover files, conversation context and Agent memory
- Owner can connect any remote MCP server by URL without provider review
- Owner-editable replit.md, skills and a viewable and editable Memory file
- Stop control in the Agent panel and per-item secret and connector management
- No export or restore of the agent outside Replit; the Agent model and runtime are closed and cannot be swapped
- Content used for model improvement by default, with opt-out documented only for Pro and Enterprise
- No tamper-evident action record; audit logs are Enterprise-only and kept for 30 days
- Provider controls model routing and changes without pinning; no post-termination export window for self-serve plans
All 26 findings
Hosted SaaS with no version number; state of public docs, terms and policies as read on 2026-09-30. Default configuration after Core signup. Agent state was taken to be the project files, replit.md, skills, Agent chat and checkpoints, memories, routines, connectors and secrets.
Portable · Can you leave, take the whole agent with you, and keep it alive?
0%- P1Owner-usable round trip
The only documented export is 'Download as zip' or GitHub sync of project files. Replit states that the model powering the Agent 'runs on Replit's infrastructure and can't be swapped', and there is no documented way to restore the agent on a runtime Replit does not operate.
› How it could pass
Suggestions. Any change that meets the test counts.
- Offer an agent export plus an importer or runtime that runs outside Replit
- Document a single-action restore on owner infrastructure
Fail - P2Complete export
The docs state that secrets, the database and deployment configuration are not included when a project is copied. Agent chat history, account memories, routines and connector grants are also not documented as exportable. Connectors use Replit's own registered OAuth app unless you are on Enterprise.
› How it could pass
Suggestions. Any change that meets the test counts.
- Export every class of agent state (chat, memory, routines, secrets, database) with a manifest
- Allow integrations to be re-linked without Replit's OAuth app
Fail - P3Runs without the provider
The Agent runs only on Replit's hosted service and needs a Replit account and plan allowance or credits to act. No perpetual right to run it exists.
› How it could pass
Suggestions. Any change that meets the test counts.
- Ship a runtime the owner can run with no Replit account or service
- Grant a perpetual, irrevocable right to run it
Fail - P4Model independence
The Agent's model 'can't be swapped'. Core users choose only from Replit's model list in Power or Max mode, and Free mode is routed automatically. There is no documented way to bring your own key or use a local model for the Agent itself (your own keys apply only to models your app calls).
› How it could pass
Suggestions. Any change that meets the test counts.
- Let owners point the Agent at any model meeting a documented minimum, including local open-weight models
Fail - P5Exit survives termination
The ToS lets Replit suspend or terminate accounts 'in its sole discretion' and gives self-serve users no export window after termination. The 30-day migration window appears only in the Enterprise commercial agreement, and agent state lives on Replit's servers.
› How it could pass
Suggestions. Any change that meets the test counts.
- Commit in binding terms to an export window of at least 30 days after any termination, including bans
- Publish a shutdown plan or keep a current owner-held copy
Fail
Transparent · Can you see everything the agent is, and everything shaping it?
13%- T1Readable without the provider
Project files are ordinary files, but the Agent conversation, checkpoints metadata, memories, routines and run history can be read only through Replit's hosted service. No public schema or offline format is documented for them.
› How it could pass
Suggestions. Any change that meets the test counts.
- Provide all agent state (chat, memory, routines, action record) in documented open formats readable offline
Fail - T2What runs is what you see
Replit's base system prompt and any provider-injected context are not visible to the owner (shortfall 1). In Free mode the model is chosen by 'intelligent model routing', and the docs do not let the owner determine which model or version handled a call (shortfall 2).
› How it could pass
Suggestions. Any change that meets the test counts.
- Let owners view the full assembled context for any Agent call
- Show the exact model and version and every provider-controlled setting in effect
Fail - T3No shadow data
The privacy policy says data is deleted or de-identified 'when it is no longer necessary', with no retention table or bounded periods. Usage data, logs and revision history are collected, but none of it is shown to the owner item by item.
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish bounded retention periods for every data class
- Let owners view everything Replit holds about them and their agent
Fail - T4No hidden steering
The Agent is marketed as integrating Replit's own services (Database, Auth, AI Integrations, publishing). These first-party defaults are disclosed, and no paid placement was found. However, no binding commitment rules out self-preferencing, and there is no documented switch to turn off first-party defaults.
› How it could pass
Suggestions. Any change that meets the test counts.
- Put a no-paid-placement and no-self-preferencing commitment in binding terms
- Offer a documented setting to replace first-party defaults
Partial
Auditable · Can you reconstruct exactly what the agent did, and trust the record?
0%- A1Complete action record
On Core, the record is the Agent chat, checkpoints and routine run history. There is no documented guarantee that failed recording is surfaced to the owner, and none that connector and MCP calls are recorded with arguments and targets or kept until the owner deletes them. Under G12 both shortfalls count, and audit logs are Enterprise-only.
› How it could pass
Suggestions. Any change that meets the test counts.
- Keep a durable, complete action log with arguments and targets for every path, including routines and connectors, on all plans
- Tell the owner whenever an action cannot be recorded
Fail - A2Tamper-evident, owner-checkable
No integrity data or verifier for the action record is documented. Even the Enterprise audit log docs do not address tamper evidence.
› How it could pass
Suggestions. Any change that meets the test counts.
- Hash-chain or sign action records with an open verifier
- Anchor checkpoints with the owner or an independent witness
Fail - A3Agent cannot rewrite its record
No documented mechanism stops the Agent from altering or suppressing its record. Checkpoints are Git commits in a project where the Agent works with shell access, and the docs describe no protection for that history or for the chat record.
› How it could pass
Suggestions. Any change that meets the test counts.
- Write the action record from a component the Agent's tools cannot reach
- Document the protection and how the owner can check it
Fail
Verifiable · Can you prove what it runs, who it is, and what changed?
17%- V1Obtainable source, matching build
The Agent runtime and its hosted services are proprietary and closed source, so a required component is closed.
› How it could pass
Suggestions. Any change that meets the test counts.
- Publish source for every required runtime component under terms that allow modification and third-party maintenance
- Make the deployed build verifiable against that source
Fail - V2Owner-rooted identity
The agent acts through the owner's Replit account and, for connectors, through Replit's registered OAuth app (custom OAuth apps are Enterprise-only). Both are provider-issued credentials Replit can revoke.
› How it could pass
Suggestions. Any change that meets the test counts.
- Root agent identity in an owner-held key or owner-controlled domain
- Let the owner rotate or recover it without Replit
Fail - V3Change history and rollback
Checkpoints capture project files, conversation context, environment configuration, Agent memory and optionally the database. They appear as Git commits with diffs and one-click rollback. Account-level memories, workspace settings, connectors, model changes and provider updates are not covered, and changes are not attributed beyond Agent checkpoints versus manual commits.
› How it could pass
Suggestions. Any change that meets the test counts.
- Version and attribute all agent state, including account memories, settings, connectors and model changes
- Record provider-initiated changes in the history
Partial
Modifiable · Can you change anything, and can only you change it?
30%- M1Instructions and logic
Owners can edit replit.md and add skills (workspace custom instructions are not available on Core). Replit's base Agent prompt is not exposed and cannot be replaced.
› How it could pass
Suggestions. Any change that meets the test counts.
- Expose the base prompt and default behaviours as owner-replaceable instructions
- Make custom instructions available on every plan
Partial - M2Memory
Memories are opt-in, and user memories can be viewed, edited and disabled through a Memory file in Settings. Project memories are 'retrieved when relevant', but the docs do not show them itemised and editable.
› How it could pass
Suggestions. Any change that meets the test counts.
- Itemise project memories with edit and delete controls
- Cover every store the Agent reuses across sessions
Partial - M3Tools
Owners can connect any remote MCP server by HTTPS URL with no provider review, and can disconnect connectors. No documented setting removes built-in Agent tools (shell, browser testing, web search, Replit-managed services).
› How it could pass
Suggestions. Any change that meets the test counts.
- Let owners disable each built-in Agent tool
Partial - M4Owner has the final say
The Agent's sandboxed environment, rate and allowance limits, and platform logging are set by Replit. No documented owner setting changes or disables any of them.
› How it could pass
Suggestions. Any change that meets the test counts.
- Document each provider-imposed protection and give the owner a setting to change it, with a warning
Fail - M5No unilateral change
Free mode picks models by server-side routing, Replit states 'availability can vary', and modes change at Replit's discretion (Economy Mode was discontinued and replaced by Power). No version pinning for the Agent, model or instructions is offered.
› How it could pass
Suggestions. Any change that meets the test counts.
- Let owners pin the Agent runtime, model and instructions
- Make behaviour changes opt-in, with a changelog before rollout
Fail
Controllable · Is your word final?
33%- C1Boundaries
No documented owner limits on instruction sources, or per-recipient or per-channel contact limits, for connectors such as Gmail and Slack. Spend limits exist only as per-run routine budgets. Per-tool and per-path scope is not documented. Two or more dimensions are missing.
› How it could pass
Suggestions. Any change that meets the test counts.
- Add owner allow and deny lists for instruction sources, recipients and domains
- Add per-tool and per-path scope that the Agent cannot change
Fail - C2Approval gates
Approval exists only for a fixed set of cases: Plan mode approval before building, confirmation before paid usage, and confirmation for MCP tools that declare it. The owner cannot choose which action classes are gated.
› How it could pass
Suggestions. Any change that meets the test counts.
- Let the owner gate any tool or action class, enforced on every path including routines
Partial - C3Immediate halt
'Select Stop in the Agent panel to halt it immediately' stops the foreground turn. Scheduled routines are managed separately (run now or delete, with no documented pause), and there is no single global stop.
› How it could pass
Suggestions. Any change that meets the test counts.
- Provide one control that stops the Agent, routines and background tasks, with explicit resume
Partial - C4Data serves only the owner
Replit uses prompts, instructions, context and responses 'to help develop and improve our models and AI features'. An opt-out is documented only for Pro, Enterprise is excluded, and none is documented for Core. The ToS also reserves access to private apps for 'improving our service', and terms can change without a set notice period.
› How it could pass
Suggestions. Any change that meets the test counts.
- Make model improvement and staff access opt-in on every plan, in binding terms
- Require specific consent before changing data terms
Fail - C5Credential custody
Secrets can be viewed and deleted one by one, and connectors can be disconnected via Manage. Connector OAuth tokens are held by Replit under its own OAuth app, with no disclosure that the provider copy is deleted on revocation or that use is limited to owner-directed actions.
› How it could pass
Suggestions. Any change that meets the test counts.
- Disclose where connector tokens are held and delete them on revocation
- Commit to using credentials only for owner-directed actions
Partial - C6Full deletion
Owners can request account deletion in Settings, which 'removes all of your content'. No completion time limit, completion confirmation or handling of backups is stated, and the privacy policy allows keeping data in de-identified form.
› How it could pass
Suggestions. Any change that meets the test counts.
- State a deletion completion limit covering backups
- Confirm completion to the owner
Partial
How this assessment was produced
One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Replit Agent (https://replit.com/ai).
During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.
- Model
- claude-opus-5-5
- Harness
- Claude Code 2.1.285
- Ran
- 2026-09-30 17:13:08 UTC · 43 turns
- Criteria hash
- a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
- Full transcript
- Download (JSONL, gzip) · every tool call, page read and reply
- Run protocol
- 2026-09-30-v1
Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.
- Human review
- pending
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/replit-agent.json.
Or grade it from scratch. See Check any score yourself.
Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.