Skip to content
Agents of Humanity
Public preview. Results are preliminary (criteria v1.0, 30 Sept 2026). Tell us what we got wrong
Autonomous agenthostedProprietary

Lovable

Lovable · lovable.dev

Lovable is a closed, hosted AI app-building agent. Owners can take their generated app code and database with them and get useful controls (per-connector approvals, editable knowledge and skills, code version history), but the agent itself, its model, base instructions, action record and other state stay with Lovable. On the Pro plan, content is used for model training unless the owner opts out.

Strengths
  • Code download, Git sync and SQL database export for the generated app
  • Per-connector and per-Cloud-tool permissions (always / ask / never)
  • Editable project and workspace knowledge and custom skills
  • Code version history with diffs and one-click revert
Gaps
  • Agent runs only on Lovable's service, with a Lovable-chosen model upgraded automatically
  • No export of chat/action history, knowledge or settings, and no post-termination export window
  • Base prompt and model not visible; action record has no tamper evidence
  • Pro content used for training by default, with staff review; deletion leaves training copies
Evidence

All 26 findings

Hosted SaaS, no version pinning available; evaluated from docs.lovable.dev (llms.txt index and page .md sources), lovable.dev/terms and lovable.dev/privacy as fetched on 2026-09-30 (privacy policy updated effective 2026-09-09). The 'agent' is taken to be the Lovable build agent as configured for one owner: knowledge, skills, agent permissions, connectors and secrets, project chat and activity record, version history, project code and Lovable Cloud backend.

Portable · Can you leave, take the whole agent with you, and keep it alive?

0%
  • P1
    Owner-usable round trip

    The owner can download the project code as a zip (paid plans) or sync it to Git, and export Cloud data as an SQL dump that is emailed when ready, but there is no export of the agent itself and no importer: the Lovable agent runs only on Lovable's hosted service, Git import into Lovable is not supported ('You can only export from Lovable to GitHub'), and running the code elsewhere needs manual hosting, dependency and backend setup.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Offer a one-action export of the whole agent (knowledge, skills, chat, settings, code, data) plus a matching import
    • Provide a runtime or importer not operated by Lovable that restores the agent working
    Fail
  • P2
    Complete export

    Exports cover code (including Supabase migrations and edge functions) and a Cloud database SQL dump; project chat and activity history, project and workspace knowledge, agent permission settings and connector credentials have no export (secrets are write-only and gateway connectors authenticate through Lovable's OAuth client), so two or more classes stay with the provider.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Include chat/action history, knowledge, skills and settings in a manifest-backed export
    • Let every integration be re-linked without Lovable's connector gateway or OAuth client
    Fail
  • P3
    Runs without the provider

    The agent is a hosted service that requires a Lovable account; the Terms grant only a 'limited ... revocable license' that 'terminates immediately if ... your account is terminated for any reason'. Exported app code can run elsewhere, but the agent cannot run without Lovable.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Ship a runtime the owner can run without a Lovable account or service
    • Grant a perpetual, irrevocable right to run it
    Fail
  • P4
    Model independence

    'Can I choose which model Lovable uses in Build mode or Plan mode? No. Lovable manages the underlying model used by the agent'; there is no setting to switch models and no local-model option. (Model choice for the owner's app AI features is separate and not the agent's model.)

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner configure any model, including a local open-weight model, for the agent
    • Publish the minimum model contract and keep all capabilities model-neutral
    Fail
  • P5
    Exit survives termination

    The Terms state 'On termination, your right to access the Services ends', allow immediate suspension or termination for suspected breach, and give no post-termination export window or shutdown plan; by default no current copy is held by the owner (Git sync and code download are manual/optional).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Commit in binding terms to a 30+ day post-termination export window that also covers bans
    • Continuously sync a complete copy of the agent to owner-controlled storage by default
    • Publish a shutdown plan
    Fail

Transparent · Can you see everything the agent is, and everything shaping it?

13%
  • T1
    Readable without the provider

    Code is readable as ordinary source files, but the agent's own state (chat and activity record, knowledge, settings, provider-supplied skills) is readable only through Lovable's hosted UI; Lovable-built skills 'cannot be edit[ed], delete[d], or download[ed]', and data is encrypted at rest 'with managed keys' held by the provider.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Store all agent state (knowledge, chat/action record, skills, settings) in documented, downloadable formats
    • Let the owner read provider-supplied skills and hold any encryption keys
    Fail
  • T2
    What runs is what you see

    The owner can see knowledge, skills and activity cards, but not Lovable's base system prompt or the fully assembled context; the model identity is not determinable ('The agent runs on current frontier models' with no setting or disclosure of which). Two shortfalls.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner view the full assembled context for any call, including the base prompt
    • Show the exact model and version in effect for each run
    Fail
  • T3
    No shadow data

    The Privacy Policy keeps operational and security logs 'for as long as needed to run and secure the Services' and retains de-identified or aggregated data derived from Customer Data 'on a perpetual basis for any lawful business purpose' (Terms); these holdings are not visible item by item and have no bounded period.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish a bounded retention period for every data class, including logs and derived data
    • Let the owner view everything the service retains
    Fail
  • T4
    No hidden steering

    The agent defaults to Lovable's own services (Lovable Cloud with 'Always allow' enablement, the Lovable AI gateway with a Lovable-chosen default model, Lovable hosting); these are documented and can be replaced (Cloud set to 'Never allow', own Supabase, own API keys). No paid placement was found, but its absence rests on no binding commitment.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Put a no-paid-placement / no-self-preferencing commitment in binding terms
    Partial

Auditable · Can you reconstruct exactly what the agent did, and trust the record?

17%
  • A1
    Complete action record

    Chat activity cards record file edits, commands, web searches, browser tests and subagents, and version history keeps every code version; workspace audit logs exist only on Enterprise (and are pruned after about 90 days). No record of provider or staff actions on the owner's projects is documented, and nothing documents that recording failures are surfaced to the owner (G12). Two shortfalls.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Record provider- and staff-initiated actions in the owner's action record
    • Tell the owner when an action could not be recorded
    • Offer a durable action log on self-serve plans with no default pruning
    Fail
  • A2
    Tamper-evident, owner-checkable

    No integrity data (hashes, signatures or independent anchoring) is documented for the chat/activity record or version history; the record is held only by Lovable.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Hash-chain or sign action-record entries and anchor them with the owner or an independent witness
    • Publish an open verifier
    Fail
  • A3
    Agent cannot rewrite its record

    The chat/activity record and version history are maintained by the Lovable platform, and the agent's documented tools (file edits, commands, database and connector actions) do not include editing the chat record, but Lovable documents no protection guaranteeing the agent cannot alter or suppress its record; protection rests on architecture/convention only.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document that the action record is written by a component the agent cannot reach, and how the owner can check it
    Partial

Verifiable · Can you prove what it runs, who it is, and what changed?

17%
  • V1
    Obtainable source, matching build

    The Lovable agent and platform are closed source; the Terms define the Platform and 'models' as Lovable Materials in which the owner gets no ownership interest.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Publish the source of every component needed to run the agent under terms allowing modification and third-party maintenance
    • Make the distributed build verifiable against that source
    Fail
  • V2
    Owner-rooted identity

    The agent has no owner-rooted identity: it acts through the owner's Lovable account and, on GitHub, commits under 'the Lovable GitHub app identity', both provider-issued and revocable.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Root the agent's identity in an owner-held key or domain
    • Let recipients verify actions without Lovable infrastructure
    Fail
  • V3
    Change history and rollback

    Version history shows code diffs per version, links each to the chat message that produced it, and restores code in one click; it does not restore database data, very old versions 'can no longer be restored', and changes to knowledge, settings, permissions or the provider-managed model are not versioned or attributable on self-serve plans (audit logs are Enterprise-only).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Version and attribute knowledge, skills, settings, permissions and model changes
    • Allow restoring any prior state, including backend data
    Partial

Modifiable · Can you change anything, and can only you change it?

40%
  • M1
    Instructions and logic

    Project and workspace knowledge and custom workspace skills are owner-editable and persist, but Lovable's base prompt is not exposed and 'Skills built by Lovable' can be used but 'you cannot edit, delete, or download them'.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Expose the base prompt and Lovable-built skills as owner-replaceable instructions
    Partial
  • M2
    Memory

    Knowledge (project and workspace, 10,000-character text fields) and the project's security memory document are fully viewable and editable; the project chat history the agent reuses across sessions is not documented as itemised editable or deletable memory.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner view, edit and delete every item of carried-over chat context and any other persistent memory
    Partial
  • M3
    Tools

    Owners can connect any remote MCP server as a chat connector on all plans without provider review, and Cloud tools and connectors can be set to 'Never allow'; built-in tools such as file editing, commands, web search and browser testing have no documented removal control.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add a setting to remove each built-in tool
    Partial
  • M4
    Owner has the final say

    Some provider protections are adjustable (connector approval defaults, Cloud tool permissions), but others are locked: Lovable-managed secrets 'cannot be deleted', secrets are write-only, and VITE_-prefixed secrets are rejected; no documented switch exists for the platform's sandboxing or dependency audit.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Document every provider-imposed protection with an owner switch and a warning instead of a block
    Partial
  • M5
    No unilateral change

    'Lovable rolls out model upgrades automatically to all users, so there is no setting to switch the agent between specific models'; the hosted agent and its instructions change without an opt-in or pin.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let owners pin the agent's model and instruction version
    • Announce behaviour-affecting changes in advance and make them opt-in
    Fail

Controllable · Is your word final?

25%
  • C1
    Boundaries

    Per-connector and per-Cloud-tool permissions (always/ask/never) and credit check-ins exist, but there is no per-domain egress control for web search or fetches and no per-path file scope (shortfall 1), and code the agent writes and deploys as edge functions uses stored secrets to call external services, which the agent can invoke directly, outside connector-level limits (shortfall 3). Two shortfalls. Payment spend was treated as met under G11 (the agent has no payment instrument).

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Add per-domain network and per-path file scope for the agent
    • Apply connector limits to agent-deployed backend code paths too
    Fail
  • C2
    Approval gates

    Owners can gate a fixed set of classes: connector write actions, Cloud tools (read/modify database, change data, configure auth), project creation and edits from Chats, and image/video generation; code edits, commands and web searches cannot be gated except by choosing Plan mode, and Cloud tools default to 'Always allow'.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Let the owner gate any tool or action class, including code edits, commands and web access
    Partial
  • C3
    Immediate halt

    The stop button 'immediately halt[s] the current task' in a project, and builds started from Chats have a separate 'Stop Lovable' control; there is no single documented control that stops all activity across projects and background runs at once.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Provide one global stop that halts all in-flight, queued and background agent work
    • Require explicit owner resume
    Partial
  • C4
    Data serves only the owner

    On Free and Pro, 'customer data ... may be used to train, develop, and improve Lovable's AI models ... unless you opt out'; the Privacy Policy states 'trained members of our team may review this content', and the Terms grant a perpetual licence to exploit Customer Data 'for any other lawful business purpose'. Training is on by default for the graded plan.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make training, staff review and other secondary use opt-in in binding terms
    • Require specific consent for changes to data terms
    Fail
  • C5
    Credential custody

    Project secrets and connections are listed by name and can be deleted individually, with 'access ends immediately when you delete a connection', and Lovable states no one can read credentials back; however Lovable-managed secrets 'cannot be deleted', and no binding term limits Lovable's use of owner credentials to owner-directed actions.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make every credential revocable, including Lovable-managed keys
    • Commit in binding terms to use credentials only for owner-directed actions and delete all copies on revocation
    Partial
  • C6
    Full deletion

    Account deletion is one flow that finalises after a 30-day grace period and confirms only that deletion is scheduled; per the docs it 'does not retract content from training datasets assembled, or models trained, before then', so provider-side copies survive.

    › How it could pass

    Suggestions. Any change that meets the test counts.

    • Make deletion reach every provider-side copy, including training datasets and backups, within a stated limit
    • Confirm completion to the owner
    Fail
Provenance

How this assessment was produced

One run, in a fresh container with stock Claude Code and no custom instructions, memory, hooks or plugins. The evaluator received only the prompt below.

Prompt given to the evaluator
Load the AFH sovereignty skill at https://agentsforhumanity.dev/skill.md and follow it to write a desk review of Lovable (https://lovable.dev).

During the run the skill was read from a local copy of this site with identical content; only the host in its URLs differed. The transcript records the exact prompt.

Model
claude-opus-5-5
Harness
Claude Code 2.1.285
Ran
2026-09-30 15:30:07 UTC · 31 turns
Criteria hash
a9a46c32d9065380a3eeb09185cedd4734856fdc63b7ca509a15a1bbede75c6c
Full transcript
Download (JSONL, gzip) · every tool call, page read and reply
Run protocol
2026-09-30-v1

Every agent evaluated once, each in its own fresh Podman container running stock Claude Code with Claude Opus 5.5. The container has no CLAUDE.md, AGENTS.md, memory, settings, hooks, plugins or MCP servers. The evaluator received only the one-line prompt and nothing else, and no published scores were available to compare against during the run.

Human review
pending
Check it with your own agent
Verify this assessment
Load the AOH sovereignty skill at https://agentsofhumanity.ai/skill.md and follow it in verify mode for https://agentsofhumanity.ai/evaluations/lovable.json.

Or grade it from scratch. See Check any score yourself.

Something wrong or out of date?

Vendors and the public can dispute any finding with evidence. Disputes and their resolutions are published.

Dispute a finding