"The Model Wrote It” Is Not an Accountability Model

A risk-based acceptance model for AI-generated software.

AI coding agents are making one question less interesting:

Can AI write code?

Solved, yes.

Agents can inspect repositories, modify files, generate tests, fix build failures, explain code paths, and produce pull requests. In the right context, they can execute work that used to require hours of developer effort.

But enterprise software is not judged by whether code was generated.

It is judged by whether the resulting system is correct, safe, compliant, reliable, maintainable, and owned.

That makes the harder question:

Who owns the software when something goes wrong?

This is where the “AI will replace engineers” narrative starts to break down.

Production software touches customers, data, financial systems, regulated workflows, operational processes, security boundaries, brand trust, and legal obligations.

When AI-generated software causes an outage, leaks data, violates a policy, misprices a product, fails a compliance control, or creates customer harm, no enterprise can stand in front of an executive committee, regulator, auditor, or customer and say:

“The model wrote it.”

That is not an accountability model.

That does not mean enterprises should avoid AI coding agents.

It means they need a better operating model for accepting AI-generated change: one that classifies work by risk, gathers the right evidence, routes the right expertise, and preserves human accountability without turning every AI-assisted task into a governance bottleneck.

Code Generation Is Not Software Ownership

One of the reasons the AI coding conversation gets confused is that we collapse two very different things:

Producing code
Owning a software system

AI can increasingly help with the first.

It cannot own the second.

Software ownership includes requirements, architecture, testing strategy, security, compliance, release readiness, operational support, customer impact, and long-term maintainability.

A generated pull request may be useful.

It may be impressive.

It may save meaningful time.

But it does not answer the ownership question.

Someone still has to decide whether the requirement was correct, whether the implementation fits the architecture, whether the tests are meaningful, whether the release risk is acceptable, and who owns the production consequences.

That someone is not the model.

Liability Makes Accountability Real

In the previous article in this series, I argued that AI agents can be responsible for execution, but humans remain accountable for outcomes.

Liability is where that distinction becomes concrete.

It is one thing to say an AI agent can produce a diff.

It is another thing to ask who owns the consequences of accepting that diff into a production system.

If an AI-generated change creates a customer-impacting defect, introduces a security vulnerability, bypasses a regulatory control, mishandles data, or causes a critical workflow to fail, the organization will not treat the model as the accountable party.

The organization will ask:

Who approved the change?
Who reviewed the risk?
Who validated the behavior?
Who owned the service?
Who accepted the release?
Who had the authority to say this was safe?

The model may be part of the evidence trail.

It may be part of the root cause.

It may be part of the tooling chain.

But it is not the accountable owner.

This also creates an important boundary between AI providers and enterprise owners.

In a model-only or API-based relationship, the provider may own the model platform, service commitments, data handling terms, and tooling behavior. But the enterprise still owns the application context: the use case, requirements, codebase, integration, validation, release decision, production behavior, and customer outcome.

That is why “the model wrote it” is not enough. The model did not own the requirement, operate the production environment, approve the release, or accept the business risk.

That distinction changes the adoption question.

Not:

How much engineering work can we automate?

But:

What work can we safely accept, and what evidence do we need before accepting it?

Approval Is Not Authorship

As AI generates more implementation work, approval has to become more explicit.

When a human writes code, approval is often tied to authorship. The engineer understands the choices because they made the choices.

When an AI agent writes code, the accountable human may not have authored every line.

But accountability cannot degrade into rubber-stamping.

Approval needs to become an evidence-based act.

The human approver is effectively saying:

I understand the intent, scope, risk, validation evidence, and operational impact well enough to accept accountability for this change.

That does not mean a single executive, engineer, or system owner must personally understand every detail of a complex system. Modern software is too interconnected for that.

Accountability has always depended on delegated expertise.

A technology executive may be accountable for a platform, but that accountability is supported by architects, engineers, product owners, security teams, compliance partners, QA, and operations specialists.

AI can help scale that expertise by making evidence easier to gather, analyze, and communicate. It can summarize changes, map diffs back to requirements, identify affected areas, run tests, compare changes to policy, and surface risks.

But AI cannot become the domain owner of record without recreating the same accountability problem.

Someone still decides which risks are acceptable, which evidence is sufficient, and whether the system is ready to move forward.

Accountability does not require omniscience.

It requires a governed chain of expertise and a clear owner who accepts the decision.

A Risk-Based Acceptance Model

This is where enterprise adoption needs to mature.

The impressive demo is:

The agent wrote the code.

The enterprise adoption question is:

Can we safely accept the change?

The answer should not be the same for every type of work.

A low-risk documentation update does not need the same acceptance standard as a payment workflow change. A test fixture does not need the same review as a data retention policy. A refactor in an isolated component does not carry the same risk as a change to authentication, pricing, regulatory reporting, or customer data handling.

The goal is not to make every AI-generated change feel scary or burdensome.

The goal is to classify the work, match governance to risk, and require enough evidence to accept the outcome.

This requires more than policy language. Enterprises need business processes and operating models that make risk-based acceptance as frictionless as possible. The workflow should help teams classify the change, gather the right evidence, route the right reviewers, and preserve the acceptance trail without turning every AI-assisted task into a governance ceremony.

Good governance should reduce ambiguity, not create drag.

This is an ideal use case for agentic workflow orchestration: classify the change, gather evidence, route reviewers, preserve the acceptance trail, and escalate when risk requires it. But the workflow should be designed around human accountability, not agent autonomy. Agents can help orchestrate the process. Humans still accept the decision.

That means organizations need practical acceptance tiers.

Some AI-generated changes can move forward with lightweight review:

Documentation updates
Test scaffolding
Localized bug fixes

Some changes require deeper engineering validation:

Shared library updates
Integration behavior
Data model changes

Some changes require additional domain review:

Authentication and authorization
Pricing or financial calculations
Customer data handling or regulatory reporting

The issue is not whether AI was involved.

The issue is whether the organization understands the risk of the change and has enough evidence to accept it.

That evidence should be explicit:

What was the agent asked to do?
What context and constraints were provided?
What changed?
What tests or checks were run?
What risks were identified?
Who reviewed and accepted the change?

This is the practical shift AI coding agents require.

Acceptance cannot be based on the fact that code was generated quickly.

It has to be based on evidence that the change is understood, validated, and owned.

Accountable Systems Are the Product

The central mistake in the replacement narrative is treating code as the product.

Code matters.

But a pull request is not the product.

A generated file is not the product.

A passing test suite is not the product.

A working demo is not the product.

The product is a system that meets a real need, behaves safely, protects data, complies with policy, operates reliably, can be maintained, and has clear ownership.

Accountable systems are the product.

That is why the future is not software with no accountable owner because an AI agent wrote it.

The future is AI-assisted delivery with clearer ownership, risk-based acceptance, and agentic workflows that preserve the evidence trail around human decisions.

AI executes.

Workflows orchestrate.

Humans accept.

Organizations own the result.

The enterprise question is no longer:

Can AI write code?

It is:

Can we govern AI-generated change?

Originally published on LinkedIn. View the original post.