Agentic engineering has a cognitive debt problem
AI is changing the relationship between doing the work and understanding the work
Teams can produce analyses, campaigns, workflows, and products faster than ever—but that doesn’t always mean the people responsible for those outputs can explain how they were created, which assumptions shaped them, or what could go wrong.
Software engineering offers one of the clearest and most consequential examples of this tension. AI agents are dramatically accelerating development, while creating a new question for technical and business leaders alike: If a system works, but no one fully understands why, who owns it?
Nowhere is that tension clearer than in software development.
AI agents can read a repository, plan a change, write code, run tests, and prepare a pull request in minutes. But as development accelerates, teams risk producing software faster than they can build a shared understanding of how it works, why it was designed that way, and how to change it safely.
This gap is called cognitive debt. It can grow while the code remains readable, the tests stay green, and delivery metrics look healthy, compounding operational risk precisely when the business assumes feature output is at its safest.
The answer is not to slow agentic engineering down. It is to be deliberate about where agents lead, where human judgment remains essential, and who can defend every consequential decision after the chat window closes.
Agents can create code without creating shared theory
In the 80s, computer scientist Peter Naur argued that programming is a form of theory building. The product is not only the source code but also the theory in the builders’ heads: what the system does, why it is shaped this way, and how it can safely change. Forty years later, agents can produce code at industrial scale without necessarily leaving that theory in anyone’s head.
Two terms help us describe this result.
- Comprehension debt is the widening gap between how much code exists and how much of it any human genuinely understands.
- Cognitive debt is the loss of shared understanding across the people responsible for the system. As software engineering researcher Margaret-Anne Storey explains in her triple-debt model, the code itself can be fine while the mental model around it has eroded.
Technical debt usually announces itself through brittle dependencies, slow builds, or a module everyone dreads touching. Cognitive debt is quieter because the repo can look clean and the pull requests can keep merging right up until the bill arrives.
How engineering teams lose the thread
Cognitive debt usually builds through reasonable shortcuts, not one reckless decision.
An agent might modify 20 files and produce a polished pull request summary, so the reviewer reads the overview, checks the most important changes, confirms the tests are green, and approves. That review may be reasonable in isolation, but repeated across dozens of changes, the team’s nominal ownership can outgrow its actual understanding.
The reasoning also tends to stay inside the agent session. The agent considers several approaches, rejects two, and implements the third, but only the final code enters the repository. Months later, another engineer cannot tell whether the design reflects a deliberate trade-off, a forgotten constraint, or simply the first option that passed the tests.
Small choices then become structural commitments. A generated helper, schema, or service boundary begins as a convenience, and soon other features depend on it. By the time the rationale has faded, changing it carries a wide blast radius, so the comprehension gap compounds with every feature built on top.
Green tests don’t prove human ownership
Tests and code review remain essential, but neither can guarantee that somebody understands the wider decision.
A passing test shows that an implementation meets the expectations encoded in that test, but it can’t confirm that the requirement was interpreted correctly or that the most consequential failure modes were included. When an agent writes the implementation and its tests from the same interpretation, both can agree with the same mistaken assumption.
Imagine a pull request that changes how customer records synchronize between a CRM and a commerce platform. The code is clean and the tests pass, but six weeks later, duplicate records appear because different markets follow different system-of-record rules. Those rules were discussed in a workshop but never recorded with the implementation or made available to the agent.
The tests accurately verified an incomplete specification.
A strong review therefore needs more than green checks. It needs a named person who can explain why the change is appropriate, which assumptions it carries, and where it could fail.
The way engineers use AI matters
Emerging research suggests that AI assistance can weaken precisely the skills people need to supervise its output, but it also shows that the outcome depends on how the tool is used.
In a randomized study of 52 experienced Python developers, participants learned an unfamiliar asynchronous programming library with or without AI help. On average, the AI group finished no faster and scored 17 percentage points lower on a follow-up assessment of conceptual understanding, code reading, and debugging.
The strongest performers did not avoid AI. They remained involved by asking conceptual questions, requesting explanations, or interrogating generated code, while full delegation produced weaker learning.
A separate 2026 experiment tested an explanation gate that required novice programmers to explain generated code before integrating it. The gated and unrestricted AI groups produced comparable functional output, but when the AI was removed for a maintenance task, 77% of the unrestricted group failed compared with 39% of the gated group.
The useful distinction is not between using AI and avoiding it. It is between offloading effort and outsourcing understanding.
Match human oversight to the decision
Expecting engineers to understand every generated line equally would sacrifice much of the speed agents provide. So it’s better to distinguish between work that can be verified mechanically and decisions that depend on judgment, context, or accountability.
Human attention is most valuable when a decision is difficult to verify, expensive to reverse, dependent on information outside the repository, or consequential enough that someone may later need to defend it.
This means the central question is how much human involvement do we need?
That principle also shapes DEPT’s Empathy Framework, which helps teams assess how human or agentic different kinds of work should be. The framework considers the judgment and situational context an outcome requires, along with how difficult a mistake would be to reverse. It starts with the needs of the people affected and the nature of the work, rather than automating whatever appears technically feasible.
Four questions our engineers use to decide who owns the work
Before allowing an agent to make or implement work, our engineers ask:
Verifiability: Can the output be checked cheaply and reliably through tests, validation, CI, or a small diff?
Reversibility and blast radius: How difficult would the work be to undo, and what damage could occur first?
Context: Does the agent have the technical, commercial, regulatory, and organizational context the work requires?
Defensibility: Could a named person defend the work to a client, auditor, colleague, or incident review without reopening the agent session?
Those dimensions lead to three operating modes:
Agentic-appropriate: The agent may complete the task end-to-end. The contribution is identified as AI-generated.
AI-assisted, human-owned: The agent produces a candidate. A named person evaluates, owns, and can defend the result.
Human-required: A person authors the work. The agent may research options, summarize evidence, or challenge the reasoning.
Classification in practice
The unit of classification is important. Labeling an entire feature “AI-assisted” is usually too broad because one ticket can contain work with very different risk profiles.
Consider a team adding real-time fraud scoring to a checkout platform. Generating client-library scaffolding may be agentic-appropriate, while drafting integration tests or defining an event schema may be AI-assisted and human-owned. Selecting a model vendor and deciding which customer data may be shared require human authorship because they depend on privacy, cost, lock-in, and business context.
The classification can also change as a project matures. Agent-generated tests may be appropriate for a proof of concept, but once real users depend on the production suite, a named human needs to own what those tests cover and leave out.
Work-level classification keeps a consequential commitment from inheriting the same controls as a routine coding task simply because both appear in one ticket.
Advice doesn’t survive a deadline
A decision-making process is only useful if it survives delivery pressure. To make this durable, our engineering team embedded it in the instructions its coding agents follow on live projects.
As significant decisions happen, the agent classifies them. Low-risk work receives a tag and no additional ceremony, while human-owned and human-required work is recorded in a versioned decision log beside the code.
The agent handles the administrative details, but the owner writes a short defense in their own words, naming the rejected alternative or the constraint that shaped the choice.
The agent doesn’t draft that defense because doing so would create another artifact to approve without fully absorbing it.
Before the work ships, the log is checked against the actual changes, and another person reviews the owner’s explanation. The specific mechanics will vary between teams, but the underlying principle travels: human understanding needs a defined place in the delivery process rather than remaining a general expectation.
>> Start with one sprint
The first pilot only needs one team, one repository, and one sprint. Introduce the four questions, agree on examples for each classification, create a lightweight decision log, and make named ownership part of the definition of done.
During the retrospective, look for work that was difficult to classify or verify, decisions that depended on missing context, and explanations that still required the original chat history. Those gaps will show you where the rubric needs to become more precise.
The goal is not to document every choice or add ceremony to low-risk work. It is to preserve a minimum viable record of consequential work and ensure that someone can still explain, operate, and safely change what the team ships.
Making code cheap to generate does not make understanding cheap to skip.
