I watched one in all our engineers clarify the identical authentication sample to Claude Code for the fourth time final month. Not as a result of he forgot he’d defined it. As a result of the software forgot.
Each session, from scratch. “We use JWT validation on the gateway layer, not in particular person companies.” He’d stated it three days in the past. And the week earlier than. And each time he began a brand new session for the previous six months. Every time, the AI nodded alongside, adopted the directions completely, after which forgot every thing the second the session ended.
I saved fascinated by this, as a result of it felt just like the sort of downside that ought to already be solved. It’s 2026. These fashions are genuinely succesful. They will cause about complicated codebases, debug refined race situations, write stable exams. And but they function with what I can solely describe as aggressive amnesia — a pathological lack of ability to retain something previous the present session.
The autocomplete excuse
This made sense when AI coding instruments had been autocomplete engines. Copilot circa 2022 was finishing single traces of code. The context was one file. Why would it not want reminiscence? You kind, it suggests, you tab. Session reminiscence is irrelevant.
However that’s not what these instruments do anymore. We ask them to construct options throughout a number of information. Debug manufacturing points that require understanding system structure. Onboard new engineers to unfamiliar codebases. Run autonomously on GitHub points. And each single time, they begin from zero.
I maintain coming again to this analogy: think about hiring an excellent contractor who exhibits up each morning with whole amnesia. They will code. They’re quick. However daily you spend the primary hour explaining the venture, the crew conventions, the selections you’ve already made, the errors you’ve already discovered from. And the subsequent morning? Identical factor.
That’s the expertise proper now. For each crew. With each software.
The issues that by no means make it into code
Right here’s what bugs me most. The stuff the AI retains forgetting isn’t within the code. It’s the stuff that lives between the traces:
Why we selected Postgres over DynamoDB. (Efficiency for our question patterns, but in addition as a result of the crew has deep Postgres experience and 0 DynamoDB expertise.)
Why the notification service is a monolith module and never a microservice. (We tried microservices. It was a catastrophe. We reverted in Q3 and no one documented why.)
That the billing pipeline has a identified edge case the place occasions get silently dropped underneath excessive load. (Two engineers find out about this. Considered one of them simply gave discover.)
None of that is in a file the AI can scan. A few of it was in a Slack thread from eight months in the past. Most of it’s in folks’s heads. And it’s precisely the sort of context that determines whether or not the AI’s output is right or subtly, dangerously improper.
“Simply put it in a config file”
I do know what you’re considering, as a result of I assumed it too. CLAUDE.md. .cursorrules. System prompts. Simply write all of it down in a file and level the AI at it.
We tried. Everybody tries. And it really works — for about three weeks, till the file is stale and no one updates it as a result of updating a config file is upkeep work that doesn’t ship options. The one that wrote the unique file has moved on to different issues. New choices get made in conversations that by no means make it to the file. The file turns into a historic artifact that roughly corresponds to what the crew believed in some unspecified time in the future previously.
It’s the wiki downside once more. Somebody creates it with good intentions. It begins decaying instantly. Inside six months, builders actively mistrust it as a result of they’ve been burned by following outdated info.
“Simply use an even bigger context window”
The opposite in style reply. 200K tokens wasn’t sufficient, so now we’ve 1M. Simply stuff every thing in.
I’ve spent loads of time fascinated by this, and I believe it basically misunderstands the issue. A much bigger context window offers you extra room for the present session. It doesn’t offer you reminiscence. It doesn’t inform you why the crew made a specific architectural resolution final quarter. It doesn’t know concerning the manufacturing incident that formed how the crew thinks about error dealing with. It doesn’t know that Sarah is the one one who understands the reconciliation pipeline.
A much bigger window is an even bigger scratch pad. The scratch pad nonetheless will get erased when the session ends. You haven’t solved amnesia — you’ve given the amnesiac a bigger pocket book that additionally will get burned each evening.
“Simply add extra brokers”
This one is newer and it’s the one which will get me. The reply to “the AI doesn’t know sufficient” is outwardly “add extra AIs that additionally don’t know sufficient, however give each a narrower job.”
A evaluation agent. A testing agent. A deployment agent. Fifteen specialised brokers, every with hardcoded directions for one activity. Somebody on the crew wrote these directions. Somebody has to keep up them. When the evaluation requirements change, somebody updates the evaluation agent. When the testing framework modifications, somebody updates the take a look at agent. It’s the config file downside at a better stage of abstraction, with extra shifting components.
And you realize what none of these brokers know? Something about your group. They know what somebody hardcoded into their directions. They don’t know what the crew discovered final week.
The query that retains nagging me
Right here’s what I maintain coming again to: what if the AI simply… remembered?
Not the uncooked dialog transcript. That’s noise. However the precise data — the selections, the patterns, the errors, the conventions — extracted from conversations and out there in future classes. Not only for the engineer who had the dialog, however for the entire crew.
An engineer explains why we use occasion sourcing for the audit system. That clarification turns into a structured data merchandise — out there to each different engineer, in each future session, with out anybody sustaining a file.
Somebody discovers a refined coupling between two companies whereas debugging. That discovery will get captured. Subsequent time somebody touches both service, the AI already is aware of concerning the coupling. Not as a result of somebody remembered to doc it, however as a result of the system was listening when the data was created.
The AI that helped you debug a billing situation on Tuesday begins your Thursday session already realizing what you found. The brand new engineer who joins subsequent month has an AI that is aware of every thing the crew has discovered previously yr — from day one.
I take into consideration this lots as a result of it modifications what the software basically is. It stops being a coding assistant and begins being organizational reminiscence. Not a wiki that somebody has to keep up. A residing data base that grows as a result of folks use the software.
The place this goes
The AI coding software market is about to separate. On one facet: instruments that assist particular person engineers write code quicker. These are commoditizing. The fashions get cheaper each quarter. The wrappers get thinner. There’s no sturdy benefit.
On the opposite facet: instruments that make a company’s collective intelligence out there to each engineer, each session, completely. These don’t exist but. Probably not. Not in a means that really works.
I’ve been spending the final yr fascinated by what the second class appears like. The way you construct it. What the structure must be. The place the trade’s assumptions are improper.
All of it began from this one commentary: your AI software has amnesia, and no one appears to assume that’s an issue price fixing.
I believe it’s the one downside price fixing.
