A persistent agent can remember what happened.
That sounds useful until you realize there is a dangerous assumption hiding underneath it:
what if the system remembers something that never actually happened?
This problem shows up surprisingly quickly once an agent starts doing real work.
It decides to change a file.
It calls the tool.
The tool returns success.
The agent moves on.
Later, it remembers:
the file was updated.
But was it?
Maybe the write partially failed.
Maybe the path was wrong.
Maybe another process overwrote it a second later.
Maybe the command exited with code 0 but produced the wrong result.
Maybe the API accepted the request but the downstream system never applied it.
The dangerous part is not just that the action failed.
The dangerous part is that the system believes it succeeded.
And once that belief gets written into memory, task state, or future planning, the error stops being local.
It becomes part of the system’s reality.
That is the problem I’ve been thinking about lately.
A persistent agent should not update its state just because an action returned “success.”
It should first verify that the world now matches what it expected.
Success messages are weaker than they look
Most software systems already deal with this problem in some form.
A command can exit successfully while producing the wrong file.
An HTTP request can return 200 while the application state is still inconsistent.
A database write can succeed while some downstream cache remains stale.
A deployment script can finish without the new version actually being served.
Humans usually know this intuitively.
If I ask someone to deploy a website, I don’t really care that the deployment command completed.
I care that the new page is actually online.
Those are not the same thing.
Agents need to learn the same distinction.
Right now, many agent workflows are built around something like:
decide → call tool → receive response → continue
That is fine for short tasks where a human checks the final result.
It becomes much more dangerous in a long-running system.
A persistent agent may use the result of one action as the assumption behind ten later actions.
If the first assumption is wrong, everything after it can still look internally consistent.
That is how a small execution error turns into a corrupted future.
The action is not the result
I think this is one of the easiest mistakes to make when building agents.
We confuse an attempted action with an observed outcome.
For example:
The agent wants to create a file.
It calls write_file.
The tool says:
success
What has actually been proven?
Only that the tool believes it completed the operation.
That does not prove:
- the file exists
- the file is at the correct path
- the contents are correct
- the format is valid
- the file can actually be used by the next step
- nothing else immediately changed it
The same thing happens with almost every external action.
Send an email.
Did it enter the recipient’s mailbox?
Update a record.
Did the application read the new value?
Deploy a service.
Is the expected version actually serving traffic?
Restart a process.
Did it come back healthy?
Create a task.
Is it really visible to the system that is supposed to execute it?
The difference between “I tried” and “the world changed as expected” is where verification starts.
A better loop
The loop I increasingly want AdamI to follow looks more like this:
Intent → Attempt → Observation → Verification → State Commit
The first two parts are familiar.
The system decides what it wants to happen.
Then it attempts an action.
But after that, it should not immediately update its internal truth.
It should observe the environment again.
Then compare what it sees with the expected postcondition.
Only if those match should the new state become something the system is willing to remember.
That extra step sounds expensive.
Sometimes it is.
But I think it becomes necessary as agents gain more authority.
If the system is only answering a question, a wrong assumption may produce one bad answer.
If the system is operating for hours or days, the same wrong assumption can affect memory, planning, permissions, retries, and future task creation.
Persistence makes verification more important because mistakes can survive.
Verification is not just checking that something exists
This part matters too.
A weak verifier can be almost as dangerous as no verifier.
Suppose the agent creates a configuration file.
A simple check might be:
does the file exist?
If yes, success.
But that only verifies presence.
It does not verify meaning.
The file may exist but contain the wrong values.
It may be malformed.
It may reference the wrong service.
It may violate an invariant elsewhere.
So verification has to depend on the expected consequence of the action.
Sometimes the right check is simple:
- file exists
- process is running
- HTTP endpoint returns 200
Other times it needs to be stronger:
- content matches expected structure
- API returns the new value
- downstream behavior changed
- tests still pass
- another component can consume the result
- invariants remain true
This is one place where I think agent systems need to be much more explicit.
“Success” is not one universal thing.
Every meaningful action has a postcondition.
And the verifier should be checking that postcondition, not just looking for a green signal.
The system also needs to know what it actually knows
There is another side to this.
Sometimes verification is impossible or incomplete.
Maybe the system sends an external request but cannot directly observe the final result.
Maybe the API only confirms receipt.
Maybe a human has to approve something later.
Maybe the real outcome will not be known for hours.
The wrong response is to treat uncertainty as success.
A better system should be able to keep different states separate:
attempted
observed
verified
still uncertain
This matters because persistent memory tends to flatten things.
If a task log says:
deployment completed
that sounds final.
But maybe what really happened was:
deployment command completed; public endpoint not yet verified
Those are very different facts.
I want AdamI to preserve that difference.
A system that knows what it has verified is useful.
A system that also knows what it has not verified is much safer.
Wrong memory is worse than missing memory
I’ve spent a lot of time thinking about memory in AdamI.
At first the problem looked like:
How do I make the system remember enough?
Now I think there is another problem that may be just as important:
How do I stop it from remembering false things?
Missing memory is inconvenient.
The system may repeat work.
It may forget context.
It may ask the same question again.
Wrong memory is different.
Wrong memory can make future reasoning look correct while being built on a false premise.
Imagine the system remembers:
backup completed successfully
when it did not.
Or:
dependency upgraded
when the old version is still running.
Or:
user approved this action
when approval was never given.
Or:
task recovered
when the underlying failure is still present.
These are not normal hallucinations inside an answer.
They are state corruption.
And in a persistent system, state corruption can survive long after the original mistake is forgotten.
That is why I increasingly think memory writes should be treated with more suspicion than memory reads.
Writing something into persistent state is a commitment.
The system should earn that commitment with evidence.
Verification should sometimes fail
This sounds obvious, but it changes the design.
A good verifier cannot exist only to confirm what the agent already expects.
It has to be allowed to say:
no, that did not happen.
Or:
I cannot confirm this.
Or:
the observed result is different from what was expected.
That creates another question:
What happens next?
Sometimes retry.
Sometimes re-plan.
Sometimes roll back.
Sometimes wait.
Sometimes ask a human.
Sometimes mark the task as failed.
Verification is not just a final checkbox.
It changes control flow.
That is why I see it as part of autonomy rather than simple testing.
An agent that can detect its own failed action and change course is more capable than one that blindly continues.
But it is also more constrained.
And I think those two things need to grow together.
Recovery depends on knowing what is true
This also connects directly to recovery.
You cannot recover safely if you do not know which state is real.
Suppose an agent makes three changes:
A succeeds.
B fails.
C is never attempted.
If the system records all three as complete, recovery becomes messy.
If it records the sequence accurately, it can reason from evidence:
A is verified.
B failed verification.
C did not run.
That creates a much cleaner recovery path.
It also makes rollback possible.
The system can ask:
What changed?
What was verified?
What remains uncertain?
What can be reversed?
Without that, “recovery” often means just trying more actions.
That is not recovery.
That is adding more uncertainty.
Verification also limits authority
There is a governance angle here too.
The more authority an agent receives, the more important verification becomes.
If the agent is allowed to edit a local note, a weak verifier is annoying.
If the agent is allowed to deploy software, move money, send messages, change permissions, or operate infrastructure, weak verification becomes dangerous.
This is why I keep coming back to the idea that capability and authority should not scale together automatically.
A more capable model can attempt more things.
That does not mean the system should trust every reported result.
Authority should come with stronger requirements around observation, verification, reversibility, and audit.
The system should not just ask:
can I do this?
It should also ask:
how will I know it actually worked?
That question should probably exist before the action is even attempted.
Verification should be designed before execution
This is a design pattern I want to explore more.
Instead of acting first and inventing a check afterward, the system should define the expected postcondition before execution.
For example:
Intent:
deploy release 42ab59a
Expected result:
production symlink points to the new release
public EN route returns 200
public ZH route returns 200
release tree matches artifact
previous release remains available for rollback
Then execution happens.
Afterward, verification compares the real world against those expectations.
That makes the process much clearer.
It also prevents a common failure mode where the system quietly changes the definition of success after something goes wrong.
If the success criteria were defined before execution, the verifier has something concrete to test.
This starts to look less like ordinary tool use and more like transactional behavior.
Not exactly database transactions, but the same instinct:
do not commit state until you know enough about what actually happened.
AdamI is not there yet
AdamI already has pieces that make this direction possible.
There is persistent state.
There are task lifecycle mechanisms.
There are control boundaries.
There is observability.
There are places where actions can be checked before the system moves forward.
But I do not want to pretend there is already a complete verification layer.
There isn’t.
Right now, verification is still uneven.
Some actions are easy to confirm.
Others depend too much on tool responses.
Some state transitions are stronger than others.
There is still work to do around explicit postconditions, evidence, uncertainty, rollback, and deciding what is safe to commit into longer-term state.
That is exactly why this feels like the next thing worth building.
Before the system believes itself
The more persistent an agent becomes, the more dangerous self-confirmation becomes.
If every action result is immediately accepted as truth, the system can slowly build a world model made from its own unchecked assumptions.
That may work for a while.
Until it doesn’t.
I think a long-running agent needs a harder rule:
Do not believe the action. Believe the evidence.
Attempt something.
Observe what changed.
Verify the consequence.
Only then update memory, task state, or future plans.
That is a slower loop than:
act → assume success → continue
But it is also a much more trustworthy one.
And for persistent agents, trust may come less from how often they succeed than from how well they can tell when they didn’t.
That is the layer I want to push on next.