Keeping autonomous agents from breaking themselves
THE QUESTION
What stops an autonomous agent from failing in a chain, or from damaging itself when it is allowed to change its own setup?
Founder, Bodham LabsUpdated
What did we build?
- RAMU / NEO: an autonomous agent based on OpenClaw, running on a remote Linux server (Feb 2026).
- A one-night self-modifying swarm experiment (20–21 Feb 2026). Agents coordinated through a shared Redis "tuple space" (the Linda model, with stigmergy). Unclaimed tasks built up "toxicity". At a threshold, a "Ribosome" rewrote a worker's code. The LLM step was simulated with time.sleep.
- A RAMU "digital organism" (9–10 Mar 2026), reviewed afterwards in our own written verdict.
- A cost-tiered design, on paper only: Tier 1 is an expensive LLM, Tier 2 summarises and routes, Tier 3 is local small models. Escalation is penalised ("Tax = 10 × Tokens"). Workers get a 4096-token context cap and run in Docker with --cap-drop ALL and a read-only filesystem.
What does the evidence show?
| WHAT | VALUE | SOURCE |
|---|---|---|
| Outage caused by the failure chain | 45 hours | agent incident notes, Feb 2026 |
| Failure chain | API billing error → model failure → SOUL.md missing → heartbeat death → blackout | agent incident notes, Feb 2026 |
| RAMU memory rows | 677, of which 676 were dummy sensor events | RAMU memory store, 9–10 Mar 2026 |
| Swarm experiment length | One night | experiment notes, 20–21 Feb 2026 |
Where has it led us?
One API billing error led to a model failure, a missing SOUL.md file, a dead heartbeat and a 45-hour blackout. Each step was small. Together they took the agent down.
We wrote rules after that. "NEVER modify own config files (openclaw.json, models.json, auth-profiles.json)." "Model deprecation = REPORT to Neo, don't self-correct." Files over mental notes.
We also wrote down: "80% quality shipped is better than 100% logic researched."
Separately, an expired API key silently stopped a worker.
In the swarm, our own lesson was: "If the Ribosome creates a syntax error, the agent dies permanently." The proposed fix is to sandbox-test every mutation in a subprocess first and cap it at 3 attempts.
Our review of the RAMU organism said it plainly: "This is not continuous artificial life; this is just a standard asynchronous RAG pipeline." And: "We have fundamentally over-engineered the philosophy and under-engineered the physics."
What is still open?
- Which failures should an agent repair by itself, and which should it only report?
- Can sandbox-tested mutations make self-modifying code safe enough to leave running?
- How should an agent notice an expired key before a worker stops?
- Does the cost-tiered design cut spend in practice? So far it exists only on paper.
NEXT STEP · NOT DONE YET
What would we do next?
Build the sandbox check for mutations (subprocess test, at most 3 attempts) and run the swarm again with real model calls instead of time.sleep.
Have a problem that needs this kind of research?
Tell us about it