State space models for long data streams
THE QUESTION
Can a Mamba-style state space model act as a cheap, long-memory context model for a fast live data stream?
Founder, Bodham LabsUpdated
What did we build?
- A GPU Mamba (S6) context model, running in a container on 11 Mar 2026.
- Before that, a component we called an "SSM" that was in fact a GRU PPO actor-critic exported to OpenVINO. We ran a 3-config tournament: 2×256, 1×512 and 3×128.
- A data integrity audit of the model's inputs.
- A documented next design, "Mamba-4": a Micro-Jamba hybrid with 24–36M parameters planned. It is not built. Its status reads "PRE-DEVELOPMENT (Phase 0)".
- The test bed was live market data streams, in paper mode only, with no real money.
What does the evidence show?
| WHAT | VALUE | SOURCE |
|---|---|---|
| Hidden state after 1,024 steps | about ±10¹⁵ | data integrity audit, Mar 2026 |
| GPU memory of the running S6 model | 396 MiB | container log, 11 Mar 2026 |
| Raw EMA input values (not normalised) | around 69,000 | data integrity audit, Mar 2026 |
| One "z-score" input | 86.74 | data integrity audit, Mar 2026 |
| How far it drowned out 14 other features | about 10¹³ | data integrity audit, Mar 2026 |
| Features that were always zero | 3 | data integrity audit, Mar 2026 |
| Earlier GRU hidden-state norm, n = 100 to 96,900 | constant at 1.3850 | model run log |
Where has it led us?
The model ran, but the inputs were not normalised. EMA values were around 69,000. The hidden state grew to about ±10¹⁵ over 1,024 steps and drowned out 14 other features. Three features were always zero.
Our earlier "SSM" was not an SSM at all. It was a GRU. Its hidden-state norm stayed at exactly 1.3850 from step 100 to step 96,900, which means the state was stuck. Reading the norm in the log is what showed us.
Neither failure tells us yet whether the model idea works. Both were found in the data and the logs. The lesson: check input ranges and the hidden-state norm before judging the model.
The next design adds shadow-mode gates. If the model underperforms in shadow mode for two weeks, we revisit the architecture and do not go live. It is a staged evaluation, not a result.
What is still open?
- With normalised inputs, does a small Mamba model beat a simple baseline on the same stream?
- Is a hybrid (some attention layers, some SSM layers, like Jamba) worth the extra complexity at 24–36M parameters?
- How long a context does a fast stream actually need?
NEXT STEP · NOT DONE YET
What would we do next?
Normalise every input, log the hidden-state norm from the first step, and re-run the S6 model against a simple baseline before building Mamba-4.
Have a problem that needs this kind of research?
Tell us about it