# On-device voice memory in English and Hinglish

- URL: https://bodhamlabs.com/research/on-device-voice-memory
- Theme: Speech & Indian languages
- Area: ON-DEVICE AI · SPEECH · RETRIEVAL
- Status: MEASURED — Benchmarked on real hardware; log on file.
- Period: Oct 2026 – ongoing
- Updated: 2026-10-12
- Author: Prabhakar Chandra, Bodham Labs
- Built into: Mementos (codename) (https://bodhamlabs.com/products/mementos)

## Question

Can a mid-range phone record, transcribe and search voice memos in English and Hinglish without ever going online, and still answer in milliseconds?

## Summary

A private voice memory that runs fully on a phone: two speech models routed piece by piece, encrypted search in milliseconds, answers in your own voice.

## What we built

- Mementos (codename), an Android app with no internet permission at all. Audio and text are stored encrypted.
- Two on-device speech models, Parakeet for English and Hinglish-Swift for Hinglish. A small language detector routes each piece of a memo, cut at pauses into pieces of up to 12 seconds.
- Search over an encrypted SQLite database (SQLCipher with FTS5), and an exact int8 vector scan tested at 365,000 chunks.
- Answers that play the matching moment of your own recording, so the answer is exactly what you said.
- Three test builds between 9 and 10 Oct 2026; the fourth is in progress. Every build has a changelog entry, a git tag and measured results.
- A rule: no model enters the app until a one-page model sheet is filled in, covering licence, size, memory, quality on our own recordings, and how to switch it off.

## Evidence

| What | Value | Where it was recorded |
| --- | --- | --- |
| English word errors (WER), 17 own recordings | 11.7% Parakeet · 24.5% Hinglish-Swift | laptop CPU, Oct 2026 |
| Hinglish word errors (WER), same set | 22.8% Hinglish-Swift · 70.9% Parakeet | laptop CPU, Oct 2026; 3 Hinglish recordings, a small sample |
| Mixed-language clips, after per-piece routing | WER 35.7% → 18.2% | 40 mixed clips joined from our own recordings, laptop CPU |
| Hinglish pieces sent to the wrong model | 29.7% → 7.2% with a 10-second window | 1,023 pieces, laptop CPU |
| Right memo first / in the top three | 9 of 10 / 10 of 10 test questions, 5–13 ms each | Redmi Note 14 Pro+, test build 1 |
| A 55-second memo ready to search | 12–14 s after recording | Redmi Note 14 Pro+, test build 3 |
| Peak memory while transcribing | 1.30 GB English · 1.66 GB mixed | Redmi Note 14 Pro+, test build 3 |
| Exact scan, 365,000 int8 vectors | 20.2 ms on 1 thread · 6.9 ms on 4 | Redmi Note 14 Pro+, Oct 2026 |

## Where it led

- One speech model was not enough. Each model won one language and lost the other, so the app routes between two.
- Letting the first 10 seconds decide the language of a whole memo broke memos that switch language. Routing piece by piece cut word errors on mixed clips by half; giving the detector 10 seconds of audio around each piece cut the misroutes.
- Much of what we tried made things worse and was dropped: telling the model which names to expect, a blank-penalty setting, a slower second pass, a single bilingual model, and state space models for saving memory.
- Small language models (0.5–0.6 billion parameters), steered only by prompts, lost to the app's own rules on all three jobs we tried. Rules stay for now.

## Open questions

- How often do real searches come back empty, and would a meaning model help then?
- Does 1.66 GB of peak memory hold up on phones with 8 GB?
- Can a small adapter trained on a user's own voice cut Hinglish errors, on the phone?

## Next step (not done yet)

Field-test build 4 with outside testers: measure how often searches come back empty and how real Hinglish memos spoken into the microphone fare, then decide on a meaning model.

## Related notes

- [Routing Hinglish speech: why a 10-second window beat a short one](https://bodhamlabs.com/notes/routing-code-switched-speech)
- [Why a half-billion-parameter model lost to our rules](https://bodhamlabs.com/notes/small-model-lost-to-rules)
- [Vector search on an Android phone: 365,000 chunks in 7 ms](https://bodhamlabs.com/notes/vector-search-on-android)

## References

- [NVIDIA Parakeet TDT 0.6B v2 (model card)](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2)
- [sherpa-onnx: on-device speech recognition (k2-fsa)](https://github.com/k2-fsa/sherpa-onnx)
- [SQLCipher: encrypted SQLite](https://www.zetetic.net/sqlcipher/)