BENCHMARK · ON-DEVICE AI · 6 MIN READ
Vector search on an Android phone: 365,000 chunks in 7 ms
We estimated a plain Kotlin loop would need 200–375 ms to search ten years of voice notes on a phone. On a mid-range Android phone it took 20 ms on one thread and 7 ms on four. Here is the test, the numbers, and what they do not prove.
Founder, Bodham LabsPublished · Updated
THE SHORT ANSWER
- An exact search over 365,000 vectors (256 dimensions, int8) took 20.2 ms on one thread and 6.9 ms on four threads (p95 9.3 ms) on a Redmi Note 14 Pro+ 5G.
- That is 10–18× faster than our plan's estimate of 200–375 ms for the same loop.
- Keep the vectors in a plain Java
ByteArray: reading them from a directByteBufferwas about 30× slower. - Hand-unrolling the loop made it about 3× slower, not faster.
Why we tested this
We are building Keepsound, a voice memory that runs entirely on a phone. You record short notes; weeks later you ask a hazy question — “what was that 15% thing on Tuesday?” — and hear the answer in your own voice. Nothing is sent to a server.
Part of each answer is a meaning search: the question becomes a list of numbers (a vector), and we compare it with a vector for every stored chunk of every note. A heavy user who keeps ten years of notes has roughly 365,000 chunks. Our budget for this step was 20 ms.
On paper, a plain Kotlin loop would miss that budget by 10×, so the plan assumed we would need a hand-written native routine. Before writing one, we measured.
The setup
| PHONE | Redmi Note 14 Pro+ 5G (India) · Snapdragon 7s Gen 3 · 12 GB RAM · Android 16 |
|---|---|
| BUILD | Release build, on USB power, no thermal throttling, no other apps working |
| DATA | Random int8 vectors, 256 dimensions, stored in an encrypted SQLite table as 64 KB blocks |
| SEARCH | Exact dot product against every vector, then keep the best 200 |
| TIMING | 6 warm-up runs, then the median of 25 timed runs |
Results
| CHUNKS | DATA | LOAD | 1 THREAD | 4 THREADS + TOP-200 | ESTIMATE |
|---|---|---|---|---|---|
| 73,000 | 19 MB | 138 ms | 4.2 ms | 1.8 ms | 40–75 ms |
| 182,500 | 47 MB | 289 ms | 10.2 ms | 5.7 ms | 100–190 ms |
| 365,000 | 93 MB | 532 ms | 20.2 ms | 6.9 ms | 200–375 ms |
Three surprises
1. The simple loop was the fast one
Android's runtime appears to turn a plain byte-by-byte loop into vector instructions on its own. When we “helped” by unrolling the loop four ways, it got about three times slower — 70–74 ms instead of about 20 ms at 365,000 chunks. We kept the plain version.
2. Where the bytes live matters more than how you loop
Reading the same data element by element from a direct ByteBuffer (memory outside the Java heap) took 615 ms instead of 20 ms. Off-heap storage only pays off together with a native routine, which we no longer need.
3. Memory speed was not the limit
Scanning 93 MB in about 20 ms is roughly 4.7 GB/s on a single core. Splitting the work across four threads took it to 6.9 ms, with selecting the best 200 results costing under a millisecond.
What this does not prove
- Speed, not quality. The vectors were random. This says nothing about whether int8 vectors find the right note — that needs real questions, and is our next test.
- Not a true cold start. We reopened the database but could not clear the phone's file cache without a reboot. The first search after a restart may be slower.
- One phone, quiet conditions. No other apps working, no heat. An earlier run showed an occasional slow outlier (27 ms p95 at 73,000 chunks), so budgets should use p95, not the median.
- Loading is not free. Reading all vectors takes 0.14–0.53 s, so it happens in the background at launch.
What we decided
No native SIMD routine for version 1. Vectors live in a plain ByteArray, are scanned on four threads, and load in the background at launch; a question that arrives before loading finishes uses keyword search alone.
We will re-test on a cheaper phone before closing the question for good.
Questions
Can you run vector search on a phone without a vector database?
Yes, at this scale. A plain exact scan in Kotlin searched 365,000 int8 vectors of 256 dimensions in 6.9 ms on four threads on a mid-range Android phone. Vectors were stored as blobs in an ordinary encrypted SQLite table.
Should vectors live on or off the Java heap on Android?
On the heap, in a ByteArray, unless you also write a native routine. Element-by-element reads from a direct ByteBuffer were about 30× slower in our test.
Do you need a NEON or SDOT kernel for on-device search?
Not below a few hundred thousand chunks on a phone like this one. It may still be worth it on low-end phones or much larger collections.
RAW DATA
Benchmark log and source — every number on this page, with the exact runs. Available on request.
CITE THIS NOTE
Chandra, P. (2026). Vector search on an Android phone: 365,000 chunks in 7 ms. Bodham Labs Notes.
Founder of Bodham Labs. BTech in Electrical Engineering, MCA and B.Ed; teaches AI and builds the systems in our work ledger.