GPUs fail, nodes get reclaimed or drained, spares join. Everserve keeps the model serving through all of it, so frontier models can run on spot and idle capacity at about half the cost.
sessions dropped when GPUs leave mid‑generation
modeled saving against the same nodes on-demand, at list prices
Illustrative recovery. Measured so far: up to a 70B model across H100 nodes, see Proof below.
Inference engines fix their GPUs at startup. Lose one and the engine aborts, with every session on it. It happens on every fleet: NVIDIA's own production cluster restarted replicas 46 times a day across about 150 of them.1 And it is why the largest models only run on capacity nobody will take back.
Everserve runs inside vLLM as a plugin, with a control plane beside it. Everything a change needs is ready before anything fails.
A fault with no warning, or a reclaim, drain or handback with notice.
Communicators are pre-built. No rendezvous, no restart.
A free GPU in the same node, or a shared spare node that holds the model in host memory and covers every engine serving it. KV restored from the mirror.
From the last committed step. No prompt is recomputed.
Your scheduler decides where capacity goes; Everserve moves the running engine. Call it from your scheduler or from a dashboard.
A spare takes its place. Sessions keep going, nothing is recomputed.
Reclaim spot or idle capacity, or hand it to a customer, on the deadline you set.
Empty it before the window without waiting for long generations to finish.
# Can node-d be taken back, and what happens if it is? GET /v1/readiness/my-engine { "engine": "my-engine", "epoch": 41, "world_size": 32, "nodes": [{ "node": "node-d", "ranks": [24, 25, 26, 27, 28, 29, 30, 31], "reclaimable": true, "outcome": "replace", "min_notice_ms": 5000, "expected_pause_ms": 900 }] } # Take it back in 30 seconds. POST /v1/drain { "node": "node-d", "lead_ms": 30000 } 202 { "node": "node-d", "deadline_ms": 1791400030000, "engines": [{ "engine": "my-engine", "ranks": [24, 25, 26, 27, 28, 29, 30, 31], "outcome": "replace" }] }
| Model | Setup | Event | Serving pause |
|---|---|---|---|
| Planned change · spare staged ahead | |||
| Qwen2.5 72B | 1 H100 node | GPU drained | 0.20 s |
| Qwen3 1.7B | 3 L40S nodes | Two nodes drained | 0.59–1.68 s |
| Qwen2.5 0.5B | 2 H100 nodes, NVIDIA Dynamo | Pod deleted | 0.67 s |
| Failure · shared spare node | |||
| Llama 3.3 70B | 2 H100 nodes | Node lost | 13.1 s |
| Failure · spare GPU in the node | |||
| Qwen2.5 72B | 1 H100 node | GPU lost | 1.46–1.82 s |
Every session kept in every run. Stock vLLM shuts down on each of these and restarts in minutes.
Next: frontier MoE models on 8-GPU nodes.
Run on spot or idle capacity with a shared spare node behind it. When a node leaves, the spare takes over and nothing restarts.
CoreWeave H200 list prices, October 2026. The spare can be shared across engines.
Your router and scheduler keep their jobs. Everserve keeps the engine beneath them serving, so a lost GPU reaches the router as a short pause, not a dead replica.
Routes requests and balances replicas, unchanged. Its request migration stays on as the backstop.
Keeps every session when GPUs change underneath it. No fork of vLLM.
Decides where capacity goes, and calls Everserve's API to drain or reclaim a node.
Tested with NVIDIA Dynamo and Grove on Kubernetes.
We're looking for a small number of design partners among GPU clouds and model providers. We'll run the drill on your fleet and kill a GPU while you watch.
Cort Fisher, ex-AWS distributed systems engineer. LinkedIn