KV cache is the real inference bottleneck (Not GPUs)

by Pop Goes the Stack