Tsinghua open-sources AKV, a cache engineering server for LLM agents on SGLang
Tsinghua researchers have released AKV, a serving system built on SGLang that lets LLM agents drop and reposition parts of their KV cache rather than summarise their history. Across four agent benchmarks, it cut average cost by 27.7%.