https://jjl357.github.io/blog/tags/reasoning-model/ Recent content in Reasoning Model on JJ's Blog Hugo -- 0.152.2 zh-cn Sat, 25 Oct 2025 00:00:00 +0000 https://jjl357.github.io/blog/posts/r-kv---redundancy-aware-kv-cache-compression-for-reasoning-models---neurips25/ Sat, 25 Oct 2025 00:00:00 +0000 https://jjl357.github.io/blog/posts/r-kv---redundancy-aware-kv-cache-compression-for-reasoning-models---neurips25/ <p><img loading="lazy" src="https://jjl357.github.io/blog/image/RKV/title.png"></p> <p><strong>Conference:</strong> NeurIPS'25</p> <p><strong>Github:</strong> <a href="https://github.com/Zefan-Cai/R-KV">https://github.com/Zefan-Cai/R-KV</a></p> <hr> <h2 id="my-thoughts">My Thoughts</h2> <p>R-KV 针对previous works在efficient reasoning中由于只依靠attention scores而造成过多地保留了redundant tokens(重复而且对推理没有信息增益的token),R-KV的重点我觉得是增加了Redundancy Estimation via Semantic Similarity来解决这个问题,就Evaluation的结果来说，R-KV的效果是非常好的，即提升了Throughput还maintain了performance，甚至在budget充足的情况下还可以做到提点。</p> <h2 id="1-motivation">1. Motivation</h2> <p>Reasoning models have demonstrated impressive performance in self-reflection and chain-of-thought reasoning. However, they often produce excessively long outputs, leading to prohibitively large key-value (KV) caches during inference.</p> <p>For instance, a <strong>DeepSeek-R1-Distill-Llama-8B</strong> model may generate <strong>32K tokens</strong> to solve a complex math problem, consuming <strong>15.5GB</strong> of memory to load model weights and <strong>4.1GB</strong> to store the KV cache. This long CoT (chain-of-thought) generation necessitates the development of <strong>KV cache compression</strong>.</p>