wehub-resource-sync
14 lines
432 B
ReStructuredText
14 lines
432 B
ReStructuredText
Compression
|
|
===========
|
|
|
|
KV cache compression can greatly reduces the size of the cache, which can be beneficial for both storage/memory usage and loading speed.
|
|
Currently, we support the following compression algorithms:
|
|
|
|
- :ref:`CacheGen <cachegen>`: `CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving <https://dl.acm.org/doi/10.1145/3651890.3672274>`_
|
|
|
|
|
|
.. toctree::
|
|
:maxdepth: 1
|
|
|
|
cachegen
|