wehub-resource-sync
54 lines
1.4 KiB
ReStructuredText
54 lines
1.4 KiB
ReStructuredText
lmcache tool
|
|
============
|
|
|
|
The ``lmcache tool`` command groups offline analysis utilities bundled with
|
|
LMCache.
|
|
|
|
.. code-block:: bash
|
|
|
|
lmcache tool <tool-name> <action> [options]
|
|
|
|
Currently one tool is available: ``cache-simulator``.
|
|
|
|
.. note::
|
|
|
|
``cache-simulator`` depends on the optional ``plot`` extras
|
|
(``sortedcontainers`` / ``matplotlib``). If they are not installed, the
|
|
sub-command is silently omitted from the CLI. Install the extras to enable
|
|
it.
|
|
|
|
|
|
cache-simulator
|
|
---------------
|
|
|
|
Replay LMCache lookup-hash JSONL logs through an LRU cache to measure the
|
|
KV-cache token hit rate. It has three actions:
|
|
|
|
.. code-block:: bash
|
|
|
|
lmcache tool cache-simulator {simulate,sweep,gen-dataset} [options]
|
|
|
|
.. list-table::
|
|
:header-rows: 1
|
|
:widths: 20 80
|
|
|
|
* - Action
|
|
- Description
|
|
* - ``simulate``
|
|
- Replay logs at a fixed cache capacity; print a text report and save a
|
|
7-panel statistics PNG.
|
|
* - ``sweep``
|
|
- Sweep across a range of cache capacities and save a hit-rate vs.
|
|
capacity PNG.
|
|
* - ``gen-dataset``
|
|
- Generate a ``vllm bench serve`` custom dataset (JSONL) from
|
|
lookup-hash JSONL logs, preserving prefix-sharing structure.
|
|
|
|
Each action has its own flags. Run the built-in help for the full list:
|
|
|
|
.. code-block:: bash
|
|
|
|
lmcache tool cache-simulator simulate --help
|
|
lmcache tool cache-simulator sweep --help
|
|
lmcache tool cache-simulator gen-dataset --help
|