wehub-resource-sync
19 lines
607 B
ReStructuredText
19 lines
607 B
ReStructuredText
KV Cache Size Calculator
|
|
========================
|
|
|
|
Use this interactive calculator to:
|
|
|
|
- **Calculate KV Cache Size**: Estimate the memory required for caching a specific number of tokens
|
|
- **Calculate Maximum Tokens**: Find out how many tokens you can cache given your available GPU RAM
|
|
|
|
This helps you plan memory requirements for your LMCache deployment.
|
|
|
|
.. raw:: html
|
|
|
|
<div style="margin: 20px 0;">
|
|
<iframe src="../_static/kv_cache_calculator.html"
|
|
style="width: 100%; height: 1400px; border: none;"
|
|
title="KV Cache Size Calculator">
|
|
</iframe>
|
|
</div>
|