wehub-resource-syncExamples vLLM + LMCache w. remote backends
LMCache should be able to reduce the generation time of the second and following calls.
We have examples for the following backends:
- Infinistore:
infinistore/ - Mooncake:
mooncakestore/ - External:
external/
LMCache should be able to reduce the generation time of the second and following calls.
We have examples for the following backends:
infinistore/mooncakestore/external/