llm-d/llm-d-kv-cache

Speculative Indexing for the KV-Cache Aware Routing 17 days ago

help wanted lifecycle/rotten

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Add support for externally invalidate/reset the KV cache 17 days ago

good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

FS Backend: Enable unit tests in CI 3 months ago

good first issue kv-connector lifecycle/stale

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Align repository with llm-d repo template (7 items) 3 months ago

help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

PVC Evictor optimizations 4 months ago

enhancement good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Abstract KVEvents API 4 months ago

good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Profile the different `kvblock.Index` implementations 5 months ago

good first issue kvblock-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Test different attention mechanisms 5 months ago

good first issue lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

[Performance] Code Profiling 7 months ago

enhancement good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

E2E Tests 7 months ago

good first issue help wanted lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Document and test the `extra` field in block hashes 7 months ago

good first issue kvevents-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

KVEvents Testing 7 months ago

good first issue kvevents-pkg lifecycle/rotten

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Benchmark UDS vs embedded tokenizers 7 months ago

good first issue help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Enable pre-commit checks in CI 7 months ago

good first issue

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Add UDS tokenizer tests to CI 7 months ago

good first issue python

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Update examples to support UDS-only builds (no embedded tokenizers) 7 months ago

good first issue help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

E2E tests for UDS tokenizer 7 months ago

AI Summary: The UDS tokenizer is being integrated into the main codebase. To ensure its proper functioning with the live service, end-to-end tests are required, as current unit tests only mock its behavior and do not interact with the actual external service.

Complexity: 4/5

help wanted

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

[CI] examples test action takes 20 mins 8 months ago

good first issue

llm-d/llm-d-kv-cache

171

Distributed KV cache scheduling & offloading libraries

Go

#ai#incubating

Open Issues Need Help