DaBestCode/randkv

Add a SmolLM2 compatibility smoke result and support matrix 13 days ago

help wanted good first issue area: transformers

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Profile the Transformers compaction path with torch.profiler 13 days ago

help wanted good first issue area: evaluation

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Support batch-size-greater-than-one Transformers generation 13 days ago

enhancement help wanted area: transformers

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Run a reproducible NVIDIA vLLM serving benchmark 13 days ago

help wanted launch blocker needs GPU area: evaluation area: vllm

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Design the minimal vLLM integration boundary 13 days ago

help wanted launch blocker area: vllm

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Build a matched-budget reasoning quality evaluation harness 13 days ago

help wanted launch blocker area: evaluation

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Validate benchmark JSON with a versioned schema 13 days ago

documentation help wanted good first issue

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Add an offline tiny-model Transformers integration test 13 days ago

enhancement help wanted good first issue

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Add a deterministic SVG KV-retention visualization 13 days ago

enhancement help wanted good first issue

DaBestCode/randkv

1

Deterministic, prompt-protected random KV-cache eviction for Hugging Face Transformers.

Python

#attention#cache-compression#kv-cache#llm-inference#pytorch#transformers

Open Issues Need Help