rajveer43/VeloxQuant-MLX

KVQuantKVCache exposes a public .bits property, accidentally hijacking mlx_lm's SDPA dispatch to the quantized-matmul path — crashes on first attention call about 22 hours ago

bug good first issue

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

KIVIKVCache never quantizes decode tokens — residual fp16 window grows unboundedly instead of staying capped at residual_length about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

SinkProtectedKVCache selects sink positions from batch element 0 only, silently misprotecting other batch elements about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

SnapKVKVCache silently exceeds snap_budget under mlx_lm's chunked prefill (multiple S>1 update_and_fetch calls) about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

17 eviction/compression cache classes never populate mlx_lm's base .keys/.values/.offset — crash on .state access during ordinary generate() about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

QJLKVCache and PolarQuantKVCache crash in attend() once tokens exceed configured capacity about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

KVCacheConfig.sliding_window silently produces a broken cache for all mlx_lm-protocol methods (35+ of 40) about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

XQuantCoordinator never reclaims published anchor segments — generation hard-crashes at xquant_max_ctx tokens about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

MiniCacheCoordinator never reclaims published primary entries — generation hard-crashes at minicache_max_ctx tokens about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

zipcache_reconstruct hardcodes group_size=32, silently corrupting reconstruction for any other zipcache_group_size about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

KVTC entropy coding has no fixed-width fallback, so entropy_coding_gain regularly falls below 1.0 (contradicting its own docs) about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

pyramid_budgets silently exceeds avg_budget for beta > 2.0, breaking the fixed-memory-budget contract about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

water_fill_bits silently violates total_bit_budget when all eigenvalues are ~zero about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

apply_dual_transform_keys/queries: 3D per-head input silently falls into the GQA-averaging branch instead of erroring or broadcasting correctly about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

calibrate_layer_sensitivities() permanently corrupts model.make_cache if the calibration forward pass raises about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

allocate_bits_ratequant() can return bit-widths not in bit_choices when the set is non-contiguous about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

NpyArtifactStore: concurrent exists()-then-save() race can corrupt or duplicate-write shared artifact files about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

DistortionObserver.report() computes unweighted mean-of-batch-means MSE, badly skewed when batch sizes vary across events about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

PolarQuantizer: level-1 angle codebook assumes [0, 2pi) range but atan2 actually returns (-pi, pi] — severe silent reconstruction corruption about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

AdaptiveScalarCodebook.observe() silently drops calibration rows beyond n_calib within a single large batch about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

is_hadamard_compatible(d) returns True for d=12,20,28 which crash mx.hadamard_transform (uncaught Metal compile error, not a Python exception) about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

SortedChannelIndex.top_k() returns duplicate channels when magnitude re-converges; heap grows unbounded about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

core: EncodedVector.memory_bytes() undercounts outlier_idx about 22 hours ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

Two test directories (tests/ vs veloxquant_mlx/tests/) with no documentation of the split 3 days ago

good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

P0: mlx-lm used throughout but not declared as a dependency in pyproject.toml 3 days ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

P0: Fix broken Netlify quickstart — uses nonexistent APIs (bits=, KVCacheBuilder.build, kv_cache=, MemoryObserver.attach) 3 days ago

bug good first issue

rajveer43/VeloxQuant-MLX

11

implementation for Apple Silicon Faster KV cache quantization optimized for MLX

Python

#kv#kv-cache#llm#macos#macos-package#mlx#ondeviceai#ondevicemachinelearning#quantization

Open Issues Need Help