COG-GTM/llama.cpp

llama : integer type consistency in `llama.h` about 2 months ago

enhancement good first issue refactoring roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama cpp server cant open to public 3 months ago

enhancement good first issue server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Mapping model name to LoRA config 4 months ago

enhancement good first issue server

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

kubernetes example 5 months ago

enhancement help wanted server/webui kubernetes

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server : add "token healing" support 5 months ago

enhancement good first issue roadmap server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Study how LM Evaluation Harness works and try to implement it 5 months ago

enhancement help wanted research 🔬 high priority generation quality

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Test replit-code-v1-3b model 5 months ago

help wanted model

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Support CoreML like whisper.cpp? 5 months ago

help wanted performance macos

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

feature request - disabling tokenizer in conversion / inference 5 months ago

enhancement good first issue help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Eos and bos tokens can be redefined as additional tokens with other ids 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

[IDEA] Global token enhancement/depression 5 months ago

help wanted research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

mpi : attempt inference of 65B LLaMA on a cluster of Raspberry Pis 5 months ago

help wanted research 🔬 🦙. hardware

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Using MPI w/ 65b model but each node uses the full RAM. 5 months ago

help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

CUDA non-determinism on identical requests 5 months ago

bug good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : combined beam search + grammar sampling strategy 5 months ago

good first issue roadmap research 🔬 generation quality

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Request: Nougat OCR Integration 5 months ago

help wanted model

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

convert.py incorrectly detects LLaMAv1 65B as a LLaMAv2 model 5 months ago

good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ci : fix Docker workflow 5 months ago

help wanted build

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

GGUF endianness cannot be determined from GGUF itself 5 months ago

enhancement good first issue breaking change

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server : improvements and maintenance 5 months ago

help wanted refactoring roadmap server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

in situ auto-Frankenmerges 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Include `eos_token` and `bos_token` from `tokenizer_config.json` for chat templating 5 months ago

enhancement help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : refactor the llm.build_xxx functions 5 months ago

good first issue refactoring roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

MiniCPM 2b model support? 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Support for Sparse MoE models like Camelidae and Sparsetral 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llava-cli: improve llava-cli and the API for using LLaVA 5 months ago

enhancement good first issue help wanted llava

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

split: allow --split-max-size option 5 months ago

enhancement good first issue help wanted split

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

split: include the option in ./convert.py and quantize 5 months ago

enhancement good first issue help wanted split

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: exit failure if `--embedding` is set with an incoherent `--ubatch-size` 5 months ago

enhancement good first issue help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Docker: Embedding Issue & possible Fix 5 months ago

bug good first issue server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: doc: document the `--defrag-thold` option 5 months ago

documentation enhancement help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: support control vectors 5 months ago

enhancement good first issue server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Question: How to generate an MPS gputrace 5 months ago

help wanted high priority

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

common: download from URL, improve parallel download progress status 5 months ago

enhancement good first issue help wanted split

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Server: Add prompt processing progress endpoint? 5 months ago

enhancement help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: process prompt fairly accross slots 5 months ago

enhancement good first issue help wanted server/webui

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Inference not running when using a tokenizer with word model 5 months ago

bug help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add GPU support for Mamba models 5 months ago

enhancement help wanted roadmap Nvidia GPU

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

tts : add support for Orpheus 5 months ago

good first issue devin tts

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Support for Phi-3 models 5 months ago

good first issue model

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server: avoid full prompt eval when 'prompt >= ctx' 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Llava functions compiled as extern "C" throw exceptions 5 months ago

bug good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Refactor: Existing examples refactoring opportunities 5 months ago

help wanted refactoring

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add DirectML backend 5 months ago

help wanted roadmap research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add WebGPU backend 5 months ago

help wanted roadmap research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Installable package via winget 5 months ago

enhancement help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Improve `cvector-generator` 5 months ago

enhancement good first issue help wanted

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Refactor: Add more typechecking to GGUFWriter.add_key_value 5 months ago

help wanted refactoring

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Bug: Failed to run qwen2-57b-a14b-instruct-fp16. 5 months ago

bug good first issue high severity

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Bug: Unexpected output length (Only one token response!) when set configs "-n -2 -c 256" for llama-server 5 months ago

bug good first issue low severity

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

ggml : add ANE backend 5 months ago

help wanted roadmap research 🔬

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: llama-server hot swapping cvectors via API like we can do with LoRA adapters now 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

llama : add CLI assistant 5 months ago

enhancement good first issue roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Misc. bug: server provides strutured output for response_format: json_object, but not for response_format: json_schema 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Compile bug: ios swift xcode build error when upgrade to llama : use cmake for swift build 5 months ago

good first issue help wanted build

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Allow Filtering LLama Server Response Fields 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

examples : add configuration presets 5 months ago

documentation enhancement good first issue help wanted examples

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

server : crash when -b > -ub with embeddings 5 months ago

bug good first issue server embeddings

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

tutorials : list for llama.cpp 5 months ago

good first issue help wanted roadmap

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Feature Request: Implement missing ops from backends 5 months ago

enhancement good first issue

COG-GTM/llama.cpp

0

LLM inference in C/C++

C++

Open Issues Need Help