NVIDIA-NeMo/Automodel

Fix PEFT adapter outputs for v5 support about 3 hours ago

bug good first issue MoE qa_rcca_done

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support Hy MT2 7 days ago

enhancement good first issue Feature

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

tools/merge_lora.py should save processor artifacts for VLMs 8 days ago

good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Profile and reduce Nemotron Nano single-GPU checkpoint load time 11 days ago

good first issue ckpt Performance ci automodel

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

VLM validation drops the batch dimension for tensor-valued hidden_states 12 days ago

good first issue community-request waiting-on-customer

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

fix(vlm): validation crashes with FusedLinearCrossEntropy — out.hidden_states[-1] indexes the batch dim when the model returns a single hidden-state tensor 12 days ago

bug good first issue community-request waiting-on-customer

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support gigatoken 12 days ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Gemma4 NaN with SDPA 17 days ago

good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Make StatefulDataloader be DP-aware about 1 month ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

staging-free consolidation for databricks about 2 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support MiMo-V2.5-Pro about 2 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support MiniCPM5-1B 2 months ago

enhancement good first issue community-request

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support Falcon H1 3 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support inclusionAI/Ling-mini/flash/1T-2.0 3 months ago

enhancement good first issue community-request

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

YAML linter 3 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

bug: MoE expert LoRA checkpoints incompatible with HF PEFT and vLLM (Nemotron Super 120B) 3 months ago

good first issue community-request

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support Windows RTX GPU for model finetuning 3 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support LLaVA-OneVision-1.5 Model Training 4 months ago

enhancement good first issue community-request waiting-on-maintainers

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

use lazy map for dataset preprocessing 4 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Optimize causal-conv1d/transformer engine installation in CI 4 months ago

enhancement good first issue Automation

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

(small) package deprecation 4 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support Qwen-Image finetune (T2I) 4 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

S3 + DCP 4 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Skypilot + k8s tutorial 4 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

workaround required for padding_mask 4 months ago

bug good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Support Skypilot 5 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

MFU logging in train recipes 5 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Some configs missing dataset.vocab_size in MockIterableDataset 5 months ago

bug good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

[Bug] Model summary reports incorrect "Total parameters" for Quantized Models (Llama 3.1 8B QLoRA) 6 months ago

good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Add get_input_embeddings method to nemo_automodel.components.models.gpt_oss.model.GptOssForCausalLM 6 months ago

bug good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Can you support DoRA? 6 months ago

enhancement good first issue community-request external x-tencent t-algo

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Distributed signal handling 6 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

num_epochs should not default to 10 if max_steps is specified in step scheduler 7 months ago

bug good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Add MoE load balancing metrics 8 months ago

enhancement good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

MCore Dataset needs to launch with uv instead of system python 8 months ago

good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

issue: missing FirstRankPerNode in recipes 11 months ago

good first issue Stale

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

fix(checkpointing): Dataloader state 12 months ago

good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

feat: deterministic job resume about 1 year ago

AI Summary: Implement deterministic job resume functionality for the NeMo AutoModel framework. This involves ensuring that resuming a training job from a saved state produces identical results to a continuous run, verifying the correct restoration of model, optimizer, dataloader, and data sampler states.

Complexity: 4/5

good first issue

NVIDIA-NeMo/Automodel

707

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

Python

#agent#deepseek-v3-2#deepseek-v4#finetuning#gemma3#gemma4#glm#gpt-oss#kimi-k2#llama#llama3#llm#minimax-m2#mistral#openai#qwen3#qwen3-6#qwen3-next#vlm

Open Issues Need Help