bolubo/checkpoint-governor

measure the client glue at 100k steps and decide whether to throttle step() about 8 hours ago

help wanted good first issue kind/test area/experiments area/python

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

waste: a Megatron row with an explicit interval about 8 hours ago

enhancement help wanted good first issue kind/feature

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

demo: a --json flag, with a test about 8 hours ago

enhancement help wanted good first issue kind/feature area/python

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

examples: a Composer loop that uses the callback about 8 hours ago

documentation help wanted good first issue kind/docs area/python

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

examples: a Lightning loop that uses the callback about 8 hours ago

documentation help wanted good first issue kind/docs area/python

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

bench: a benchmark for the directory scan about 8 hours ago

enhancement help wanted good first issue kind/test area/controller

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

test: the README's code blocks should be executable about 8 hours ago

enhancement help wanted good first issue kind/test

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

ckptctl world --fake --evicted about 8 hours ago

enhancement help wanted good first issue kind/feature area/cli

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

ckptctl run: a machine-readable receipt about 8 hours ago

enhancement help wanted good first issue kind/feature area/cli

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

ckptctl explain --json about 8 hours ago

enhancement help wanted good first issue kind/feature area/cli

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

CONTRIBUTING: how to run one experiment about 8 hours ago

documentation help wanted good first issue kind/docs

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

docs: a troubleshooting page built from the messages the CLI prints about 8 hours ago

documentation help wanted good first issue kind/docs

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

docs: a first-ten-minutes page about 8 hours ago

documentation help wanted good first issue kind/docs

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

docs: a table of probes and their intervals 3 days ago

documentation good first issue

bolubo/checkpoint-governor

2

One process that decides when a training job writes a checkpoint, and explains every decision.

Go

#checkpoint#checkpointing#controller#distributed-training#fault-tolerance#golang#kubernetes#training-infrastructure

Open Issues Need Help