Megatron-LM

What it does

Megatron-LM is NVIDIA's open-source framework for training transformer models at scale on GPUs, with Megatron Core as its library of building blocks. It combines tensor, pipeline, context, expert and data parallelism with mixed precision down to FP8 and FP4, and covers post-training, quantization, reinforcement learning and export to inference engines such as TensorRT-LLM.

Compare with

• vsNanotron
• vsDeepSpeed
• vsHorovod

People who use it also use

Not enough marks yet to tell. When people who use Megatron-LM mark the rest of their stack, it shows up here.