Megatron-DeepSpeed is queued for analysis. Crawling candidates and ranking them takes a few minutes. This page updates itself when the list exists — you can also come back later.
What is known so far comes from the other direction:
These projects were analysed and named Megatron-DeepSpeed among their alternatives. The relationship is not symmetric — how Megatron-DeepSpeed rates them is a separate judgement, made when Megatron-DeepSpeed is analysed in its own right.
calls Megatron-DeepSpeed “Same Megatron+DeepSpeed training stack but unmaintained for over two years; choose it only when reproducing BLOOM-era work.”
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries