Megatron on vllbc02's blogs https://blog.vllbc.top/tags/megatron/ Recent content in Megatron on vllbc02's blogs Hugo -- gohugo.io zh-cn Fri, 06 Nov 2026 00:00:00 +0000 长上下文训练与 remove_padding https://blog.vllbc.top/visual-reading/long-context-training-remove-padding/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/long-context-training-remove-padding/ Tensor Parallel（Megatron） https://blog.vllbc.top/visual-reading/tensor-parallel-megatron/ Thu, 05 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/tensor-parallel-megatron/ Expert Parallel（MoE 训练） https://blog.vllbc.top/visual-reading/expert-parallel-moe-training/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/expert-parallel-moe-training/ Pipeline Parallel https://blog.vllbc.top/visual-reading/pipeline-parallel/ Thu, 05 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/pipeline-parallel/ Sequence Parallel & Context Parallel https://blog.vllbc.top/visual-reading/sequence-parallel-context-parallel/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/sequence-parallel-context-parallel/ Device Mesh 与 3D/4D 并行 https://blog.vllbc.top/visual-reading/device-mesh-3d-parallel/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/device-mesh-3d-parallel/ 训练显存、FLOPs 与参数量 https://blog.vllbc.top/visual-reading/training-memory-flops-params/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/training-memory-flops-params/ FP8 训练 https://blog.vllbc.top/visual-reading/fp8-low-precision-training/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/fp8-low-precision-training/ Megatron-LM 与 PAI-Megatron：训练框架怎么组织一次训练 https://blog.vllbc.top/visual-reading/megatron-lm-pai-megatron/ Fri, 06 Nov 2026 00:00:00 +0000 https://blog.vllbc.top/visual-reading/megatron-lm-pai-megatron/