Pre-Norm - 标签 - vllbc02's blogs https://blog.vllbc.top/tags/pre-norm/ Pre-Norm - 标签 - vllbc02's blogs Hugo -- gohugo.iozh-CNThis work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.Sat, 06 Jun 2026 00:00:00 +0000 Post-Norm vs Pre-Norm https://blog.vllbc.top/post-norm-vs-pre-norm/ Sat, 06 Jun 2026 00:00:00 +0000 vllbc https://blog.vllbc.top/post-norm-vs-pre-norm/ Post-Norm vs Pre-Norm

来自 Sebastian Raschka 的 The Big LLM Architecture Comparison。汇总 2025 年 LLM 中 normalization 放置的各种设计。

一句话

Normalization 放在 attention / FFN 之前还是之后，对训练稳定性影响很大。2025 年的 LLM 至少分 4 派。

]]>