<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0">
    <channel>
        <title>Reading-Note/Pretraining - 标签 - vllbc02&#39;s blogs</title>
        <link>https://blog.vllbc.top/tags/reading-note/pretraining/</link>
        <description>Reading-Note/Pretraining - 标签 - vllbc02&#39;s blogs</description>
        <generator>Hugo -- gohugo.io</generator><language>zh-CN</language><copyright>This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.</copyright><lastBuildDate>Tue, 01 Sep 2026 00:00:00 &#43;0000</lastBuildDate><atom:link href="https://blog.vllbc.top/tags/reading-note/pretraining/" rel="self" type="application/rss+xml" /><item>
    <title>LLM 训练教程 · Ch12 端到端 CPT（继续预训练）</title>
    <link>https://blog.vllbc.top/cpt-%E7%BB%A7%E7%BB%AD%E9%A2%84%E8%AE%AD%E7%BB%83%E7%AC%94%E8%AE%B0/</link>
    <pubDate>Tue, 01 Sep 2026 00:00:00 &#43;0000</pubDate>
    <author>zsc</author>
    <guid>https://blog.vllbc.top/cpt-%E7%BB%A7%E7%BB%AD%E9%A2%84%E8%AE%AD%E7%BB%83%E7%AC%94%E8%AE%B0/</guid>
    <description><![CDATA[本章讲 Continued Pre-Training 的端到端实践：CPT 介于从零预训练与 SFT 之间，
负责&quot;学什么&quot;（知识注入）而非&quot;怎么说&quot;（行为塑造）。核心敌人是灾难性遗忘——
激进训练会把权重从通用分布的平坦极小值拽进新领域的尖锐深谷。
三大支柱：数据混比（5-20%，动态采样）、优化器与 LR（1/10 峰值、短 warmup、
重算 decay 总步数、丢弃旧 m/v）、双 PPL 监控诊断。附 7B 配置清单与 5 大陷阱查表。]]></description>
</item>
</channel>
</rss>
