熵坍塌
UloRL：An Ultra-Long Output Reinforcement Learning Approach for Advancing Large Language Models’ Reasoning Abilities
ProRL：Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models
BEYOND PASS@1：SELF-PLAY WITH VARIATIONAL PROBLEM SYNTHESIS SUSTAINS RLVR
Skywork Open Reasoner 1 Technical Report
利用熵
Beyond the 80 20 Rule：High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
First Return, Entropy-Eliciting Explore
AGENTIC REINFORCED POLICY OPTIMIZATION
CURE：Critical-Token-Guided Re-concatenation for Entropy-collapse Prevention
ETTRL：BALANCING EXPLORATION AND EXPLOITATION IN LLM TEST-TIME REINFORCEMENT LEARNING VIA ENTROPY MECHANISM
GTPO and GRPO-S：Token and Sequence-Level Reward Shaping with Policy Entropy
EDGE-GRPO：Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity
GTPO：Trajectory-Based Policy Optimization in Large Language Models
SSRL：SELF-SEARCH REINFORCEMENT LEARNING
利用熵来判断是依赖自己的内部知识还是依赖于外部工具。
如何缓解熵坍塌？
- clip-higher
- on-policy
- entropy loss
参考