itsublog
Posts/by Series
Posts/by Year
All Posts
Friendly Links
Light
Dark
Auto
2025 Fall
Note: Gated DeltaNet & Qwen3-Next
Notes for Gated DeltaNet & Qwen3-Next.
2025-10-31
1 min read
Paper Reading
2025 Paper Reading
,
Architectures
Paper Reading: SEER (Structured Reasoning and Explanation via RL)
SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning.
2025-10-20
3 min read
Paper Reading
2025 Paper Reading
INFOTH Note 5: Fano Ineq., AEP
Fano's Inequality, Markov Chain, Entropy Rate, Asymptotic Equipartition Property and Lossless Compression
2025-09-16
3 min read
Notes
Information Theory and Coding (UCB FA24 EE229A, rewritten in 2025)
INFOTH Note 4: Cond. MI & Cond. KL, Data Processing Ineq.
Conditional Mutual Information, Conditional Relative Entropy, Chain Rule for these things, Properties, and Data Processing Inequality
2025-09-11
1 min read
Notes
Information Theory and Coding (UCB FA24 EE229A, rewritten in 2025)
INFOTH Note 3: Relative Ent. (KL), Mutual Info.
Log-sum Ineq., Relative Intropy (aka. KL Divergence) and Mutual Information
2025-09-04
1 min read
Notes
Information Theory and Coding (UCB FA24 EE229A, rewritten in 2025)
INFOTH Note 2: Joint & Conditional Entropy, Chain Rule
Definition of Many Entropies and the Chain Rule. We have a specific def for joint entropy.
2025-09-02
2 min read
Notes
Information Theory and Coding (UCB FA24 EE229A, rewritten in 2025)
INFOTH Note 1: Axioms & Def of Entropy
Shannon Entropy, Axioms, Entropy and Concavity
2025-08-28
2 min read
Notes
Information Theory and Coding (UCB FA24 EE229A, rewritten in 2025)
