Info Theory and Coding

INFOTH Note 20: Channel Coding Schemes

Maximum Likelihood Decoding, Block Coding (Binary Linear, Hamming, etc.) & Polar Codes

INFOTH Note 19: Channel Coding Theorem for DMC 2

Proof of Shannon's Channel Coding Theorem, Converse Theorem

INFOTH Note 18: Channel Coding Theorem for DMC 1

Discrete Memoryless Channel, Shannon's Channel Coding Theorem (2nd)

INFOTH Note 9: Differential Entropy

Differential Entropy (for Real-valued/Continuous R.V.s)

INFOTH Note 8: BW Transform

Burrows-Wheeler Transform

INFOTH Note 7: Huffman & Arithmetic Coding

Huffman Coding and Arithmetic Coding

INFOTH Note 6: Kraft's Inequality, Shannon Code

Codes, Kraft's Inequality, Shannon Code

INFOTH Note 5: Fano Ineq., Entropy Rate, AEP

Fano's Inequality, Markov Chain, Entropy Rate, Asymptotic Equipartition Property and Lossless Compression

INFOTH Note 4: Cond. MI & Cond. KL, Data Processing Ineq.

Conditional Mutual Information, Conditional Relative Entropy, Chain Rule for these things, Properties, and Data Processing Inequality

INFOTH Note 3: Relative Ent. (KL), Mutual Info.

Log-sum Ineq., Relative Intropy (aka. KL Divergence) and Mutual Information