Information Theory and Coding

Entropy

• What Is Information Theory?

• What is entropy in Information Theory?

• Average Information Content of Symbols

• What is Information rate?

• Joint Entropy and Conditional Entropy

• Chain rule for Entropy

• What is relative Entropy?

• What is Mitual Information?

• What is Chain rule of Mutual Information?

• What is Chain Rule for Distance?

• What is data processing inequality?

Data Compression

• What is Data Compression?

• What do you mean by "Codes" in data compression?

• What are the Basic properties of codes?

• How to construct an Instantaneous Codes?

• State Kraft Inequality with example

• Explain Shannon – Fano Binary Encoding Method

• What is Huffman Coding?

• Explain Arithmetic Coding with example

Channel Capacity

• What is Channel in information theory?

• What is information channel?

• What is Discrete Channel?

• What is discrete memoryless channel (DMC)?

• What is Binary Symmetric Channel (BSC)?

• Binary Erasure Channel (BEC)

• What is the Mutual Information of a BSC channel?

• What is Channel Coding and why channel coding is important?

• What is Channel Capacity?

• Shannon’s theorem: on channel capacity

• What is Gaussian Channel?

Error Control Coding

• What is Error Correcting Code?

• Some Important Definitions on Error Correcting Codes

• What do you mean by Linear Block Code?

• Generation Matrix and codeword for a linear Block Code

• How to check whether there is any errors in receiving message?

• Systematic code vs Non-systematic code

• How to generate Generation Matrix?

• What is Cyclic Code?

• Encoding the cyclic Code: Non-Systematic Encoding

• Encoding and decoding the cyclic Code: Non-Systematic Encoding

• Binary Convolutional Codes

Advanced Coding Techniques

ABOUT

Statlearner

Statlearner STUDY

Statlearner