Theory of Information and Coding
This book is a self-contained introduction to basic results in the theory of information and coding. It has three parts. The introduction contains a short and elementary overview leading the reader to basic concepts of coding. The first part is devoted to the main results of Shannon’s mathematical theory of communication. After a reference section collecting technical results about entropy, mutual information, etc., the author develops techniques that are necessary to prove Shannon’s fundamental results, the channel and source coding theorems. The first part ends with a survey of advanced results. The second part is devoted to a study of specific codes, which can be used for channel and source coding. It deals with linear codes in general, with cyclic codes, BCH, Reed-Solomon, and related codes. There are also chapters on convolutional codes and on variable-length source coding. The second part ends with a survey of advanced topics, as well. Appendices contain basic information on probability theory, convex functions, Jensen’s inequality, finite fields, and path enumeration in directed graphs. A prior knowledge of these areas is not a necessary prerequisite but it will certainly make it easier to follow the presentation.