The very title of this book is borrowed from the information theory vocabulary,
and, quite naturally, it is an outline of this theory that will serve as an introduction.
The subject of information theory is the scientific study of communications. To this end it defines a quantitative measurement of the communicated content, i.e. information,and deals with two operations essential for communication techniques: source coding and channel encoding. Its main results are two fundamental theorems related to each of these operations. The possibility of channel encoding itself has been essentially revealed by information theory. That shows, to which point a brief summary of this theory is essential for its introduction. Apart from some capital knowledge of its possibilities and limits, the theory has, however, hardly contributed to the invention of means of implementation: whereas it is the necessary basis for the understanding of channel encoding, it by no means suffices for its description. The reader interested in information theory, but requiring more information than is provided in this brief introduction, may refer to [1], which also contains broader bibliographical references.
To start with, we will comment on the model of a communication, known as
the Shannon paradigm after the American engineer and mathematician Claude E.
Shannon, born in 1916, who set down the foundations for information theory and
established the principal results [2], in particular, two fundamental theorems. This model is represented in Figure 1.1. A source generates a message directed
to a recipient. The source and the recipient are two separated, and therefore distant, entities, but between them there exists a channel, which, on the one
hand, is the medium of the propagation phenomena, in the sense that an excitation of its receptor by the source leads to a response observable by the recipient at the exit, and, on the other hand, of the disturbance phenomena. Due to the latter, the excitation applied is not enough to determine with certainty the response of the channel. The recipient cannot perceive the message transmitted other than by observing the response of the channel.