When we manipulate physical systems for various purposes, we are essentially
encoding and decoding the information content in those systems in precisely
defined terms tailored to our purpose. Conversationally speaking of informa-
tion, one thinks of the “new knowledge” gained when a particular physical
process is completed, such as watching television, reading an article, or mea-
suring the output voltage at the end of a circuit. When we get used to the idea
that information is not something abstract that is a result of cognition, but is
actually physically carried by the system that’s being observed or measured,
we are closer to a scientific understanding of information.
Classically, information theory gained respectability when Claude Shannon
[63] quantified the information content in a physical system or communica-
tion channel. He was working at Bell Labs at that time and was interested in
optimizing telephonic communication. He also realized the importance of in-
formation in the context of data compression and cryptography. We will start
with his theory, and see how it can be adapted to describing the information
content of a quantum system. For a more in-depth treatment of the subject,
you can refer to the excellent book by Barnett [4]. Matters regarding quantum
information are beautifully discussed in the work by Mark Wilde [73], and in
Chapter 5 of Preskill’s lecture notes [57].
11.1 Measures of Information
We would like to develop a measure for the rather abstract concept of
information, so that efficiencies of different protocols or physical systems of
communication can be compared, and more efficient systems designed.
We need to first have a model for the process we are describing, and Shan-
non’s proposed model has stuck (Figure 11.1). We start with a source of
information, which, like a talking person or a buzzing telephone receiver, gen-
erates messages using some predefined language consisting of symbols we
will call the alphabet. The alphabet could, for example, be the set of English
Leave a Reply