![\"\"](\"https:\/\/learning.oreilly.com\/api\/v2\/epubs\/urn:orm:book:9781482238129\/files\/bged.png\")
<\/p>\n\n\n\nWe would like to develop a measure for the rather abstract concept of<\/p>\n\n\n\n
information, so that e\ufb03ciencies of di\ufb00erent protocols or physical systems of<\/p>\n\n\n\n
communication can be compared, and more e\ufb03cient systems designed.<\/p>\n\n\n\n
We need to \ufb01rst have a model for the process we are describing, and Shan-<\/p>\n\n\n\n
non\u2019s proposed model has stuck (Figure 11.1). We start with a source of<\/p>\n\n\n\n
information, which, like a talking person or a buzzing telephone receiver, gen-<\/p>\n\n\n\n
erates messages using some prede\ufb01ned language consisting of symbols we<\/p>\n\n\n\n
will call the alphabet. The alphabet could, for example, be the set of English<\/p>\n\n\n\n
211<\/p>\n\n\n\n
<\/p>\n\n\n\n
212 Introduction to Quantum Physics and Information Processing<\/p>\n\n\n\n
FIGURE 11.1: Simpli\ufb01ed model for communication<\/p>\n\n\n\n
letters for communicating through a written message, or a set of \u201cdit\u201ds and<\/p>\n\n\n\n
\u201cda\u201ds for a message in Morse code, or 0\u2019s and 1\u2019s for a computerized message.<\/p>\n\n\n\n
Any message emitted by the source is then encoded for transmission using<\/p>\n\n\n\n
the physical system involved. For a talking person, the message is encoded in<\/p>\n\n\n\n
the vibration of the air molecules. For a telephonic message, the encoding is<\/p>\n\n\n\n
in terms of analog electric pulses in the wire. For the now-obsolete telegraph,<\/p>\n\n\n\n
encoding was done in binary dit\u2019s and da\u2019s represented as short or long electric<\/p>\n\n\n\n
pulses which we have now re\ufb01ned into the 0s and 1s of the modern binary<\/p>\n\n\n\n
computer. The encoded message is then transmitted across a channel. This is<\/p>\n\n\n\n
obvious as the air carrying the spoken message, the wire carrying the telegraph<\/p>\n\n\n\n
signal, or the optical cable transmitting long distance digital messages. In the<\/p>\n\n\n\n
context of quantum information, the channel may well just be the environment<\/p>\n\n\n\n
that the quantum system \ufb01nds itself in between computational steps.<\/p>\n\n\n\n
The importance of the channel in information theory lies in how much<\/p>\n\n\n\n
it costs in terms of its usage, and how it may distort or reduce the quality<\/p>\n\n\n\n
of the message being sent. We would need to quantify the compression of<\/p>\n\n\n\n
the message in order to more e\ufb03ciently utilize the resources, as well as the<\/p>\n\n\n\n
capacity of the channel to carry information in the presence of noise. Noise<\/p>\n\n\n\n
could be literal in the case of a talking person, random electrical signals in<\/p>\n\n\n\n
a telephone or telegraph wire or the computer circuitry, or the change in the<\/p>\n\n\n\n
state of a quantum system interacting inadvertently with its environment. At<\/p>\n\n\n\n
the other end of the communication model is the decoder and \ufb01nally the<\/p>\n\n\n\n
receiver of the message, which is either the ear of the audience, the ear-piece<\/p>\n\n\n\n
of the telephone, the printed output of the telegraph, or the monitor of the<\/p>\n\n\n\n
computer.<\/p>\n\n\n\n
We will now see how we can develop a measure for the information carried<\/p>\n\n\n\n
by a message, or a symbol in the message. The appearance of a symbol at the<\/p>\n\n\n\n
receiver\u2019s end is an event, whose probability can be predicted if we have some<\/p>\n\n\n\n
idea of the properties of the source. We will talk in the language of events and<\/p>\n\n\n\n
their probabilities of occurrence.<\/p>\n\n\n\n
Characterization of Quantum Information 213<\/p>\n\n\n\n
11.1.1 Classical picture: Shannon entropy<\/p>\n\n\n\n
According to Shannon, information associated with events is related to<\/p>\n\n\n\n
their probability of occurrence. Let\u2019s take a simple example. Suppose a magi-<\/p>\n\n\n\n
cian has hidden a ball in one of three boxes labeled 1, 2, and 3. Now he asks<\/p>\n\n\n\n
you to choose the box in which is ball is hidden. How would you choose? It de-<\/p>\n\n\n\n
pends on the information you have about the ball\u2019s location. In the beginning,<\/p>\n\n\n\n
you do not know which box it is in, so you think it is equally probable to be<\/p>\n\n\n\n
in any of the three boxes. Thus each box carries 1\/3 of the information about<\/p>\n\n\n\n
which box the ball is in. You\u2019d alternatively say that the ball is in each box<\/p>\n\n\n\n
with equal probability. The situation is thus described by an initial probability<\/p>\n\n\n\n
distribution P<\/p>\n\n\n\n
in<\/p>\n\n\n\n
:<\/p>\n\n\n\n
P<\/p>\n\n\n\n
in<\/p>\n\n\n\n
(ball is in 1) = 1\/3, P<\/p>\n\n\n\n
in<\/p>\n\n\n\n
(ball is in 2) = 1\/3, P<\/p>\n\n\n\n
in<\/p>\n\n\n\n
(ball is in 3) = 1\/3.<\/p>\n\n\n\n
An event such as opening any one box now will give you further information<\/p>\n\n\n\n
on where the ball is among the three. Suppose you open one box, say box 2, and<\/p>\n\n\n\n
the ball is not in it. The probability distribution has now changed: conditioned<\/p>\n\n\n\n
by the event of having opened box 2:<\/p>\n\n\n\n
P<\/p>\n\n\n\n
2<\/p>\n\n\n\n
(ball is in 1) = 1\/2, P<\/p>\n\n\n\n
2<\/p>\n\n\n\n
(ball is in 2) = 0, P<\/p>\n\n\n\n
2<\/p>\n\n\n\n
(ball is in 3) = 1\/2.<\/p>\n\n\n\n
Opening a box now gives information only about where the ball is in the two.<\/p>\n\n\n\n
On the other hand, what about the information with the magician for the<\/p>\n\n\n\n
same situation? He knows that he has put the ball in box1, so his distribution<\/p>\n\n\n\n
is<\/p>\n\n\n\n
P<\/p>\n\n\n\n
m<\/p>\n\n\n\n
(ball is in 1) = 1, P<\/p>\n\n\n\n
m<\/p>\n\n\n\n
(ball is in 2) = 0, P<\/p>\n\n\n\n
m<\/p>\n\n\n\n
(ball is in 3) = 0.<\/p>\n\n\n\n
In this case opening a box adds no information at all to the magician\u2019s knowl-<\/p>\n\n\n\n
edge!<\/p>\n\n\n\n
This example teaches us a few things:<\/p>\n\n\n\n
The information carried by an event is related inversely to its probability of<\/p>\n\n\n\n
occurrence before it has occurred. (After the event, the probability is of course<\/p>\n\n\n\n
1!) The more probable the occurrence, the less information the event carries.<\/p>\n\n\n\n
The occurrence of an event removes doubts about the possibilities before it<\/p>\n\n\n\n
occurs. The information it carries is thus the doubt it removes by occurring.<\/p>\n\n\n\n
The information carried by an event is changed if a previous event carries<\/p>\n\n\n\n
related information. If an event has happened, it carries no new information<\/p>\n\n\n\n
as compared to an event that has not yet happened.<\/p>\n\n\n\n
Let\u2019s now try to quantify the information I carried by an event E. If we<\/p>\n\n\n\n
are talking about messages encoded in symbols sent across a channel, then<\/p>\n\n\n\n
the event is the reading of the symbol by the receiver. The occurrence of a<\/p>\n\n\n\n
particular symbol x is the event E(x) with probability p(x),<\/p>\n\n\n\n
P<\/p>\n\n\n\n
x<\/p>\n\n\n\n
p(x) = 1. Now<\/p>\n\n\n\n
the mathematical formulation of information is concerned with the syntactic<\/p>\n\n\n\n
form of the message rather than the semantic. This means that we are not<\/p>\n\n\n\n
going to worry about the meaning conveyed by a message in a literal sense<\/p>\n\n\n\n
<\/p>\n\n\n\n
214 Introduction to Quantum Physics and Information Processing<\/p>\n\n\n\n
(which would depend on how the read symbol is translated by the receiver\u2019s<\/p>\n\n\n\n
brain), but rather by the form of the message itself, in terms of the symbols<\/p>\n\n\n\n
it carries. In other words, information carried by a symbol does not depend<\/p>\n\n\n\n
on which symbol it is, but only on our ignorance, or uncertainty, about its<\/p>\n\n\n\n
occurrence, before it is read. Thus the information carried by the symbol x<\/p>\n\n\n\n
on the occurrence of event E(x), must depend inversely on p(x).<\/p>\n\n\n\n
There are some properties we expect the information function to have.<\/p>\n\n\n\n
Suppose two events E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
and E<\/p>\n\n\n\n
2<\/p>\n\n\n\n
both occur. The information carried by this<\/p>\n\n\n\n
joint occurrence is I(E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
+ E<\/p>\n\n\n\n
2<\/p>\n\n\n\n
). If the result of one event E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
is revealed, then<\/p>\n\n\n\n
the information carried by the other event must be the di\ufb00erence: I(E<\/p>\n\n\n\n
2<\/p>\n\n\n\n
) =<\/p>\n\n\n\n
I(E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
+E<\/p>\n\n\n\n
2<\/p>\n\n\n\n
)\u2212I(E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
). Thus, the information carried by the joint event is the sum<\/p>\n\n\n\n
of the information carried by each. So if many events occur sequentially, then<\/p>\n\n\n\n
the information also builds up in the same order.<\/p>\n\n\n\n
Remember that if an event is certain to occur, then it carries no informa-<\/p>\n\n\n\n
tion. A certainty is represented by probability 1, so that I(1) = 0. (That\u2019s<\/p>\n\n\n\n
like a computer that is switched o\ufb00, so it reveals no information!)<\/p>\n\n\n\n
Collecting all these properties together, we require our mathematical in-<\/p>\n\n\n\n
formation I(E<\/p>\n\n\n\n
i<\/p>\n\n\n\n
) to be a function that satis\ufb01es<\/p>\n\n\n\n
1. inverse relation to probability: I(E<\/p>\n\n\n\n
i<\/p>\n\n\n\n
) \u223c<\/p>\n\n\n\n
1<\/p>\n\n\n\n
p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
,<\/p>\n\n\n\n
2. monotonously increasing, continuous function of p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
,<\/p>\n\n\n\n
3. additivity: I(E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
+ E<\/p>\n\n\n\n
2<\/p>\n\n\n\n
) = I(E<\/p>\n\n\n\n
1<\/p>\n\n\n\n
) + I(E<\/p>\n\n\n\n
2<\/p>\n\n\n\n
),<\/p>\n\n\n\n
4. identity corresponding to information of certainty, I(1) = 0.<\/p>\n\n\n\n
One function that satis\ufb01es all these properties is the logarithm. This led Shan-<\/p>\n\n\n\n
non to de\ufb01ne the self-information of an event E<\/p>\n\n\n\n
i<\/p>\n\n\n\n
as<\/p>\n\n\n\n
I(E<\/p>\n\n\n\n
i<\/p>\n\n\n\n
) =<\/p>\n\n\n\n
1<\/p>\n\n\n\n
log p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
= \u2212log p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
. (11.1)<\/p>\n\n\n\n
Here, the base of the logarithm denotes the units in which information is<\/p>\n\n\n\n
measured. If we use the natural logarithm then the unit of information is<\/p>\n\n\n\n
nats. If all events are binary in nature (yes-no answers), then the logarithm is<\/p>\n\n\n\n
taken to the base 2 and the units of information is bits. Note that the di\ufb00erence<\/p>\n\n\n\n
between di\ufb00erent units for information is a multiplicative constant since<\/p>\n\n\n\n
log<\/p>\n\n\n\n
b<\/p>\n\n\n\n
x = log<\/p>\n\n\n\n
b<\/p>\n\n\n\n
a log<\/p>\n\n\n\n
a<\/p>\n\n\n\n
x.<\/p>\n\n\n\n
Example 11.1.1. Let\u2019s calculate the information in bits carried by the drawing<\/p>\n\n\n\n
of a card out of a playing deck of 64 cards. To rephrase the problem, suppose a<\/p>\n\n\n\n
magician asks you to pull out a card at random, and then tries to guess which<\/p>\n\n\n\n
card you drew. The logical way for the magician to remove his ignorance<\/p>\n\n\n\n
<\/p>\n\n\n\n
Characterization of Quantum Information 215<\/p>\n\n\n\n
about the card would be to ask you questions about the card. (Of course<\/p>\n\n\n\n
he cannot ask you which card it is!) Since we want the answer in bits, the<\/p>\n\n\n\n
answers to these questions must be binary: \u201cyes\u201d or \u201cno.\u201d The problem then<\/p>\n\n\n\n
translates to how many such binary-answer questions the magician must ask<\/p>\n\n\n\n
for correctly guessing the number. The procedure he adopts is the \u201cbinary<\/p>\n\n\n\n
search\u201d algorithm of dividing the range of possible answers into two at each<\/p>\n\n\n\n
step and asking if the card is in one of the two ranges. Your yes or no will allow<\/p>\n\n\n\n
him to select one of the sections and further divide it. Consider the following<\/p>\n\n\n\n
sample scenario:<\/p>\n\n\n\n
Q 1. Is it between 1 and 32? Ans: No.<\/p>\n\n\n\n
Q 2. … between 33 and 49? Ans: Yes.<\/p>\n\n\n\n
Q 3. … 33 and 41? Ans: No.<\/p>\n\n\n\n
.<\/p>\n\n\n\n
.<\/p>\n\n\n\n
.<\/p>\n\n\n\n
Q n.<\/p>\n\n\n\n
What is the total number n of such questions? It\u2019s the number of times the<\/p>\n\n\n\n
range [1 \u2212 64] can be bifurcated, which, if you follow through the above se-<\/p>\n\n\n\n
quence of questions, will be 6.<\/p>\n\n\n\n
n = 6 = log<\/p>\n\n\n\n
2<\/p>\n\n\n\n
64 = \u2212log<\/p>\n\n\n\n
2<\/p>\n\n\n\n
1<\/p>\n\n\n\n
64<\/p>\n\n\n\n
,<\/p>\n\n\n\n
and 1\/64 is the probability of choosing one particular card out of the deck.<\/p>\n\n\n\n
Clearly this answer is independent of which card it is (the semantic meaning<\/p>\n\n\n\n
of the event).<\/p>\n\n\n\n
A message is the occurrence of a string of events: the appearance of each<\/p>\n\n\n\n
symbol constituting the message. Thus the total information carried by a<\/p>\n\n\n\n
message is the weighted average of all the symbols in the message. This is<\/p>\n\n\n\n
given by<\/p>\n\n\n\n
H(m) =<\/p>\n\n\n\n
X<\/p>\n\n\n\n
i<\/p>\n\n\n\n
p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
I(E<\/p>\n\n\n\n
i<\/p>\n\n\n\n
) = \u2212<\/p>\n\n\n\n
X<\/p>\n\n\n\n
i<\/p>\n\n\n\n
p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
log p<\/p>\n\n\n\n
i<\/p>\n\n\n\n
. (11.2)<\/p>\n\n\n\n
What if a particular symbol doesn\u2019t occur in the message? In that case too,<\/p>\n\n\n\n
p = 0. Even though log p is then unde\ufb01ned, the symbol cannot contribute<\/p>\n\n\n\n
to the information carried by the message, and for our purposes, we de\ufb01ne<\/p>\n\n\n\n
0 log 0 \u2261 0.<\/p>\n\n\n\n
From its similarity to the thermodynamic quantity of the same name, the<\/p>\n\n\n\n
information function H(m) has been called the entropy of the message. In sta-<\/p>\n\n\n\n
tistical thermodynamics, we seek to relate the macroscopic properties system<\/p>\n\n\n\n
to the microstates of its constituents. Notably, the energy of a gas is related<\/p>\n\n\n\n
to the momenta of its constituent molecules. If the number of microstates<\/p>\n","protected":false},"excerpt":{"rendered":"
We would like to develop a measure for the rather abstract concept of information, so that e\ufb03ciencies of di\ufb00erent protocols or physical systems of communication can be compared, and more e\ufb03cient systems designed. We need to \ufb01rst have a model for the process we are describing, and Shan- non\u2019s proposed model has stuck (Figure 11.1). […]<\/p>\n","protected":false},"author":1,"featured_media":4045,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[500],"tags":[],"class_list":["post-4136","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-characterization-of-quantum-information"],"jetpack_featured_media_url":"https:\/\/workhouse.sweetdishy.com\/wp-content\/uploads\/2024\/09\/quantum-computing-2.png","_links":{"self":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/4136","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/comments?post=4136"}],"version-history":[{"count":2,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/4136\/revisions"}],"predecessor-version":[{"id":4572,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/posts\/4136\/revisions\/4572"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media\/4045"}],"wp:attachment":[{"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/media?parent=4136"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/categories?post=4136"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/workhouse.sweetdishy.com\/index.php\/wp-json\/wp\/v2\/tags?post=4136"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}