My comments on Loet’s Chapter 4
klaus.krippendorff@asc.upenn.edu
Prepared for a Club of Remy discussion on 2022.3.16
Points of contention:
On the concept of information
Loet’s view Shannon/Ashby/my view
“Information can be defined as ‘uncertainty’ and is not ‘informative’ in the sense of reducing uncertainty”
Information received is the reduction of a receiver’s uncertainty after a sender’s messages become known.
=
Information sent is the reduction of uncertainty in the sender by what the receiver knows about the sender after the sender’s messages have been received.
=
We Agree:
Neither uncertainty nor the transmission of information has anything to do with quantities of meanings.
Bateson’s catchy definition:
“Information is the difference that makes a difference”
agrees that information reduces a receiver’s uncertainty provided the uncertainty is meaningful, important, or significant to the receiver.
On the concept of redundancy
Loet’s view Commonly
“Redundancy R is defined … as the fraction of the capacity of a communication channel (Hmax) that is not used.” (Hobserved)
R = (Hmax – Hobserved)/Hmax
Redundancy is the use of more characters (words, signals, time or information in Shannon’s sense) than needed to make the same difference (in Bateson’s sense of being meaningful).
Loet quoting Weaver acknowledges that redundancy might be a prime candidate for a theory of meaning. …
I agree, if a reader can guess missing parts of a message then there must be predictable connections between the parts received and the parts missing but guessed.
Correcting typos is another example of what the ordinary conception of redundancy enables.
However, meanings can be bypassed e.g. by spell checkers
I can’t see how channel capacity, = Loet’s measure of redundancy, comes close to the role that meanings could play in the transmission of information.
Interactions in the decomposition of entropies and information
Shannon’s measures take advantage of the mathematical fact that the logarithms of products of probabilities are additive. For example, the information transmitted between two variables can be expressed by
McGill & Garner (1954) have shown that as soon as one starts calculating with three or more variables, interaction terms Q are needed to account for the over-determination of lower level dependencies, e.g.:
T(A:B:C) =T(A:B)+T(A:C)+T(B:C)+Q(ABC)
The values of Q may turn out to be negative when the number of variables are uneven and positive otherwise.
QABC)= 0
On the calculus of redundancy
Loet’s view My view
In 1972, I could show that whenever systems contained circular relationships of adequate strength, the fractions whose logarithms defined the Q-values were no longer probabilities proper but the artifacts of adding and subtracting entropies. They had no observable reality.
Whenever three of more variables are involved one had to obtain interactions among them by an iterative algorithm I had developed.
Loet’s calculus of redundancies, (no longer expressed as a proportion of Hmax), adds and subtracts quantities of any combinations of variables. When negative, they no longer describe empirically realistic probability distributions, are artifacts of the analyst’s accounting equations.
Even if the space of relational measures is defined in terms of meanings as suggested by Shannon’s co-author W. Weaver, that conception of meaning is the most primitive one I can think of.
Sketch of my concept of information in evolutionary processes
To me: