11 · Information

Third hearing of Part II. Chapters 09 and 10 measured what computation can do and how long it takes; this one asks what computation actually moves when it runs at all — and whether that thing is the whole world, or a very useful abstraction from it.

Is information the stuff the world is made of?

Claude Shannon opened "A Mathematical Theory of Communication" in the Bell System Technical Journal in July and October 1948 with a sentence that would outlive the telephone-switching problem it was written to solve: "Frequently the messages have meaning; that is, they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem." Chapter 00 quoted the line to mark computing's original forgetting; this chapter audits what the bracket actually bought, and what it cost. What Shannon needed was a way to quantify how much a signal could be compressed and how reliably it could survive noise, regardless of what it was about — and his measure, entropy, H = −Σp log p, does exactly that. A page of Shakespeare and a page of the same length generated by a fair coin flip can carry identical Shannon information if their symbol frequencies match, because the quantity is about surprise, not significance. The bracket was honest and correct for the problem in front of him. It has been treated as a metaphysics ever since — a discipline that can measure information without asking what it is of eventually builds systems it can quantify and cannot interpret, and that gap is this chapter's subject.

Shannon's entropy measures a message against a known probability distribution of possible messages; it says nothing about a single object's own intrinsic structure. Andrei Kolmogorov's "Three Approaches to the Quantitative Definition of Information" (Problems of Information Transmission 1, no. 1, 1965) supplied that: the complexity of a string is the length of the shortest program that outputs it and stops. A million zeros compresses to a short program ("print 0, one million times"); a string of the same length generated by fair coin flips compresses, with overwhelming probability, to nothing shorter than itself — a formal definition of randomness as incompressibility. Kolmogorov was not first. Ray Solomonoff had described algorithmic probability independently at Caltech in 1960 and published "A Formal Theory of Inductive Inference" (Information and Control 7, 1964: 1–22, 224–254) for a different purpose entirely: not measuring randomness but formalizing induction itself. Solomonoff's universal prior weights every hypothesis by the length of the shortest program that generates the observed data, turning "prefer the simplest explanation" from a maxim into an equation — Occam's razor with a ruler. Solomonoff's work went almost unnoticed until Kolmogorov's later, more rigorously presented papers cited it; priority runs one way, fame ran the other, and Jorma Rissanen's minimum description length principle (1978) turned the same idea into a working tool for choosing between statistical models.

Rolf Landauer supplied the fact that keeps information from floating free of physics altogether. "Irreversibility and Heat Generation in the Computing Process" (IBM Journal of Research and Development 5, 1961: 183–191) showed that erasing one bit of memory — a logically irreversible operation, since the bit's prior state cannot be recovered from its blank successor — necessarily dissipates a minimum quantity of heat, on the order of kT ln 2. The claim sat as a theoretical curiosity for half a century until Antoine Bérut and colleagues built a single colloidal particle in a bistable potential and measured the predicted heat directly, published as "Experimental Verification of Landauer's Principle Linking Information and Thermodynamics" (Nature 483, 2012: 187–189). Landauer's own slogan, "information is physical," is not a metaphor: a bit erased anywhere in the universe has a thermodynamic price tag, which is exactly why Maxwell's demon — the imagined creature that seems to violate the second law of thermodynamics by sorting fast molecules from slow ones for free — turns out to owe its debt not at the sorting but at the moment it must erase its own memory of which molecule went where.

John Archibald Wheeler took the physical entanglement of information and pushed it all the way to ontology. "Information, Physics, Quantum: The Search for Links," delivered at a Santa Fe Institute meeting in 1989 and published the following year, proposed that "every it — every particle, every field of force, even the spacetime continuum itself — derives its function, its meaning, its very existence entirely" from "the apparatus-elicited answers to yes-or-no questions, binary choices, bits." Existence, on this reading, is downstream of measurement. Luciano Floridi's philosophy of information generalizes the same instinct into a proposed successor discipline to metaphysics, built on the claim that reality is best described at "levels of abstraction" defined by the observables a system tracks — informational structure, not stuff, as what is ultimately real.

Here the examination turns, because a fact about physical cost is not the same claim as a fact about physical constitution, and philosophy is entitled to press exactly that distinction. Landauer proved that erasing information costs energy; he did not thereby prove that energy, mass, and spacetime are made of information rather than merely correlated with it whenever computation happens inside them. Floridi's own framework quietly concedes the point by making "information" relative to a chosen level of abstraction — a physicist's level and a biologist's level carve the same event into different informational units, which is difficult to square with information being the one substrate underneath both. Wheeler's participatory universe remains a research program rather than an established result, and most physicists who work on quantum foundations treat "it from bit" as a suggestive slogan, not a settled dependency. The identification overreaches exactly where Shannon's original bracket was most honest: a theory that measures information while setting meaning aside cannot, without further argument, turn around and claim that meaning was never there to set aside.

Connections

Chapter 00 quoted Shannon's bracket to mark computing's forgetting; this chapter audits it in full. Chapter 01 keeps to Shannon's earlier, narrower 1937 circuit work. Chapter 02 takes up Floridi's informational realism as a question about what a program is. Chapter 10 supplies the sibling notion — computational complexity measures time, Kolmogorov complexity measures description length, and the two rarely coincide. Chapter 15 takes digital physics — it-from-bit's descendants, from Zuse to Wolfram — to the question of whether the universe itself computes.

Open questions

If Kolmogorov complexity is uncomputable in general — no algorithm can determine the shortest program for an arbitrary string — what does that limit say about how far Occam's razor can ever be mechanized? Does Landauer's principle really license "information is physical," or only "erasing information has a physical cost," and is the difference between those two claims the whole of the dispute over digital physics? And if levels of abstraction are chosen by observers rather than discovered in nature, can philosophy of information still claim to be describing what is real rather than what is useful to track?

Go deeper

• James Gleick, The Information: A History, a Theory, a Flood (New York: Pantheon, 2011).
• Claude Shannon, "A Mathematical Theory of Communication," Bell System Technical Journal 27 (1948): 379–423, 623–656; Ray Solomonoff, "A Formal Theory of Inductive Inference," Information and Control 7 (1964): 1–22, 224–254.
• Stanford Encyclopedia of Philosophy, "Information" — plato.stanford.edu/entries/information.