The Setting
Let be a random Gaussian integer taking finitely many values, and let be a Gaussian prime of norm . The residue of modulo is a random class in , a group of elements, and the residues of modulo several Gaussian primes are random variables on one probability space. How evenly a residue is spread is measured by its entropy; how much one residue tells about another, once a third is known, by a conditional mutual information.
The library defines these quantities for random variables on a finite probability space with values in any set.
Finite Laws and Entropy
A finite probability law on a finite set is a function with for every and (1)✦
A random variable on with values in a set is a map . When is finite, the law of is the pushforward of along , the finite probability law on given by (2)✦
The entropy of a finite probability law on a finite set is the Shannon sum (3)✦ with the natural logarithm. The entropy of a random variable with values in a finite set is the entropy of its law: (4)✦
The entropy of a random variable with values in an arbitrary set is the Shannon sum of the masses of its fibers, over the finitely many values that takes: (5)✦
The joint variable of random variables and on is the random variable , with values in the product of their value sets.
The Claim
Definition 1. Let be a finite probability law on a finite set . For random variables and on with values in finite sets, the conditional entropy of given is (6)✦ For random variables and on with values in arbitrary sets, the conditional entropy of given is (7)✦ and, for with values in a finite set, (8)✦ For random variables , and on with values in arbitrary sets, the conditional mutual information of and given is (9)✦
In the paper, the result reads: , and are the entropy, the conditional entropy and the conditional mutual information of random variables , and taking finitely many values.
What Is Checked
The claim is stated over finite probability laws on finite sets and their pushforwards (2) along maps into finite sets, with entropy given by the Shannon sum (3).