OctoInformBeta

OpenAI preprint, September 2026 · §2. Entropy between two scales

Lemma 2.4: For compactly supported probabilities, with explicit errors at the two endpoint scales

Intermediate deficits control two-point entropy gains.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

Entropy between two scales records the distribution of mass across an interval of resolutions. For a fair two-point convolution, the distance between the two translations supplies an intermediate scale.

A measure μ\mu on ℝ\mathbb R is compactly supported if there is a number B≥0B\geq 0 such that μ(ℝ\[−B,B])=0\mu(\mathbb R\setminus[-B,B])=0.✦ For a real random variable ZZ, write ℒ(Z)\mathcal L(Z) for its probability law.

Grid Entropy and Translations

For a measure ν\nu on a measurable space Ω\Omega, a map f:Ω→Ef:\Omega\to E, and a label b∈Eb\in E, the fiber mass mν(f,b)m_\nu(f,b) is the real mass of the fiber over bb, as in (1).✦ The partition entropy Hν(f)H_\nu(f) of a map f:Ω→Ef:\Omega\to E with respect to a measure ν\nu on Ω\Omega is the sum of the entropy contributions of its fiber masses mν(f,b)m_\nu(f,b), as in (2). mν(f,b)=ν({z∈Ω:f(z)=b}).m_\nu(f,b)=\nu\bigl(\{z\in\Omega:f(z)=b\}\bigr).(1)✦ Hν(f)=∑b∈E−mν(f,b)ln⁡mν(f,b),0ln⁡0=0.H_\nu(f)=\sum_{b\in E}-m_\nu(f,b)\ln m_\nu(f,b), \qquad 0\ln 0=0.(2)✦ All entropies here use natural logarithms; division by ln⁡2\ln 2 expresses them in bits.

The shifted grid cell index qs,u:ℝ→ℤq_{s,u}:\mathbb R\to\mathbb Z at scale ss and offset uu assigns the integer in (3). The shifted grid entropy Hsu(ν)H_s^u(\nu) of a measure ν\nu on ℝ\mathbb R is its partition entropy under the shifted grid cell index qs,uq_{s,u}, and the averaged grid entropy Hs(ν)H_s(\nu) is its Lebesgue average over u∈[0,1]u\in[0,1], as in (3). qs,u(z)=⌊zs+u⌋,Hsu(ν)=Hν(qs,u),Hs(ν)=∫01Hsu(ν)du.q_{s,u}(z)=\left\lfloor\frac{z}{s}+u\right\rfloor, \qquad H_s^u(\nu)=H_\nu(q_{s,u}), \qquad H_s(\nu)=\int_0^1 H_s^u(\nu)\,du.(3)

The entropy window Gs,t(ν)G_{s,t}(\nu) of a measure ν\nu on ℝ\mathbb R is the difference of its averaged grid entropies at ss and tt.✦ The dyadic entropy deficit Δs(ν)\Delta_s(\nu) of a measure ν\nu on ℝ\mathbb R is ln⁡2\ln 2 minus its entropy window from ss to 2s2s, as in (4).✦ Gs,t(ν)=Hs(ν)−Ht(ν),Δs(ν)=ln⁡2−Gs,2s(ν).G_{s,t}(\nu)=H_s(\nu)-H_t(\nu), \qquad \Delta_s(\nu)=\ln 2-G_{s,2s}(\nu).(4)

The translate τbν\tau_b\nu of a measure ν\nu on ℝ\mathbb R is its image under z↦z+bz\mapsto z+b.✦ For a measure ν\nu on ℝ\mathbb R, the fair two-point convolution FvνF_v\nu and the equal mixture of translates Px,yνP_{x,y}\nu are the measures given in (5). Fvν=12ν+12τvν,Px,yν=12τxν+12τyν.F_v\nu=\frac12\nu+\frac12\tau_v\nu, \qquad P_{x,y}\nu=\frac12\tau_x\nu+\frac12\tau_y\nu.(5)

The Claim

Theorem 1. Let μ\mu be a compactly supported probability measure on ℝ\mathbb R. For scales 0<s<v<t0<s<v<t, the fair two-point convolution satisfies Gs,t(Fvμ)−Gs,t(μ)≥Δv(μ)−(sv+vt)ln⁡2.G_{s,t}(F_v\mu)-G_{s,t}(\mu) \geq \Delta_v(\mu)-\left(\frac{s}{v}+\frac{v}{t}\right)\ln 2.(6)✦ For x,y∈ℝx,y\in\mathbb R and scales 0<s<|y−x|<t0<s<|y-x|<t, the equal mixture of translates satisfies Gs,t(Px,yμ)−Gs,t(μ)≥Δ|y−x|(μ)−(s|y−x|+|y−x|t)ln⁡2.G_{s,t}(P_{x,y}\mu)-G_{s,t}(\mu) \geq \Delta_{|y-x|}(\mu) -\left(\frac{s}{|y-x|}+\frac{|y-x|}{t}\right)\ln 2.(7)✦

In the paper, the result reads: for a bounded random variable XX, an independent fair bit JJ, and 0<s<v<t0<s<v<t, (Gs,t(ℒ(X+vJ))−Gs,t(ℒ(X)))/ln⁡2≥Δv(ℒ(X))/ln⁡2−s/v−v/t(G_{s,t}(\mathcal L(X+vJ))-G_{s,t}(\mathcal L(X)))/\ln 2 \geq\Delta_v(\mathcal L(X))/\ln 2-s/v-v/t, with the same inequality for convolution by any fair two-point law whose points have distance vv.

What Is Checked

The inequalities (6) and (7) are stated for compactly supported probability measures on ℝ\mathbb R, their averaged shifted-grid entropies, and strict separation of the two endpoint scales from the translation distance.