OctoInformBeta

OpenAI preprint, September 2026 · §2. Entropy between two scales

Lemma 2.1: Integer-ratio windows have bounds and finite-mixture concavity, with logarithmic errors for grid and scale comparisons

Averaged entropy respects affine scale changes.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

The entropy of a law on the line depends on the length and the origin of the intervals used to observe it. Averaged grid entropy measures this distribution of mass after averaging over the origin of the grid.

A measure μ\mu on ℝ\mathbb R is compactly carried if it assigns zero mass to the complement of a bounded closed interval, as in (1): μ(ℝ\[−B,B])=0for some B≥0.\mu\bigl(\mathbb R\setminus[-B,B]\bigr)=0 \qquad\text{for some }B\geq 0.(1)✦

For a measure ν\nu on a measurable space Ω\Omega and a map f:Ω→Af:\Omega\to A, the fiber mass mν,f(a)m_{\nu,f}(a) is the real mass of f−1{a}f^{-1}\{a\}✦; the partition entropy Hν(f)H_\nu(f) is the sum in (2), with 0ln⁡0=00\ln 0=0: mν,f(a)=ν(f−1{a}),Hν(f)=−∑a∈Amν,f(a)ln⁡mν,f(a).m_{\nu,f}(a)=\nu\bigl(f^{-1}\{a\}\bigr), \qquad H_\nu(f)=-\sum_{a\in A}m_{\nu,f}(a)\ln m_{\nu,f}(a).(2)

For a measure ν\nu on ℝ\mathbb R and real parameters s,us,u, the shifted grid cell index ℓsu:ℝ→ℤ\ell_s^u:\mathbb R\to\mathbb Z assigns a point to its cell at scale ss and offset uu; the shifted grid entropy Hsu(ν)H_s^u(\nu) is the partition entropy of this index, as defined in (3): ℓsu(x)={⌊x/s+u⌋,s≠0,⌊u⌋,s=0,Hsu(ν)=Hν(ℓsu).\ell_s^u(x)= \begin{cases} \lfloor x/s+u\rfloor,&s\neq 0,\\ \lfloor u\rfloor,&s=0, \end{cases} \qquad H_s^u(\nu)=H_\nu(\ell_s^u).(3)

For a measure ν\nu on ℝ\mathbb R and a real scale ss, the averaged grid entropy ℋs(ν)\mathcal H_s(\nu) is the mean of Hsu(ν)H_s^u(\nu) over one unit interval of offsets.✦ The entropy window Gs,t(ν)G_{s,t}(\nu) of a measure ν\nu on ℝ\mathbb R at real scales s,ts,t is the difference of its averaged entropies at the two scales, as in (4): ℋs(ν)=∫01Hsu(ν)du,Gs,t(ν)=ℋs(ν)−ℋt(ν).\mathcal H_s(\nu)=\int_0^1 H_s^u(\nu)\,du, \qquad G_{s,t}(\nu)=\mathcal H_s(\nu)-\mathcal H_t(\nu).(4)

These entropies use natural logarithms. A hat denotes division by ln⁡2\ln 2, giving entropy in bits. For a random variable, entropy notation refers to its law. For real parameters a,ba,b, write Ta,b(x)=ax+bT_{a,b}(x)=ax+b and τb=T1,b\tau_b=T_{1,b}; T#νT_\#\nu denotes the image measure under TT.

The Claim

Theorem 1. Let μ\mu be a compactly carried probability measure on ℝ\mathbb R and let s>0s>0. For a≠0a\neq 0 and b∈ℝb\in\mathbb R, affine covariance is (5): ℋs((Ta,b)#μ)=ℋs/|a|(μ).\mathcal H_s\bigl((T_{a,b})_\#\mu\bigr) =\mathcal H_{s/|a|}(\mu).(5)✦

For a positive integer NN, the integer-ratio window satisfies (6): 0≤Gs,Ns(μ)≤ln⁡N.0\leq G_{s,Ns}(\mu)\leq\ln N.(6)✦

Let (νi)i∈I(\nu_i)_{i\in I} be a finite family of compactly carried probability measures, and let wi≥0w_i\geq 0 with ∑i∈Iwi=1\sum_{i\in I}w_i=1. For their mixture η=∑i∈Iwiνi\eta=\sum_{i\in I}w_i\nu_i, finite-mixture concavity is (7): ∑i∈IwiGs,Ns(νi)≤Gs,Ns(η).\sum_{i\in I}w_iG_{s,Ns}(\nu_i)\leq G_{s,Ns}(\eta).(7)✦

For real translations (bi)i∈I(b_i)_{i\in I} and the same weights, the translated mixture ρ=∑i∈Iwi(τbi)#μ\rho=\sum_{i\in I}w_i(\tau_{b_i})_\#\mu satisfies (8): Gs,Ns(μ)≤Gs,Ns(ρ).G_{s,Ns}(\mu)\leq G_{s,Ns}(\rho).(8)✦

For every t>0t>0, the logarithmic scale comparison is (9): ℋs(μ)≤ℋt(μ)+ln⁡(t/s+2).\mathcal H_s(\mu)\leq\mathcal H_t(\mu)+\ln(t/s+2).(9)✦

When t≤st\leq s, the averaged and fixed-grid comparisons are (10) and (11), with arbitrary real offsets u,vu,v: ℋs(μ)≤ℋt(μ)+ln⁡2.\mathcal H_s(\mu)\leq\mathcal H_t(\mu)+\ln 2.(10)✦ Hsu(μ)≤Htv(μ)+ln⁡2.H_s^u(\mu)\leq H_t^v(\mu)+\ln 2.(11)✦

For every Borel measure ν\nu on ℝ\mathbb R and real r,u,br,u,b, grid translation is (12): Hru((τb)#ν)={Hru+b/r(ν),r≠0,H0u(ν),r=0.H_r^u\bigl((\tau_b)_\#\nu\bigr)= \begin{cases} H_r^{u+b/r}(\nu),&r\neq 0,\\ H_0^u(\nu),&r=0. \end{cases}(12)✦

For every real offset uu, the grid upper deviation is (13): Hsu(μ)≤ℋs(μ)+ln⁡2.H_s^u(\mu)\leq\mathcal H_s(\mu)+\ln 2.(13)✦

In the paper, the result reads: for every compactly supported real random variable XX and s>0s>0, with VsV_s independent of XX and uniform on [0,s][0,s] and hdiff,2h_{\mathrm{diff},2} denoting differential entropy in bits, ℋˆs(X)=hdiff,2(X+Vs)−log⁡2s\widehat{\mathcal H}_s(X)=h_{\mathrm{diff},2}(X+V_s)-\log_2 s and ℋˆs(aX+b)=ℋˆs/|a|(X)\widehat{\mathcal H}_s(aX+b)=\widehat{\mathcal H}_{s/|a|}(X) for a≠0a\neq 0; for s,t>0s,t>0 with t/s=Nt/s=N a positive integer, Gˆs,t\widehat G_{s,t} is concave in the law, 0≤Gˆs,t≤log⁡2N0\leq\widehat G_{s,t}\leq\log_2 N and Gˆs,t(X+Y)≥Gˆs,t(X)\widehat G_{s,t}(X+Y)\geq\widehat G_{s,t}(X) for independent bounded X,YX,Y; for z≥s>0z\geq s>0, ℋˆs(X)−log⁡2(z/s+2)≤ℋˆz(X)≤ℋˆs(X)+1\widehat{\mathcal H}_s(X)-\log_2(z/s+2)\leq\widehat{\mathcal H}_z(X) \leq\widehat{\mathcal H}_s(X)+1; and |ℋˆs(X)−Hˆs0(X)|≤1|\widehat{\mathcal H}_s(X)-\widehat H_s^0(X)|\leq 1 and |Hˆs0(X+b)−Hˆs0(X)|≤1|\widehat H_s^0(X+b)-\widehat H_s^0(X)|\leq 1.

What Is Checked

The claims are stated over the grid and averaged entropies of compactly carried probability measures, their affine images and finite mixtures, with the grid translation identity applying to arbitrary Borel measures and real scale parameters.