OctoInformBeta

OpenAI preprint, September 2026 · §2. Entropy between two scales

§2 (setting): grid entropy, averaged entropy and deficits: Shift averages and dyadic deficits for compactly carried measures

Entropy from grids to scale windows.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

Entropy between two scales concerns how a real distribution is resolved as the observation scale changes. Let μ\mu be a measure on ℝ\mathbb R; positive numbers s,ts,t denote observation scales, and uu denotes a shift of the grid. The library uses natural logarithms in this entropy calculus, so division by ln⁡2\ln 2 gives entropy in bits.

The Definition

Definition 1. For a measure ν\nu on a measurable space AA, a map f:A→Bf\colon A\to B and a label b∈Bb\in B, the real fiber mass pb(ν,f)p_b(\nu,f) is defined by (1): pb(ν,f)={ν({x∈A:f(x)=b}),ν({x∈A:f(x)=b})<∞,0,ν({x∈A:f(x)=b})=∞.p_b(\nu,f)= \begin{cases} \nu(\{x\in A:f(x)=b\}),&\nu(\{x\in A:f(x)=b\})<\infty,\\ 0,&\nu(\{x\in A:f(x)=b\})=\infty. \end{cases}(1)✦ The Shannon entropy Hν(f)H_\nu(f) of the label map ff under ν\nu is the real sum in (2), with the convention 0ln⁡0=00\ln 0=0: Hν(f)=−∑b∈Bpb(ν,f)ln⁡pb(ν,f).H_\nu(f)=-\sum_{b\in B}p_b(\nu,f)\ln p_b(\nu,f).(2)✦

For real s,us,u, the grid cell index Ls,uL_{s,u} is the integer-valued map on ℝ\mathbb R defined by (3): Ls,u(x)=⌊xs+u⌋.L_{s,u}(x)=\left\lfloor\frac{x}{s}+u\right\rfloor.(3)✦ The shifted grid entropy Hsu(μ)H_s^u(\mu) of a measure μ\mu on ℝ\mathbb R, for real s,us,u, is the entropy of the grid cell index Ls,uL_{s,u}, as in (4): Hsu(μ)=Hμ(Ls,u).H_s^u(\mu)=H_\mu(L_{s,u}).(4)✦ Averaging the shifted grid entropy of a measure μ\mu on ℝ\mathbb R over a unit interval of shifts gives its averaged grid entropy ℋs(μ)\mathcal H_s(\mu) at a real scale ss, as in (5): ℋs(μ)=∫01Hsu(μ)du.\mathcal H_s(\mu)=\int_0^1 H_s^u(\mu)\,du.(5)✦

The entropy window Gs,t(μ)G_{s,t}(\mu) of a measure μ\mu on ℝ\mathbb R between real scales s,ts,t is the difference of its averaged grid entropies in (6): Gs,t(μ)=ℋs(μ)−ℋt(μ).G_{s,t}(\mu)=\mathcal H_s(\mu)-\mathcal H_t(\mu).(6)✦ The difference between ln⁡2\ln 2 and the entropy window from ss to 2s2s is the one-step deficit Δs(μ)\Delta_s(\mu) of a measure μ\mu on ℝ\mathbb R at a real scale ss, as in (7): Δs(μ)=ln⁡2−Gs,2s(μ).\Delta_s(\mu)=\ln 2-G_{s,2s}(\mu).(7)✦

A measure μ\mu on ℝ\mathbb R is compactly carried if it gives zero mass to the complement of some bounded interval, as specified in (8): μ is compactly carried⇔there exists R≥0 such that μ(ℝ\[−R,R])=0.\text{$\mu$ is compactly carried} \quad\Longleftrightarrow\quad \text{there exists $R\geq 0$ such that } \mu\bigl(\mathbb R\setminus[-R,R]\bigr)=0.(8)✦

In the paper, the result reads: for a bounded real variable XX and positive s,ts,t, writing HH for Shannon entropy in bits, Hs0(X)=H(⌊X/s⌋)H_s^0(X)=H(\lfloor X/s\rfloor), ℋs(X)=∫01H(⌊X/s+u⌋)du\mathcal H_s(X)=\int_0^1 H(\lfloor X/s+u\rfloor)\,du, Gs,t=ℋs−ℋtG_{s,t}=\mathcal H_s-\mathcal H_t and Δs=1−Gs,2s\Delta_s=1-G_{s,2s}, with differential entropy h(f)=−∫flog⁡2fh(f)=-\int f\log_2 f for a density ff, and VsV_s an independent uniform variable on [0,s][0,s].

What Is Checked

The entropy in (2) is defined for a measure and an arbitrary label map, while the grid constructions are defined for measures on ℝ\mathbb R and real scales and shifts, with compact carriage given by (8).