OctoInformBeta

OpenAI preprint, September 2026 · §5. Disjoint windows of entropy gain

Lemma 5.1: Finite future averages admit the bound before and after restriction to retained windows

Entropy drops control dyadic pair capacity.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

At a cut in a finite symbolic address, the remaining similarities act on the self-similar measure. The quantity of interest is the change in entropy between two fixed spatial scales when the cut moves across a block and the future types are averaged.

A weighted affine iterated function system (IFS) SS on a finite alphabet II consists of similarities with signed slopes, real translations and strictly positive probability weights, as in (1): φi(x)=rix+ti,0<|ri|<1,pi>0,∑i∈Ipi=1.\varphi_i(x)=r_i x+t_i,\qquad 0<|r_i|<1,\qquad p_i>0,\qquad \sum_{i\in I}p_i=1.(1) A probability measure μ\mu on ℝ\mathbb R is self-similar for SS when it satisfies (2): μ=∑i∈Ipi(φi)*μ.\mu=\sum_{i\in I}p_i(\varphi_i)_*\mu.(2)✦ A probability measure μ\mu on ℝ\mathbb R is compactly supported when μ(ℝ\[−R,R])=0\mu(\mathbb R\setminus[-R,R])=0 for some R≥0R\ge0.

For a finite word w=i0⋯im−1w=i_0\cdots i_{m-1}, its word similarity and complete affine map are specified in (3); composition follows address order: φw=φi0∘⋯∘φim−1,φw(x)=rwx+tw,(r⌀,t⌀)=(1,0).\varphi_w=\varphi_{i_0}\circ\cdots\circ\varphi_{i_{m-1}},\qquad \varphi_w(x)=r_wx+t_w,\qquad (r_\varnothing,t_\varnothing)=(1,0).(3) A collision of complete affine maps is equality of their signed slopes and translations. The word probability pwp_w, symbol depth d(i)d(i) and word contraction depth d(w)d(w) are given by (4); σj(W)\sigma_j(W) denotes the accumulated depth of the first at most jj letters of a finite word WW: pw=∏k<mpik,d(i)=−ln⁡|ri|ln⁡2,d(w)=−ln⁡|rw|ln⁡2,σj(W)=∑k<min⁡(j,|W|)d(Wk).\begin{aligned} p_w&=\prod_{k<m}p_{i_k},& d(i)&=-\frac{\ln|r_i|}{\ln2},& d(w)&=-\frac{\ln|r_w|}{\ln2},\\ \sigma_j(W)&=\sum_{k<\min(j,|W|)}d(W_k). \end{aligned}(4)

Conditional Tails

A finite probability law ν\nu on ℝ\mathbb R consists of a finite support FF and masses ν(x)>0\nu(x)>0 for x∈Fx\in F, with ∑x∈Fν(x)=1\sum_{x\in F}\nu(x)=1.✦ For a finite source EE, real weights qq and a map f:E→Yf:E\to Y, the fiber masses are given by (5): mE,q,f(y)=∑v∈E:f(v)=yqv.m_{E,q,f}(y)=\sum_{v\in E:\ f(v)=y}q_v.(5)✦ For a real-valued map ff on a finite source EE with strictly positive normalized weights qq, the observable distribution of ff is the finite probability law with fiber masses given by (5).

For a tag map τ:E→T\tau:E\to T on a finite source EE, strictly positive weights qq on EE and a source point a∈Ea\in E, the tag fiber Fτ(a)F_\tau(a), its mass Mτ(a)M_\tau(a) and the conditional symbol probability πτ(v∣a)\pi_\tau(v\mid a) for v∈Ev\in E are defined in (6): Fτ(a)={v∈E:τ(v)=τ(a)},Mτ(a)=∑v∈Fτ(a)qv,πτ(v∣a)={qv/Mτ(a),τ(v)=τ(a),0,τ(v)≠τ(a).\begin{aligned} F_\tau(a)&=\{v\in E:\tau(v)=\tau(a)\},& M_\tau(a)&=\sum_{v\in F_\tau(a)}q_v,\\ \pi_\tau(v\mid a)&= \begin{cases} q_v/M_\tau(a),&\tau(v)=\tau(a),\\ 0,&\tau(v)\ne\tau(a). \end{cases} \end{aligned}(6) For a real-valued observable f:E→ℝf:E\to\mathbb R on a finite source EE with strictly positive weights qq, a tag map τ:E→T\tau:E\to T and a source point a∈Ea\in E, the conditional observable distribution of ff given τ(a)\tau(a) is the finitely supported probability law on ℝ\mathbb R obtained from the values of ff on Fτ(a)F_\tau(a) with weights qv/Mτ(a)q_v/M_\tau(a).

Choose a type map τ:I→T\tau:I\to T for the IFS and use qi=piq_i=p_i. The word tag sequence τ(w)\tau(w) of a finite word ww over II is the sequence specified by τ(w)=(τ(wk))k<|w|\tau(w)=(\tau(w_k))_{k<|w|}.✦ For a finite word ww over II, the type-conditioned translation law νw\nu_w is the conditional observable distribution of v↦tvv\mapsto t_v on words vv of the same length as ww, given τ(v)=τ(w)\tau(v)=\tau(w) and using the word weights pvp_v. For a word ww over II, the conditional word probability πw(v)\pi_w(v) of a word vv of the same length is the product in (7), the conditional tail QwQ_w is the probability measure given by its weighted sum, and Tb(w)T_b(w) is the probability measure obtained by dilating QwQ_w by the signed real factor bb: πw(v)=∏k<|w|πτ(vk∣wk),Qw=∑v∈I|w|πw(v)(φv)*μ,Tb(w)=(Db)*Qw,Db(x)=bx.\begin{aligned} \pi_w(v)&=\prod_{k<|w|}\pi_\tau(v_k\mid w_k),\\ Q_w&=\sum_{v\in I^{|w|}}\pi_w(v)(\varphi_v)_*\mu,\\ T_b(w)&=(D_b)_*Q_w,\qquad D_b(x)=bx. \end{aligned}(7) Only the types in ww are conditioned; the continuation with law μ\mu remains unconditioned.

Entropy Windows and Pair Capacity

Let mρ,f(y)m_{\rho,f}(y) be the real mass of the fiber f−1{y}f^{-1}\{y\} for a measure ρ\rho on a measurable space XX and a map f:X→Yf:X\to Y.✦ The partition entropy Hρ(f)H_\rho(f) of a map f:X→Yf:X\to Y with respect to a measure ρ\rho on XX is the quantity in (8), with 0ln⁡0=00\ln0=0. For measures on ℝ\mathbb R, with spatial scale ss and grid shift uu, λs,u\lambda_{s,u} is the shifted grid cell index, Hs,uH_{s,u} its grid entropy and H¯s\overline H_s its averaged grid entropy: Hρ(f)=−∑ymρ,f(y)ln⁡mρ,f(y),λs,u(x)=⌊x/s+u⌋,Hs,u(ρ)=Hρ(λs,u),H¯s(ρ)=∫01Hs,u(ρ)du.\begin{aligned} H_\rho(f)&=-\sum_y m_{\rho,f}(y)\ln m_{\rho,f}(y),\\ \lambda_{s,u}(x)&=\left\lfloor x/s+u\right\rfloor,& H_{s,u}(\rho)&=H_\rho(\lambda_{s,u}),\\ \overline H_s(\rho)&=\int_0^1 H_{s,u}(\rho)\,du. \end{aligned}(8) For a measure ρ\rho on ℝ\mathbb R and spatial scales ss and tt, the entropy window Gs,t(ρ)G_{s,t}(\rho) and one-step dyadic deficit Δs(ρ)\Delta_s(\rho) are the quantities defined by (9): Gs,t(ρ)=H¯s(ρ)−H¯t(ρ),Δs(ρ)=ln⁡2−Gs,2s(ρ).G_{s,t}(\rho)=\overline H_s(\rho)-\overline H_t(\rho),\qquad \Delta_s(\rho)=\ln2-G_{s,2s}(\rho).(9) These entropies use natural logarithms; division by ln⁡2\ln2 gives their values in bits.

The set of fair pairs Eℓ(ν)E_\ell(\nu) and the incident pair mass Ib(x)I_b(x) at an atom xx are given in (10) for a finite law ν\nu, an integer ℓ\ell and weights bb assigned to its fair pairs. A feasible pair weighting bb for a finite law ν\nu in band ℓ\ell is a family of weights on Eℓ(ν)E_\ell(\nu) such that be≥0b_e\ge0 for every pair and Ib(x)≤ν(x)I_b(x)\le\nu(x) for every atom xx.✦ For a finite law ν\nu on ℝ\mathbb R and an integer ℓ\ell, the fair-pair capacity cℓ(ν)c_\ell(\nu) is the supremum of total pair weight over feasible pair weightings in band ℓ\ell, as in (10): Eℓ(ν)={{x,y}⊂F:2−ℓ≤|x−y|<21−ℓ},Ib(x)=12∑e∈Eℓ(ν):x∈ebe,cℓ(ν)=sup⁡{∑e∈Eℓ(ν)be:b is feasible}.\begin{aligned} E_\ell(\nu)&=\{\{x,y\}\subset F: 2^{-\ell}\le |x-y|<2^{1-\ell}\},\\ I_b(x)&=\frac12\sum_{e\in E_\ell(\nu):\ x\in e}b_e,\\ c_\ell(\nu)&=\sup\left\{\sum_{e\in E_\ell(\nu)}b_e: b\text{ is feasible}\right\}. \end{aligned}(10)

Retained Windows

Write W<jW_{<j} for the first at most jj letters of WW, and W≥jW_{\ge j} for its remaining suffix. For a finite word WW, a cut jj, a scale ss and a natural factor NN, the scaled suffix law ηW,j\eta_{W,j} is the probability measure specified in (11), and its suffix entropy window gjs,N(W)g_j^{s,N}(W) is the quantity specified there: ηW,j=TrW<j(W≥j),gjs,N(W)=Gs,Ns(ηW,j).\eta_{W,j}=T_{r_{W_{<j}}}(W_{\ge j}),\qquad g_j^{s,N}(W)=G_{s,Ns}(\eta_{W,j}).(11)

Fix n∈ℕn\in\mathbb N, A′∈ℝA'\in\mathbb R and integers a,B,La,B,L, and put ℓj(W)=L−⌈σj(W)⌉\ell_j(W)=L-\lceil\sigma_j(W)\rceil. The retained cuts form the set ℛW\mathcal R_W in (12): ℛW={j∈ℕ:σj+n(W)−σj(W)≤A′n,a≤ℓj(W)≤B}.\mathcal R_W=\left\{j\in\mathbb N: \sigma_{j+n}(W)-\sigma_j(W)\le A'n,\quad a\le\ell_j(W)\le B\right\}.(12)✦ For a positive integer MM, put sL=2−L/Ms_L=2^{-L}/M. Let W[j,j+n)W_{[j,j+n)} be the first at most nn letters of W≥jW_{\ge j}. The retained capacity Cj(W)C_j(W) and retained window drop Dj(W)D_j(W) are defined by (13): Cj(W)=𝟏ℛW(j)cℓj(W)(νW[j,j+n)),Dj(W)=𝟏ℛW(j)(gjsL,M2(W)−gj+nsL,M2(W)).\begin{aligned} C_j(W)&=\mathbf1_{\mathcal R_W}(j) c_{\ell_j(W)}(\nu_{W_{[j,j+n)}}),\\ D_j(W)&=\mathbf1_{\mathcal R_W}(j) \bigl(g_j^{s_L,M^2}(W)-g_{j+n}^{s_L,M^2}(W)\bigr). \end{aligned}(13) The Bernoulli word average 𝔼HF\mathbb E_H F of a real function FF on words of length HH is the weighted sum in (14): 𝔼HF=∑W∈IHpWF(W).\mathbb E_H F=\sum_{W\in I^H}p_WF(W).(14)

The Claim

Theorem 1. Let SS be a weighted affine IFS, τ:I→T\tau:I\to T a type map and μ\mu a compactly supported self-similar probability measure for SS. Suppose symbols with the same type have the same signed slope. Let δ∈ℝ\delta\in\mathbb R satisfy δ≤Δv(μ)\delta\le\Delta_v(\mu) for every v>0v>0, and let M∈ℕM\in\mathbb N satisfy M>4M>4 and 5ln⁡2/M≤δ/25\ln2/M\le\delta/2.

For j,n,t∈ℕj,n,t\in\mathbb N, u∈Iju\in I^j, w∈Inw\in I^n and L∈ℤL\in\mathbb Z, the conditional gain satisfies (15), with sL=2−L/Ms_L=2^{-L}/M and with wzwz denoting the concatenation of ww and a word zz of length tt: δ2cL−⌈d(u)⌉(νw)≤∑z∈Itpz[GsL,M2sL(Tru(wz))−GsL,M2sL(Trurw(z))].\frac\delta2\,c_{L-\lceil d(u)\rceil}(\nu_w) \le\sum_{z\in I^t}p_z \left[ G_{s_L,M^2s_L}\bigl(T_{r_u}(wz)\bigr) -G_{s_L,M^2s_L}\bigl(T_{r_ur_w}(z)\bigr) \right].(15)✦ For H,j,n∈ℕH,j,n\in\mathbb N with j+n≤Hj+n\le H, A′∈ℝA'\in\mathbb R and a,B,L∈ℤa,B,L\in\mathbb Z, the expected retained gain satisfies (16): δ2𝔼HCj≤𝔼HDj.\frac\delta2\,\mathbb E_H C_j\le\mathbb E_H D_j.(16)✦

In the paper, the result reads: with the preceding suffix laws conditioned on an entire infinite type environment ω\omega, with θ\theta denoting its shift, ℱj+n\mathcal F_{j+n} the sigma algebra of conditioning and νθjω,n\nu_{\theta^j\omega,n} the corresponding type-conditioned translation law for a block of length nn, positive δ\delta, MM and suffix windows gjg_j, for every integer j≥0j\ge0, n≥1n\ge1 and L≥1L\ge1, 𝔼[gj−gj+n∣ℱj+n]≥(δ/2)cL−⌈σj⌉(νθjω,n)\mathbb E[g_j-g_{j+n}\mid\mathcal F_{j+n}] \ge(\delta/2)c_{L-\lceil\sigma_j\rceil}(\nu_{\theta^j\omega,n}).

What Is Checked

The statements concern finite type partitions of weighted affine systems with constant signed slope on each type, compactly supported self-similar probability measures and finite Bernoulli word averages. Every word has positive Bernoulli weight✦, and the normalized weights on the tag fiber of every source point sum to one.✦