OctoInformBeta

OpenAI preprint, September 2026 · §5. Disjoint windows of entropy gain

Lemma 5.2: Disjoint intervals bound losses of decreasing profiles and their finite Bernoulli averages

Retained windows share a finite entropy budget.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

A block of symbols occupies an interval of an address, while its contraction determines which spatial scales it reaches. Retention selects blocks by their contraction depths and the bands containing their starting scales. The entropy losses concern conditional suffix laws at the two ends of each selected block.

A weighted affine iterated function system on a finite alphabet α\alpha is a family of maps and strictly positive probabilities satisfying (1); the slopes are signed and the translations are arbitrary real numbers. φa(x)=rax+ta,0<|ra|<1,pa>0,∑a∈αpa=1.\varphi_a(x)=r_ax+t_a,\qquad 0<|r_a|<1,\qquad p_a>0,\qquad \sum_{a\in\alpha}p_a=1.(1)✦ The complete affine map of a finite word ww is its signed slope and translation pair (rw,tw)(r_w,t_w), and its word similarity fwf_w is given by (2). The recursion composes letters in address order; pwp_w is the word probability. (r⌀,t⌀)=(1,0),(rav,tav)=(rarv,ratv+ta),fw(x)=rwx+tw,pw=∏j=1|w|pwj.\begin{gathered} (r_{\varnothing},t_{\varnothing})=(1,0),\qquad (r_{av},t_{av})=(r_ar_v,r_at_v+t_a),\\ f_w(x)=r_wx+t_w,\qquad p_w=\prod_{j=1}^{|w|}p_{w_j}. \end{gathered}(2) For a symbol a∈αa\in\alpha, a finite word ww, a real function FF on words and a nonnegative integer HH, the symbol contraction depth d(a)d(a), word contraction depth d(w)d(w) and Bernoulli word average 𝔼HF\mathbb E_H F are the real numbers given in (3). d(a)=−log⁡2|ra|,d(w)=∑j=1|w|d(wj),𝔼HF=∑W∈αHpWF(W).d(a)=-\log_2|r_a|,\qquad d(w)=\sum_{j=1}^{|w|}d(w_j),\qquad \mathbb E_H F=\sum_{W\in\alpha^H}p_W F(W).(3)

Finite Conditional Suffix Laws

For a set β\beta of tags, a type map τ:α→β\tau:\alpha\to\beta assigns a tag to each symbol. For a type map τ:α→β\tau:\alpha\to\beta and a symbol a∈αa\in\alpha, the tag fiber Fτ(a)F_\tau(a) is the set of symbols in α\alpha with the tag of aa; the fiber mass mτ(a)m_\tau(a) and conditional symbol probability qa(v)q_a(v) are the probabilities given in (4). Fτ(a)={v∈α:τ(v)=τ(a)},mτ(a)=∑v∈Fτ(a)pv,qa(v)={pv/mτ(a),τ(v)=τ(a),0,τ(v)≠τ(a).\begin{gathered} F_\tau(a)=\{v\in\alpha:\tau(v)=\tau(a)\},\qquad m_\tau(a)=\sum_{v\in F_\tau(a)}p_v,\\ q_a(v)= \begin{cases} p_v/m_\tau(a),&\tau(v)=\tau(a),\\ 0,&\tau(v)\ne\tau(a). \end{cases} \end{gathered}(4) Fix a compactly supported probability measure μ\mu on ℝ\mathbb R, meaning that some closed interval [−R,R][-R,R], with R≥0R\ge0, has complement of μ\mu-measure zero.✦ The conditional word probability Pτ(v∣w)P_\tau(v\mid w) and conditional tail law TwμT_w\mu for words of length mm are the product and affine mixture in (5). Pτ(v∣w)=∏j=1mqwj(vj),Twμ=∑v∈αmPτ(v∣w)(fv)*μ.P_\tau(v\mid w)=\prod_{j=1}^{m}q_{w_j}(v_j),\qquad T_w\mu=\sum_{v\in\alpha^m}P_\tau(v\mid w)\,(f_v)_*\mu.(5) This conditions only the finitely many types of ww; the continuation with law μ\mu remains unconditioned. For a finite word WW, write W≤jW_{\le j} for its first min⁡(j,|W|)\min(j,|W|) letters and W>jW_{>j} for the remaining suffix. For a real dilation factor bb, a finite word ww, a finite word WW and a prefix length jj, the scaled tail law Tw(b)μT_w^{(b)}\mu and scaled conditional suffix law νW,j\nu_{W,j} are the probability measures given in (6), where Db(x)=bxD_b(x)=bx. Tw(b)μ=(Db)*Twμ,νW,j=TW>j(rW≤j)μ.T_w^{(b)}\mu=(D_b)_*T_w\mu,\qquad \nu_{W,j}=T_{W_{>j}}^{(r_{W_{\le j}})}\mu.(6)

For a measure ν\nu and a partition map ff into a label set, the real fiber mass mν(f,b)m_\nu(f,b) is the real value of the measure of the fiber labelled bb; the partition entropy 𝖧(ν,f)\mathsf H(\nu,f) is the real number given by (7), with 0ln⁡0=00\ln0=0. mν(f,b)=ν{x:f(x)=b},𝖧(ν,f)=−∑bmν(f,b)ln⁡mν(f,b).m_\nu(f,b)=\nu\{x:f(x)=b\},\qquad \mathsf H(\nu,f)=-\sum_b m_\nu(f,b)\ln m_\nu(f,b).(7) For a measure ν\nu on ℝ\mathbb R, a real scale ss and a real offset uu, the shifted grid cell index ℓs,u\ell_{s,u} is an integer-valued function on ℝ\mathbb R, and the shifted grid entropy Hs,u(ν)H_{s,u}(\nu) and averaged grid entropy H¯s(ν)\overline H_s(\nu) are the real numbers given in (8). ℓs,u(x)=⌊x/s+u⌋,Hs,u(ν)=𝖧(ν,ℓs,u),H¯s(ν)=∫01Hs,u(ν)du.\ell_{s,u}(x)=\left\lfloor x/s+u\right\rfloor,\qquad H_{s,u}(\nu)=\mathsf H(\nu,\ell_{s,u}),\qquad \overline H_s(\nu)=\int_0^1 H_{s,u}(\nu)\,du.(8) For a measure ν\nu on ℝ\mathbb R and real scales s,ts,t, the entropy window Gs,t(ν)G_{s,t}(\nu) is the difference of averaged grid entropies in (9). For a finite word WW, a cut j∈ℕj\in\mathbb N, a real scale ss and a nonnegative integer NN, the suffix entropy window is the real number obtained by evaluating the entropy window of the scaled conditional suffix law νW,j\nu_{W,j} from ss to NsNs. For a positive integer MM and integer LL, gW(j)g_W(j) is this suffix entropy window with the fine scale and factor specified in (9). These entropies use natural logarithms; division by ln⁡2\ln2 gives base-two units. Gs,t(ν)=H¯s(ν)−H¯t(ν),s=2−L/M,gW(j)=Gs,M2s(νW,j).G_{s,t}(\nu)=\overline H_s(\nu)-\overline H_t(\nu),\qquad s=2^{-L}/M,\qquad g_W(j)=G_{s,M^2s}(\nu_{W,j}).(9)

Retained Indices

Let ℕ\mathbb N include zero. A depth profile σ:ℕ→ℝ\sigma:\mathbb N\to\mathbb R assigns a cumulative depth to every index. Fix finitely many block lengths ni∈ℕn_i\in\mathbb N, counts Ki∈ℕK_i\in\mathbb N and integer band endpoints ai,Bia_i,B_i, indexed by i=1,…,qi=1,\ldots,q, and a real depth bound A′A'. Their separation condition is (10). Bh<ai−A′ni−1(1≤h<i≤q).B_h<a_i-A'n_i-1\qquad(1\le h<i\le q).(10) A retained index jj for block ii is an index satisfying the condition 𝒜i,j,L(σ)\mathcal A_{i,j,L}(\sigma) in (11); JiJ_i is the set of aligned candidate starts. 𝒜i,j,L(σ):σ(j+ni)−σ(j)≤A′ni,ai≤L−⌈σ(j)⌉≤Bi,Ji={kni:0≤k<Ki}.\begin{gathered} \mathcal A_{i,j,L}(\sigma):\quad \sigma(j+n_i)-\sigma(j)\le A'n_i,\qquad a_i\le L-\lceil\sigma(j)\rceil\le B_i,\\ J_i=\{kn_i:0\le k<K_i\}. \end{gathered}(11)✦ For a finite word WW, the prefix depth profile σW\sigma_W is a real-valued function of the prefix length, and the retained window entropy loss Ri(W,j)R_i(W,j) is a real number, given by (12), where 𝟏\mathbf1 denotes the indicator of the retention condition. σW(j)=d(W≤j),Ri(W,j)=𝟏𝒜i,j,L(σW)(gW(j)−gW(j+ni)).\sigma_W(j)=d(W_{\le j}),\qquad R_i(W,j)=\mathbf1_{\mathcal A_{i,j,L}(\sigma_W)} \bigl(g_W(j)-g_W(j+n_i)\bigr).(12)✦

The Claim

Theorem 1. Let σ\sigma be nondecreasing, and let j,k,n∈ℕj,k,n\in\mathbb N, L,a,B∈ℤL,a,B\in\mathbb Z and d∈ℝd\in\mathbb R satisfy σ(j+n)−σ(j)≤d\sigma(j+n)-\sigma(j)\le d, a≤L−⌈σ(j)⌉a\le L-\lceil\sigma(j)\rceil, L−⌈σ(k)⌉≤BL-\lceil\sigma(k)\rceil\le B and B<a−d−1B<a-d-1. Then (13) holds. j+n<k.j+n<k.(13)✦ For n,j,k∈ℕn,j,k\in\mathbb N with j≠kj\ne k, the aligned half-open integer intervals satisfy (14). [jn,jn+n)∩[kn,kn+n)=⌀.[jn,jn+n)\cap[kn,kn+n)=\varnothing.(14)✦ Let EE be a finite index set with integer endpoints 0≤ue≤ve0\le u_e\le v_e, and suppose that the intervals [ue,ve)[u_e,v_e) are pairwise disjoint. Every nonincreasing g:ℕ→[0,∞)g:\mathbb N\to[0,\infty) satisfies (15). ∑e∈E(g(ue)−g(ve))≤g(0).\sum_{e\in E}\bigl(g(u_e)-g(v_e)\bigr)\le g(0).(15)✦ For the separated bands in (10), a nondecreasing profile σ\sigma and a nonincreasing nonnegative function gg, the retained sum satisfies (16). ∑i=1q∑k=0Ki−1𝟏𝒜i,kni,L(σ)(g(kni)−g(kni+ni))≤g(0).\sum_{i=1}^{q}\sum_{k=0}^{K_i-1} \mathbf1_{\mathcal A_{i,kn_i,L}(\sigma)} \bigl(g(kn_i)-g(kn_i+n_i)\bigr)\le g(0).(16)✦ Suppose also that symbols with equal tags have equal signed slopes. For every word length HH, the compactly supported probability measure μ\mu, positive integer MM and integer scale index LL, the retained losses satisfy (17). ∑i=1q∑k=0Ki−1𝔼H[W↦Ri(W,kni)]≤ln⁡(M2).\sum_{i=1}^{q}\sum_{k=0}^{K_i-1} \mathbb E_H\bigl[W\mapsto R_i(W,kn_i)\bigr]\le\ln(M^2).(17)✦

In the paper, the result reads: under its separated-band and candidate-start choices, for every fixed infinite type environment and L∈{1,…,T}L\in\{1,\ldots,T\}, where TT is the upper scale-index bound, all retained half-open windows are pairwise disjoint and their losses satisfy (18). ∑i∑j∈Ji𝟏𝒜i,j,L(g(j)−g(j+ni))≤2log⁡2M.\sum_i\sum_{j\in J_i}\mathbf1_{\mathcal A_{i,j,L}} \bigl(g(j)-g(j+n_i)\bigr)\le2\log_2 M.(18)

What Is Checked

The statements concern finite families of half-open integer intervals, nondecreasing depth profiles and nonincreasing nonnegative functions, and finite conditional affine mixtures of a compactly supported probability measure, with equal tags requiring equal signed slopes in the Bernoulli average.