OctoInformBeta

OpenAI preprint, September 2026 · §3. Common mass, smoothing, and score moments

Lemma 3.2: A finite bound with explicit constants for powers separated by a positive repetition count

Positive trace overlap grows under sufficient repetition.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

Consecutive powers of a binary word can have trace laws with only a small amount of common probability mass. The amplification question concerns the common mass when the repetition counts are enlarged and separated by a prescribed positive number of copies. Write q∈[0,1]q\in[0,1] for the requested deletion probability, so each bit survives with probability 1−q1-q.

A binary word xx of length nn is an assignment of a Boolean bit xix_i to each zero-based position i∈{0,…,n−1}i\in\{0,\ldots,n-1\}.✦ For a finite binary word ww and a natural number mm, the repeated word wmw^m is the concatenation of mm copies of ww, as in (1). wm=ww⋯w⏟m copies,w0=⌀.w^m=\underbrace{w\,w\,\cdots\,w}_{m\text{ copies}}, \qquad w^0=\varnothing.(1)✦

Complete Trace Laws

A binary trace tt is a finite list of bits, with its surviving length |t||t| included in the observation.✦ For a natural number MM, the complete trace space 𝒯M\mathcal T_M is the finite set in (2); a bounded trace tt with bound MM is an element of the complete trace space 𝒯M\mathcal T_M. 𝒯M=⨆ℓ=0M{0,1}ℓ.\mathcal T_M=\bigsqcup_{\ell=0}^{M}\{0,1\}^{\ell}.(2)

A finite probability law PP on a finite set AA is a real mass function satisfying (3).✦ The point mass δa\delta_a at an outcome a∈Aa\in A is the probability law assigning mass one to aa and zero to every other outcome.✦ P(a)≥0(a∈A),∑a∈AP(a)=1.P(a)\ge0\quad(a\in A),\qquad \sum_{a\in A}P(a)=1.(3)

For a finite probability law PP on AA and a family K(a)K(a) of finite probability laws on a finite set BB, the mixture law PKPK is the law on BB with masses given by (4). (PK)(b)=∑a∈AP(a)K(a)(b)(b∈B).(PK)(b)=\sum_{a\in A}P(a)K(a)(b)\qquad(b\in B).(4)✦ The pushforward law f*Pf_*P of a finite probability law PP on a finite set AA under a map f:A→Bf:A\to B to a finite set BB is the probability law on BB obtained by mixing the point masses at f(a)f(a) with weights P(a)P(a), as in (5). f*P=P(a↦δf(a)).f_*P=P\bigl(a\mapsto\delta_{f(a)}\bigr).(5)✦

The set ℳn\mathcal M_n of deletion masks of length nn consists of Boolean assignments aa to positions 0,…,n−10,\ldots,n-1, with true retaining a bit and false deleting it.✦ For a deletion mask a∈ℳna\in\mathcal M_n of length nn at deletion probability qq, the deletion mask probability mq(a)m_q(a) is the real number given by the product in (6). mq(a)=∏i=0n−1{1−q,a(i)=true,q,a(i)=false.m_q(a)=\prod_{i=0}^{n-1} \begin{cases} 1-q,&a(i)=\mathrm{true},\\ q,&a(i)=\mathrm{false}. \end{cases}(6) For q∈[0,1]q\in[0,1], the deletion mask law μn,q\mu_{n,q} is the finite probability law on ℳn\mathcal M_n assigning mass mq(a)m_q(a) to each mask aa.✦

The retained subsequence τx(a)\tau_x(a) of a binary word xx of length nn under a deletion mask a∈ℳna\in\mathcal M_n is the trace of bits at retained positions in increasing position order, as in (7). τx(a)=(xi:0≤i<n,a(i)=true).\tau_x(a)=\bigl(x_i:\ 0\le i<n,\ a(i)=\mathrm{true}\bigr).(7) For a finite source word ww with |w|≤M|w|\le M, the complete trace law Tr⁡q(M)(w)\operatorname{Tr}^{(M)}_q(w) is the probability law on 𝒯M\mathcal T_M defined by the pushforward in (8). Tr⁡q(M)(w)=(τw)*μ|w|,q.\operatorname{Tr}^{(M)}_q(w)=(\tau_w)_*\mu_{|w|,q}.(8)✦

For finite probability laws P,QP,Q on the same finite set AA, the total variation distance TV⁡(P,Q)\operatorname{TV}(P,Q) is the number in (9), and their probability overlap ov⁡(P,Q)\operatorname{ov}(P,Q) is the common mass in (10). TV⁡(P,Q)=12∑a∈A|P(a)−Q(a)|.\operatorname{TV}(P,Q)=\frac12\sum_{a\in A}|P(a)-Q(a)|.(9) ov⁡(P,Q)=∑a∈Amin⁡{P(a),Q(a)}=1−TV⁡(P,Q).\operatorname{ov}(P,Q)=\sum_{a\in A}\min\{P(a),Q(a)\} =1-\operatorname{TV}(P,Q).(10)✦ The complete trace overlap 𝒪q(x,y)\mathcal O_q(x,y) of finite binary words x,yx,y is the probability overlap of their complete trace laws in the common space specified in (11). 𝒪q(x,y)=ov⁡(Tr⁡q(M)(x),Tr⁡q(M)(y)),M=max⁡{|x|,|y|}.\mathcal O_q(x,y)=\operatorname{ov}\bigl( \operatorname{Tr}^{(M)}_q(x),\operatorname{Tr}^{(M)}_q(y)\bigr), \qquad M=\max\{|x|,|y|\}.(11)✦

The Claim

Theorem 1 (Lemma 3.2). Let ww be a finite binary word, let k,Rk,R be positive integers, and let FF be a nonnegative integer. Let q∈[0,1]q\in[0,1], 0<ω≤1/80<\omega\le1/8 and C∈ℝC\in\mathbb R. Suppose that ωC≥1073741824\omega C\ge1073741824, F≥CkRF\ge CkR, and 𝒪q(wk,wk+1)≥ω\mathcal O_q(w^k,w^{k+1})\ge\omega. Then 𝒪q(wFR,wFR+R)≥23.\mathcal O_q\bigl(w^{FR},w^{FR+R}\bigr)\ge\frac23.(12)✦

In the paper, the result reads: for a nonempty ww, a positive integer kk and 0<ω<1/80<\omega<1/8 satisfying 𝒪q(wk,wk+1)≥ω\mathcal O_q(w^k,w^{k+1})\ge\omega, there is an absolute CabsC_{\mathrm{abs}} such that whenever C≥Cabs/ωC\ge C_{\mathrm{abs}}/\omega, RR is a sufficiently large positive integer with an absolute lower threshold uniform in the other parameters, and FF is an integer satisfying F≥CkRF\ge CkR, one has 𝒪q(wFR,wFR+R)≥2/3\mathcal O_q(w^{FR},w^{FR+R})\ge2/3.

What Is Checked

The bound (12) is stated over complete trace laws of finite repeated binary words under deletion at q∈[0,1]q\in[0,1], compared in the common bounded trace space of (11).

For real parameters qq outside [0,1][0,1], the mask probabilities also satisfy (13). ∑a∈ℳnmq(a)=1.\sum_{a\in\mathcal M_n}m_q(a)=1.(13)✦ For q∈[0,1]q\in[0,1], every deletion mask probability mq(a)m_q(a) is nonnegative.✦