OctoInformBeta

OpenAI preprint, September 2026 · Section 3. Retaining more variance than the degree cost

Section 3.2 (setting): the Gaussian deficit calculation used in place of Lemma 3.2: An explicit finite grid bounds the kernel integral and identifies the deficit with moment gain

Continuous Gaussian tests give a positive deficit.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

A quadratic deficit compares the weight of a product construction with its weighted squared centered score. The Gaussian calculation uses continuous tent tests on the leaf coordinates and selector weights on a distinguished coordinate. The library fixes an explicit finite grid for this calculation.

Let γ\gamma be the standard Gaussian law on ℝ\mathbb R, and let GG have law γ\gamma. For a measure PP on ℝ\mathbb R and a real weight ww, the weighted mass BP(w)B_P(w) is its integral in (1). BP(w)=∫ℝw(x)dP(x).B_P(w)=\int_{\mathbb R}w(x)\,dP(x).(1)✦ The weighted mean mP(w)m_P(w) is the normalized first moment in (2). mP(w)=∫ℝw(x)xdP(x)BP(w).m_P(w)=\frac{\int_{\mathbb R}w(x)x\,dP(x)}{B_P(w)}.(2)✦ The weighted variance vP(w)v_P(w) is the normalized second moment minus the squared mean, as in (3). vP(w)=∫ℝw(x)x2dP(x)BP(w)−mP(w)2.v_P(w)=\frac{\int_{\mathbb R}w(x)x^2\,dP(x)}{B_P(w)}-m_P(w)^2.(3)✦

Weighted Tests and Selectors

The tent function τa,h:ℝ→ℝ\tau_{a,h}:\mathbb R\to\mathbb R with real center aa and width parameter hh is defined by (4). τa,h(x)=max⁡{0,1−|x−a|h}.\tau_{a,h}(x)=\max\left\{0,1-\frac{|x-a|}{h}\right\}.(4)✦ For a natural grid half-size MM, the grid index set is IM={0,…,2M}I_M=\{0,\ldots,2M\}, with middle index MM. Clipping to [0,1][0,1] is the real function CC in (5). C(u)=max⁡{0,min⁡{1,u}}.C(u)=\max\{0,\min\{1,u\}\}.(5)✦ For real spacing δ\delta and i∈IMi\in I_M, the leaf test center ciM,δc_i^{M,\delta} and the grid ramp difference HiM,δ:ℝ→ℝH_i^{M,\delta}:\mathbb R\to\mathbb R are given by (6). ciM,δ=1+δ(i−M),HiM,δ(t)=C(t/δ+M−i+1)−C(t/δ+M−i).\begin{aligned} c_i^{M,\delta}&=1+\delta(i-M),\\ H_i^{M,\delta}(t) &=C(t/\delta+M-i+1)-C(t/\delta+M-i). \end{aligned}(6) The selector partition weight WiM,δ:ℝ→ℝW_i^{M,\delta}:\mathbb R\to\mathbb R adds the middle-index correction in (7). WiM,δ(t)=HiM,δ(t)+𝟏{i=M}(1−∑j∈IMHjM,δ(t)).W_i^{M,\delta}(t)=H_i^{M,\delta}(t) +\mathbf 1_{\{i=M\}}\left(1-\sum_{j\in I_M}H_j^{M,\delta}(t)\right).(7)✦

The fixed grid half-size NN and leaf tent width hh are the constants in (8). N=(2128)8,h=1(2128)12.N=(2^{128})^8,\qquad h=\frac{1}{(2^{128})^{12}}.(8) The grid radius RR is fixed by (9). R=96.R=96.(9)✦ The grid spacing δ\delta is the radius divided by the half-size, as in (10). δ=RN.\delta=\frac{R}{N}.(10)✦ The leaf indices form INI_N.✦ For i∈INi\in I_N, the leaf tent test ϕi:ℝ→ℝ\phi_i:\mathbb R\to\mathbb R is the function in (11). ϕi(x)=τciN,δ,h(x).\phi_i(x)=\tau_{c_i^{N,\delta},h}(x).(11)✦ The selector test ψi:ℝ→ℝ\psi_i:\mathbb R\to\mathbb R is the fixed-grid partition weight in (12). ψi(t)=WiN,δ(t).\psi_i(t)=W_i^{N,\delta}(t).(12)✦

The Gaussian leaf mass βi\beta_i is the weighted mass in (13). βi=Bγ(ϕi).\beta_i=B_\gamma(\phi_i).(13)✦ The Gaussian leaf mean μi\mu_i is defined by (14). μi=mγ(ϕi).\mu_i=m_\gamma(\phi_i).(14)✦ The leaf variance σi\sigma_i, total leaf variance SS, and remaining leaf variance S−iS_{-i} are the quantities in (15). σi=vγ(ϕi),S=∑i∈INσi,S−i=∑j∈INj≠iσj.\sigma_i=v_\gamma(\phi_i),\qquad S=\sum_{i\in I_N}\sigma_i,\qquad S_{-i}=\sum_{\substack{j\in I_N\\j\ne i}}\sigma_j.(15) The Gaussian deficit kernel K:ℝ→ℝK:\mathbb R\to\mathbb R is the weighted sum in (16); its Gaussian integral is JJ in (17). K(t)=∑i∈INψi(t)(1+t−μi)2+S−iβi.K(t)=\sum_{i\in I_N}\psi_i(t) \frac{(1+t-\mu_i)^2+S_{-i}}{\beta_i}.(16)✦ J=∫ℝK(t)dγ(t).J=\int_{\mathbb R}K(t)\,d\gamma(t).(17)✦

Moment Gain and Finite Deficits

For an index set II, a measure PP on a measurable space Ω\Omega, a real score TT on Ω\Omega and real weight functions wiw_i on ℝ\mathbb R indexed by II, the weighted moments ℳk(P,T;w)i\mathcal M_k(P,T;w)_i of orders k=0,1,2k=0,1,2 are the integrals in (18). ℳk(P,T;w)i=∫Ωwi(T(x))T(x)kdP(x).\mathcal M_k(P,T;w)_i=\int_\Omega w_i(T(x))T(x)^k\,dP(x).(18)✦ The Gaussian leaf moment array ℓγ\ell^\gamma and Gaussian selector moment array rγr^\gamma use the identity score, as in (19). ℓk,iγ=ℳk(γ,id;ϕ)i,rk,iγ=ℳk(γ,id;ψ)i.\ell^\gamma_{k,i}=\mathcal M_k(\gamma,\mathrm{id};\phi)_i,\qquad r^\gamma_{k,i}=\mathcal M_k(\gamma,\mathrm{id};\psi)_i.(19) For real moment arrays ℓ,r\ell,r indexed by {0,1,2}×I\{0,1,2\}\times I, where II is finite, the weighted means mi(ℓ)m_i(\ell) and variances vi(ℓ)v_i(\ell) are defined by (20). mi(ℓ)=ℓ1,iℓ0,i,vi(ℓ)=ℓ2,iℓ0,i−mi(ℓ)2.m_i(\ell)=\frac{\ell_{1,i}}{\ell_{0,i}},\qquad v_i(\ell)=\frac{\ell_{2,i}}{\ell_{0,i}}-m_i(\ell)^2.(20) The moment gain 𝒢(ℓ,r)\mathcal G(\ell,r) is the real quantity in (21), where all sums and products range over II unless restricted. Ai(ℓ,r)=r2,i+2(1−mi(ℓ))r1,i+((1−mi(ℓ))2+∑j≠ivj(ℓ))r0,i,𝒢(ℓ,r)=(∏iℓ0,i)(1+∑ivi(ℓ)−∑iAi(ℓ,r)ℓ0,i).\begin{aligned} A_i(\ell,r)&=r_{2,i}+2(1-m_i(\ell))r_{1,i}\\ &\quad+\left((1-m_i(\ell))^2+\sum_{j\ne i}v_j(\ell)\right)r_{0,i},\\ \mathcal G(\ell,r)&=\left(\prod_i\ell_{0,i}\right) \left(1+\sum_i v_i(\ell)-\sum_i\frac{A_i(\ell,r)}{\ell_{0,i}}\right). \end{aligned}(21)✦

For a finite alphabet 𝒜\mathcal A and a real function ff on 𝒜\mathcal A, the uniform mean 𝔼𝒜f\mathbb E_{\mathcal A}f is the real number in (22), with value zero for the empty alphabet. 𝔼𝒜f=1|𝒜|∑a∈𝒜f(a).\mathbb E_{\mathcal A}f=\frac{1}{|\mathcal A|}\sum_{a\in\mathcal A}f(a).(22)✦ For a finite alphabet 𝒜\mathcal A, a finite leaf set II, a real score TT on 𝒜\mathcal A and real weights wiw_i on 𝒜\mathcal A indexed by II, the finite weighted moment array RT(w)R_T(w) is the array indexed by {0,1,2}×I\{0,1,2\}\times I given in (23). RT(w)k,i=𝔼a∈𝒜[wi(a)T(a)k],k=0,1,2.R_T(w)_{k,i}=\mathbb E_{a\in\mathcal A}[w_i(a)T(a)^k],\qquad k=0,1,2.(23)✦ The soft event weight as,wa_{s,w} associated with real selector weights sis_i and leaf weights wiw_i is the real function on 𝒜×𝒜I\mathcal A\times\mathcal A^I in (24), where x=(x0,(xj)j∈I)x=(x_0,(x_j)_{j\in I}). as,w(x)=∑i∈Isi(x0)∏j≠iwj(xj)−∏j∈Iwj(xj).a_{s,w}(x)=\sum_{i\in I}s_i(x_0)\prod_{j\ne i}w_j(x_j) -\prod_{j\in I}w_j(x_j).(24)✦ For a finite alphabet 𝒜\mathcal A, a finite leaf set II, a real score TT on 𝒜\mathcal A, a real weight function aa on 𝒜×𝒜I\mathcal A\times\mathcal A^I and a real centering value bb, the quadratic deficit 𝒟(T,b;a)\mathcal D(T,b;a) is the real number in (25), using the uniform mean on 𝒜×𝒜I\mathcal A\times\mathcal A^I. 𝒟(T,b;a)=𝔼x[a(x)(1−(T(x0)+∑j∈IT(xj)−b)2)].\mathcal D(T,b;a)=\mathbb E_x\left[ a(x)\left(1-\left(T(x_0)+\sum_{j\in I}T(x_j)-b\right)^2\right)\right].(25)✦ The leaf centering value bT(w)b_T(w) is the sum of the weighted leaf means minus one, as in (26). bT(w)=∑i∈Imi(RT(w))−1.b_T(w)=\sum_{i\in I}m_i(R_T(w))-1.(26)

The Claim

Theorem 1. For the fixed Gaussian grid and weights, the estimates (27), (29) and (30) hold, with (28) valid for every i∈INi\in I_N. (2N+1)h2<14.(2N+1)h^2<\frac14.(27)✦ σi≤h2.\sigma_i\le h^2.(28)✦ J<12.J<\frac12.(29)✦ 0<(∏i∈INβi)(1+S−J).0<\left(\prod_{i\in I_N}\beta_i\right)(1+S-J).(30)✦

In the paper, the result reads as (31) for leaf means μi\mu_i, conditional variances si2s_i^2, masses βi\beta_i and selector ι\iota, with parameters chosen so that J<1/4J<1/4 and S<1/4S<1/4: p*=∏iβi,S=∑isi2,b=∑iμi−1,dA=p*(1+S−J),J=𝔼[(1+G−μι(G))2+∑j≠ι(G)sj2βι(G)].\begin{gathered} p_*=\prod_i\beta_i,\qquad S=\sum_i s_i^2,\qquad b=\sum_i\mu_i-1,\\ d_A=p_*(1+S-J),\\ J=\mathbb E\left[ \frac{(1+G-\mu_{\iota(G)})^2+\sum_{j\ne\iota(G)}s_j^2} {\beta_{\iota(G)}}\right]. \end{gathered}(31)

What Is Checked

The Gaussian statement is over the standard Gaussian law, the fixed finite grid, and its tent leaf tests and selector tests. For every natural half-size MM and real spacing δ\delta, the selector weights satisfy (32) at every real tt. ∑i∈IMWiM,δ(t)=1.\sum_{i\in I_M}W_i^{M,\delta}(t)=1.(32)✦ The Gaussian moment arrays satisfy the gain identity (33). 𝒢(ℓγ,rγ)=(∏i∈INβi)(1+S−J).\mathcal G(\ell^\gamma,r^\gamma) =\left(\prod_{i\in I_N}\beta_i\right)(1+S-J).(33)✦

For a finite alphabet 𝒜\mathcal A and finite leaf set II, let TT have uniform mean zero and second moment one, and let si,wis_i,w_i be real weights with 𝔼𝒜wi≠0\mathbb E_{\mathcal A}w_i\ne0 for every i∈Ii\in I; the soft deficit satisfies (34). 𝒟(T,bT(w);as,w)=𝒢(RT(w),RT(s)).\mathcal D(T,b_T(w);a_{s,w}) =\mathcal G(R_T(w),R_T(s)).(34)✦