OctoInformBeta

OpenAI preprint, September 2026 · §3. Constructing a negative kernel correlation · §3

Lemma 3.2 [Coordinate expansion]: For zero-moment laws in large even dimension, t(H,Wq)t(H,W_q) tends to one plus an averaged coefficient as q→∞q\to\infty through odd primes

A large-field limit for typed matrix kernel moments.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

Throughout, HH is the incidence graph of 13 points and 22 faces: the bipartite graph on the points 0,…,120,\dots,12 and the faces 0,…,210,\dots,21 in which each point is joined to the faces containing it, the faces 0,…,210,\dots,21 being the triples {0,1,3}\{0,1,3\}, {0,1,9}\{0,1,9\}, {0,2,3}\{0,2,3\}, {0,2,9}\{0,2,9\}, {1,3,10}\{1,3,10\}, {1,7,10}\{1,7,10\}, {1,7,12}\{1,7,12\}, {1,9,12}\{1,9,12\}, {2,3,4}\{2,3,4\}, {2,4,9}\{2,4,9\}, {3,4,11}\{3,4,11\}, {3,10,11}\{3,10,11\}, {4,8,9}\{4,8,9\}, {4,8,11}\{4,8,11\}, {5,6,7}\{5,6,7\}, {5,6,11}\{5,6,11\}, {5,7,10}\{5,7,10\}, {5,10,11}\{5,10,11\}, {6,7,12}\{6,7,12\}, {6,8,11}\{6,8,11\}, {6,8,12}\{6,8,12\}, {8,9,12}\{8,9,12\}. Two points of a common face form a point pair, and there are 33 point pairs.

The kernels here are built from symmetric matrices over 𝔽q\mathbb F_q, the field with qq elements for an odd prime qq. A vertex carries a type and one symmetric matrix for each coordinate of a finite set KK, and an edge compares the matrices at its two ends at the coordinates that a random activation switches on, the activation being drawn with a law that depends on the two types. The question is the homomorphism density of HH in such a kernel as q→∞q\to\infty.

Laws and Activations

A finite probability law μ\mu on a finite set Ω\Omega is a function μ:Ω→[0,∞)\mu\colon\Omega\to[0,\infty) with ∑ω∈Ωμ(ω)=1\sum_{\omega\in\Omega}\mu(\omega)=1✦, and the expectation of a real function ff on Ω\Omega is 𝔼μf=∑ω∈Ωμ(ω)f(ω)\mathbb E_\mu f=\sum_{\omega\in\Omega}\mu(\omega)f(\omega).✦ The product of laws μ1,…,μm\mu_1,\dots,\mu_m on finite sets Ω1,…,Ωm\Omega_1,\dots,\Omega_m is the law on Ω1×…×Ωm\Omega_1\times\dots\times\Omega_m with weight ∏iμi(ωi)\prod_i\mu_i(\omega_i) at (ω1,…,ωm)(\omega_1,\dots,\omega_m), under which the coordinates are independent✦; the uniform law on Ω\Omega gives each point the weight 1/|Ω|1/|\Omega|.✦

An activation on KK is a pair z=(σ,η)z=(\sigma,\eta) of a set σ⊆K\sigma\subseteq K of active coordinates and a sign vector η∈{±1}K\eta\in\{\pm1\}^K.✦ The entry of an activation z=(σ,η)z=(\sigma,\eta) at a coordinate κ\kappa is zκ=ηκz_\kappa=\eta_\kappa if κ∈σ\kappa\in\sigma and zκ=0z_\kappa=0 otherwise✦, and its character φα\varphi_\alpha on a set α⊆K\alpha\subseteq K is the Boolean monomial of its signs on α\alpha when α\alpha is active: φα(z)={∏κ∈αηκif α⊆σ,0otherwise.\varphi_\alpha(z)= \begin{cases} \prod_{\kappa\in\alpha}\eta_\kappa & \text{if } \alpha\subseteq\sigma,\\ 0 & \text{otherwise.} \end{cases}(1)✦ An activation law on KK is a finite probability law pp on the activations on KK. An activation law pp on KK satisfies the zero-moment condition when 𝔼pφα=0\mathbb E_p\varphi_\alpha=0 for every nonempty α⊆K\alpha\subseteq K, and the activation moment ApA_p of pp is the function on pairs of subsets of KK given by Ap(α,β)=𝔼p[φαφβ].A_p(\alpha,\beta)=\mathbb E_p\bigl[\varphi_\alpha\varphi_\beta\bigr].(2)✦

Corners and Labellings

A corner k=(j,s)k=(j,s) is a face jj with a slot s∈{0,1,2}s\in\{0,1,2\}.✦ The points of each face are listed in increasing order v0<v1<v2v_0<v_1<v_2✦, the point v(k)v(k) of the corner is vsv_s✦, and the 66 corners are the edges of HH, the corner (j,s)(j,s) joining vsv_s to jj. The slots 00, 11, 22 of jj also hold its sides {v0,v1}\{v_0,v_1\}, {v0,v2}\{v_0,v_2\}, {v1,v2}\{v_1,v_2\}, which are point pairs.✦ The left pair λ(k)\lambda(k) and the right pair ρ(k)\rho(k) of the corner are the two sides of jj through v(k)v(k), the left one in the lower slot. The corners of a point pair ee form the set P(e)P(e) of corners kk with e∈{λ(k),ρ(k)}e\in\{\lambda(k),\rho(k)\}.✦ Each face carries a fixed three-bit label; the faces 0,…,210,\dots,21 carry in order 000000, 010010, 001001, 110110, 001001, 000000, 100100, 110110, 101101, 100100, 111111, 101101, 101101, 011011, 000000, 110110, 010010, 111111, 001001, 010010, 011011, 111111. The separation count σe\sigma_e of a point pair ee is the number of places, from 00 to 33, at which the three-bit labels of the two faces containing ee differ.

A labelling α\alpha assigns to each of the 33 point pairs ee a nonempty set αe⊆K\alpha_e\subseteq K.✦ For a family A=(Ak)A=(A_k) of real functions on pairs of subsets of KK, one for each corner, the active coefficient is CA=∑α∏kAk(αλ(k),αρ(k)),C_A=\sum_{\alpha}\prod_{k}A_k\bigl(\alpha_{\lambda(k)},\alpha_{\rho(k)}\bigr),(3) For a family z=(zk)z=(z_k) of activations on KK, one for each corner, the expanded model is ℰ(z)=∏κ∈K∏e(1+∏k∈P(e)(zk)κ),\mathcal E(z)=\prod_{\kappa\in K}\ \prod_{e}\Bigl(1+\prod_{k\in P(e)}(z_k)_\kappa\Bigr),(4)✦ the middle product running over the 33 point pairs.

Matrix Kernels

Fix a finite field 𝔽\mathbb F with quadratic character χ\chi and a natural number DD, and let S2DS_{2D} be the space of symmetric 2D×2D2D\times2D matrices over 𝔽\mathbb F.✦ The sign of a symmetric bilinear form BB on a finite-dimensional space VV over 𝔽\mathbb F is χ(det⁡G)\chi(\det G) for a Gram matrix GG of the nondegenerate form that BB induces on V/rad⁡BV/\operatorname{rad}B, and it is 11 when BB has rank zero✦; the sign of Z∈S2DZ\in S_{2D} is that of the form (u,v)↦uTZv(u,v)\mapsto u^{T}Zv.✦ For ξ=±1\xi=\pm1, the signed rank layer LξL_\xi is the set of matrices in S2DS_{2D} of rank DD and sign ξ\xi✦, and its mass is |Lξ|/|S2D||L_\xi|/|S_{2D}|.✦ For ξ=±1\xi=\pm1, the rank kernel RξR_\xi is the real function on S2DS_{2D} whose value at ZZ depends on whether ZZ lies in LξL_\xi.

The sample kernel wzw_z of an activation z=(σ,η)z=(\sigma,\eta) on KK and the activation kernel w‾p\bar w_p of an activation law pp on KK are the functions of two tuples 𝒙=(xκ)\mathbf x=(x_\kappa) and 𝒚=(yκ)\mathbf y=(y_\kappa) in S2DKS_{2D}^K given by wz(𝒙,𝒚)=∏κ∈σRηκ(xκ−yκ),w‾p(𝒙,𝒚)=𝔼z∼pwz(𝒙,𝒚).w_z(\mathbf x,\mathbf y)=\prod_{\kappa\in\sigma}R_{\eta_\kappa}(x_\kappa-y_\kappa), \qquad \bar w_p(\mathbf x,\mathbf y)=\mathbb E_{z\sim p}\,w_z(\mathbf x,\mathbf y).(5) The conditional moment of a family 𝒑=(pk)\mathbf p=(p_k) of activation laws on KK, one for each corner, is the average M(𝒑)=𝔼∏k=(j,s)w‾pk(𝒙v(k),𝒚j)M(\mathbf p)=\mathbb E\prod_{k=(j,s)}\bar w_{p_k}\bigl(\mathbf x_{v(k)},\mathbf y_j\bigr)(6)✦ over all choices of a tuple 𝒙i∈S2DK\mathbf x_i\in S_{2D}^K for each point ii and a tuple 𝒚j∈S2DK\mathbf y_j\in S_{2D}^K for each face jj, each choice with the same weight; over 𝔽q\mathbb F_q it is written Mq(𝒑)M_q(\mathbf p).

Typed Kernels

For finite probability laws on finite sets XX and YY, the bipartite moment of a real function UU on X×YX\times Y is the homomorphism density t(H,U)=𝔼∏k=(j,s)U(xv(k),yj),t(H,U)=\mathbb E\prod_{k=(j,s)}U\bigl(x_{v(k)},y_j\bigr),(7)✦ where x0,…,x12x_0,\dots,x_{12} are independent with the law on XX and y0,…,y21y_0,\dots,y_{21} are independent with the law on YY.

Let II and JJ be finite sets of types, with finite probability laws π\pi on II and ν\nu on JJ, and let p=(pa,b)p=(p_{a,b}) be a family of activation laws on KK indexed by I×JI\times J; its swapped family pTp^{T}, indexed by J×IJ\times I, has pb,aT=pa,bp^{T}_{b,a}=p_{a,b}. The typed matrix kernel of pp is the function on X×YX\times Y, with X=I×S2DKX=I\times S_{2D}^K and Y=J×S2DKY=J\times S_{2D}^K, given by Wp((a,𝒙),(b,𝒚))=w‾pa,b(𝒙,𝒚),W^{p}\bigl((a,\mathbf x),(b,\mathbf y)\bigr)=\bar w_{p_{a,b}}(\mathbf x,\mathbf y),(8)✦ and its typed laws on XX and YY are the products of π\pi and of ν\nu with the uniform law on S2DKS_{2D}^K.✦ Its bipartite moment t(H,Wp)t(H,W^{p}) is taken with the typed laws, and over 𝔽q\mathbb F_q the kernel WpW^{p} is written WqW_q.

A type assignment is a pair (a,b)(a,b) of a map aa from the points to II and a map bb from the faces to JJ. The type law is the product law on type assignments, with weight τ(a,b)=∏iπ(ai)∏jν(bj).\tau(a,b)=\prod_{i}\pi(a_i)\prod_{j}\nu(b_j).(9)✦ At (a,b)(a,b) the corner k=(j,s)k=(j,s) receives the law pka,b=pav(k),bjp^{a,b}_k=p_{a_{v(k)},b_j}, and these laws form the family 𝒑a,b\mathbf p^{a,b}. The typed coefficient C(a,b)C(a,b) is the active coefficient CAC_A at Ak=Apka,bA_k=A_{p^{a,b}_k}✦, and the averaged coefficient is C‾=∑a,bτ(a,b)C(a,b)\bar C=\sum_{a,b}\tau(a,b)\,C(a,b).✦

The singular tail threshold D0D_0 is a fixed integer, at least 16421642, such that the global tail dichotomy of the last section holds in every even dimension n≥D0n\ge D_0.✦

The Claim

Theorem 1. Let KK, II and JJ be finite sets, π\pi and ν\nu finite probability laws on II and JJ, and p=(pa,b)p=(p_{a,b}) a family of activation laws on KK indexed by I×JI\times J. Limits are taken as q→∞q\to\infty through the odd primes.

(i) If every pa,bp_{a,b} satisfies the zero-moment condition and DD is even with 2D≥D02D\ge D_0, then limq→∞t(H,Wq)=1+C‾.\lim_{q\to\infty}t(H,W_q)=1+\bar C.(10)✦

(ii) Over 𝔽\mathbb F and for every DD, t(H,Wp)=∑a,bτ(a,b)M(𝒑a,b).t(H,W^{p})=\sum_{a,b}\tau(a,b)\,M(\mathbf p^{a,b}).(11)✦

(iii) Over 𝔽\mathbb F, for even DD and for all (a,𝒙)∈X(a,\mathbf x)\in X and (b,𝒚)∈Y(b,\mathbf y)\in Y, Wp((a,𝒙),(b,𝒚))=WpT((b,𝒚),(a,𝒙)).W^{p}\bigl((a,\mathbf x),(b,\mathbf y)\bigr)=W^{p^{T}}\bigl((b,\mathbf y),(a,\mathbf x)\bigr).(12)✦

Let further 𝒑=(pk)\mathbf p=(p_k) be a family of activation laws on KK, one for each corner, each satisfying the zero-moment condition, and let Ak=ApkA_k=A_{p_k}.

(iv) If DD is even with 2D≥D02D\ge D_0, then limq→∞Mq(𝒑)=1+CA.\lim_{q\to\infty}M_q(\mathbf p)=1+C_A.(13)✦

(v) If the activations zkz_k are independent, each zkz_k with law pkp_k, then 𝔼ℰ(z)=1+CA.\mathbb E\,\mathcal E(z)=1+C_A.(14)✦

In the paper, the result reads: under the zero-moment condition, in an even dimension fixed as in Lemma 3.1 and for fixed type laws and activation laws, tI,J(Wq)=1+∑a,b(∏iπ(ai))(∏jν(bj))CA(a,b)+o(1)t_{I,J}(W_q)=1+\sum_{a,b}\bigl(\prod_i\pi(a_i)\bigr)\bigl(\prod_j\nu(b_j)\bigr)C_A(a,b)+o(1), where tI,J(Wq)t_{I,J}(W_q) stands for t(H,Wq)t(H,W_q) and CA(a,b)C_A(a,b) for C(a,b)C(a,b), and the transpose WqTW_q^{T} has the analogous formula with the roles of the type sets exchanged.

The library gives the transpose by (12): WqTW_q^{T} is the typed kernel of pTp^{T}, to which (10) applies with II and JJ exchanged.

What Is Checked

The claim is stated over the incidence graph HH with its corners, their points and their left and right pairs; activation laws on a finite set KK with their characters, activation moments and the zero-moment condition; the active coefficient over labellings of the point pairs and the expanded model; the signs, signed rank layers and rank kernels on S2DS_{2D} over 𝔽q\mathbb F_q, with the sample, activation and typed matrix kernels built from them; the conditional and bipartite moments with the typed laws and the type law; the typed and averaged coefficients; and the singular tail threshold D0D_0, limits being taken as q→∞q\to\infty through the odd primes.

About the threshold, the library proves the global tail dichotomy. A geometric profile (c,h)(c,h) in dimension nn is an assignment of natural numbers cj≤nc_j\le n to the faces jj and heh_e to the point pairs ee such that cj≤hec_j\le h_e for each side ee of jj and ∑e(he−cj)≤n−cj\sum_e(h_e-c_j)\le n-c_j, the sum running over the three sides of jj.✦ The profile cost of (c,h)(c,h) in dimension nn is 12∑jNj−16∑e(3−σe)b(he)\frac12\sum_j N_j-\frac16\sum_e(3-\sigma_e)\,b(h_e), where b(m)=m(m+1)/2b(m)=m(m+1)/2, the first sum runs over the 22 faces jj with Nj=(n+1)cj+∑e(he−cj+12)N_j=(n+1)c_j+\sum_{e}\binom{h_e-c_j+1}{2}, the inner sum running over the three sides ee of jj, and the second sum runs over the 33 point pairs ee.✦ A tail tuple a=(c,u,v,p,t)a=(c,u,v,p,t) consists of natural numbers cc, uu, vv and two triples of natural numbers p=(p0,p1,p2)p=(p_0,p_1,p_2) and t=(t0,t1,t2)t=(t_0,t_1,t_2)✦; its tail sizes are si=pi+tis_i=p_i+t_i✦, its tail heights are hi=c+sih_i=c+s_i✦, its maximal tail height HH is the largest of h0,h1,h2h_0,h_1,h_2✦, and its tail offset is δ=2u−c\delta=2u-c✦. The tail gain in dimension nn of a tail tuple a=(c,u,v,p,t)a=(c,u,v,p,t) is, at a face marked full, the planted gain (d+c2+c+1)c+δ2−2(c+δ2)2+∑ib(pi+ti)−δ∑ipi−∑ipi2−v(d−3δ2−∑ipi+v)−∑ib(ti)+b(v)\left(\frac{d+c}{2}+c+1\right)\frac{c+\delta}{2}-2\left(\frac{c+\delta}{2}\right)^2+\sum_i b(p_i+t_i)-\delta\sum_i p_i-\sum_i p_i^2-v\left(\frac{d-3\delta}{2}-\sum_i p_i+v\right)-\sum_i b(t_i)+b(v) with d=n−cd=n-c, the sums running over i∈{0,1,2}i\in\{0,1,2\}, and, at a face not marked full, the triangular number b(⌊H/2⌋)b(\lfloor H/2\rfloor).✦ Some faces are marked full. Put n=2rn=2r and si=he−cjs_i=h_e-c_j for the side ee of the face jj in slot ii.✦ At a face jj marked full, a tail choice is a pair (u,θ)(u,\theta) of an integer uu with 0≤u≤cj0\le u\le c_j, u≤ru\le r and r−u≤n−cjr-u\le n-c_j, and a quadruple θ=(θ0,θ1,θ2,θ3)\theta=(\theta_0,\theta_1,\theta_2,\theta_3) of integers between 00 and n−cjn-c_j such that, with r′=r−ur'=r-u, one has r′≤θ3r'\le\theta_3, θi+θi′≤r′\theta_i+\theta_{i'}\le r' for distinct i,i′∈{0,1,2}i,i'\in\{0,1,2\}, θ0+θ1+θ2+θ3≤3r′\theta_0+\theta_1+\theta_2+\theta_3\le3r', and for each i∈{0,1,2}i\in\{0,1,2\}, θi≤si\theta_i\le s_i, si+2r′≤n−cj+2θis_i+2r'\le n-c_j+2\theta_i and 2r′≤θ3+θi2r'\le\theta_3+\theta_i✦; it determines the tail tuple (cj,u,n−cj−θ3,(θ0,θ1,θ2),(s0−θ0,s1−θ1,s2−θ2))(c_j,u,n-c_j-\theta_3,(\theta_0,\theta_1,\theta_2),(s_0-\theta_0,s_1-\theta_1,s_2-\theta_2))✦. At a face not marked full the tail choice is trivial, and it determines the tail tuple (cj,0,0,(s0,s1,s2),(0,0,0))(c_j,0,0,(s_0,s_1,s_2),(0,0,0))✦. A global tail choice xx in dimension nn assigns a tail choice to each face✦, and its global tail gain is the sum, over the 22 faces jj, of the tail gain in dimension nn of the tail tuple that xx determines at jj✦. Let n≥D0n\ge D_0 be even, let (c,h)(c,h) be a geometric profile in dimension nn, and let xx be a global tail choice, which assigns to each face jj a tail choice for jj, a pair (u,θ)(u,\theta) as above if jj is marked full and the trivial choice otherwise. Then either the global tail gain of xx is strictly smaller than the profile cost of (c,h)(c,h), or every cjc_j is 00, every heh_e is 00 or 22 and the global tail gain is two thirds of the sum of the triangular numbers (he+12)\binom{h_e+1}{2} over the point pairs.✦