OctoInformBeta

OpenAI preprint, September 2026 · §5. Output-dependent truncations and energy jumps

Lemma 5.2: Unconditional Fréchet and segment derivatives of Φ(X*X)\Phi(X^* X) on finite square matrices, and an entrywise gradient bound on rectangular matrices

Differentiability and jump bounds for Gram matrix energy.

Hover over a blue or green term in the text, or a node in the graph, to find it in the other. Select it to read its declaration both in Lean and in words.

The Setting

Take finite index sets ι\iota and κ\kappa, and consider complex square matrices indexed by ι\iota together with complex ι×κ\iota \times \kappa rectangular matrices. For a complex square matrix XX indexed by ι\iota, the quantity tr⁡((X*X)3/2)\operatorname{tr}\bigl((X^* X)^{3/2}\bigr) can vary as XX passes through rank changes. The question is whether this function of XX is Fréchet differentiable everywhere and how its gradient and differences are controlled entrywise.

Definitions

The real trace Re⁡tr⁡(A)\operatorname{Re}\operatorname{tr}(A) of a complex square matrix AA indexed by a finite set ι\iota is the real number given by the real part of the trace of AA.✦ The real Hilbert-Schmidt inner product Re⁡tr⁡(X*Y)\operatorname{Re}\operatorname{tr}(X^* Y) of two complex square matrices XX and YY indexed by a finite set ι\iota is the real number given by the real trace of X*YX^* Y.✦

The matrix energy Φ(P)\Phi(P) of a complex square matrix PP indexed by a finite set ι\iota is the real number Φ(P)=Re⁡tr⁡(PP),\Phi(P) = \operatorname{Re}\operatorname{tr}(P \sqrt{P}),(1)✦ where P\sqrt{P} is the square root of PP; on positive semidefinite matrices PP, this agrees with tr⁡(P3/2)\operatorname{tr}(P^{3/2}). The Gram energy gradient 𝒢R,P\mathcal{G}_{R,P} of complex square matrices RR and PP indexed by a finite set ι\iota is the continuous real-linear functional acting on each complex square matrix HH indexed by ι\iota by 𝒢R,P(H)=3Re⁡tr⁡((RP)*H).\mathcal{G}_{R,P}(H) = 3 \operatorname{Re}\operatorname{tr}\bigl((R \sqrt{P})^* H\bigr).(2)✦

The segment matrix RA,B(t)R_{A,B}(t) associated with complex square matrices AA and BB indexed by a finite set ι\iota and a real scalar t∈ℝt \in \mathbb{R} is the complex square matrix indexed by ι\iota given by RA,B(t)=(1−t)A+tB.R_{A,B}(t) = (1 - t) A + t B.(3)✦

The row norm ∥rowiR∥2\|\mathrm{row}_i R\|_2 at row index i∈ιi \in \iota of a complex rectangular matrix RR with rows indexed by a finite set ι\iota and columns indexed by a finite set κ\kappa is the real number ∥rowiR∥2=(∑j∈κ|Rij|2)1/2.\|\mathrm{row}_i R\|_2 = \Bigl(\sum_{j \in \kappa} |R_{ij}|^2\Bigr)^{1/2}.(4)✦ The column norm ∥coljR∥2\|\mathrm{col}_j R\|_2 at column index j∈κj \in \kappa of a complex ι×κ\iota \times \kappa rectangular matrix RR over finite sets ι\iota and κ\kappa is the real number ∥coljR∥2=(∑i∈ι|Rij|2)1/2.\|\mathrm{col}_j R\|_2 = \Bigl(\sum_{i \in \iota} |R_{ij}|^2\Bigr)^{1/2}.(5)✦

The Claim

Theorem 1. Let ι\iota and κ\kappa be finite sets. For every complex square matrix RR indexed by ι\iota, the map X↦Φ(X*X)X \mapsto \Phi(X^* X) on complex square matrices indexed by ι\iota is Fréchet differentiable at RR with Fréchet derivative D(X↦Φ(X*X))(R)=𝒢R,R*R.\mathrm{D}\bigl(X \mapsto \Phi(X^* X)\bigr)(R) = \mathcal{G}_{R, R^* R}.(6)✦ For any complex square matrices AA and BB indexed by ι\iota and any real number t∈ℝt \in \mathbb{R}, the function s↦Φ(RA,B(s)*RA,B(s))s \mapsto \Phi(R_{A,B}(s)^* R_{A,B}(s)) is differentiable at tt with derivative dds|s=tΦ(RA,B(s)*RA,B(s))=3Re⁡tr⁡((RA,B(t)RA,B(t)*RA,B(t))*(B−A)).\left.\frac{d}{ds}\right|_{s=t} \Phi\bigl(R_{A,B}(s)^* R_{A,B}(s)\bigr) = 3 \operatorname{Re}\operatorname{tr}\Bigl(\bigl(R_{A,B}(t) \sqrt{R_{A,B}(t)^* R_{A,B}(t)}\bigr)^* (B - A)\Bigr).(7)✦ Whenever complex square matrices AA, BB, and MM indexed by ι\iota satisfy |Aij|≤|Mij||A_{ij}| \le |M_{ij}| and |Bij|≤|Mij||B_{ij}| \le |M_{ij}| for all i,j∈ιi, j \in \iota, the energy difference satisfies |Φ(B*B)−Φ(A*A)|≤3∑i∈ι∑j∈ι∥rowiM∥2∥coljM∥2|Bij−Aij|.\bigl|\Phi(B^* B) - \Phi(A^* A)\bigr| \le 3 \sum_{i \in \iota} \sum_{j \in \iota} \|\mathrm{row}_i M\|_2 \, \|\mathrm{col}_j M\|_2 \, |B_{ij} - A_{ij}|.(8)✦ For every complex ι×κ\iota \times \kappa rectangular matrix RR and all indices i∈ιi \in \iota and j∈κj \in \kappa, the entry (i,j)(i, j) of RR*RR \sqrt{R^* R} satisfies |(RR*R)ij|≤∥rowiR∥2∥coljR∥2.\bigl|(R \sqrt{R^* R})_{ij}\bigr| \le \|\mathrm{row}_i R\|_2 \, \|\mathrm{col}_j R\|_2.(9)✦

In the paper, the result reads: for finite rectangular complex matrices R−,R+R_-, R_+ of the same size, with R(z)=(1−z)R−+zR+R(z) = (1 - z) R_- + z R_+ and D=R+−R−D = R_+ - R_-, Φ(R+*R+)−Φ(R−*R−)=3∫01Re⁡tr⁡((R(z)R(z)*R(z))*D)dz\Phi(R_+^* R_+) - \Phi(R_-^* R_-) = 3 \int_0^1 \operatorname{Re}\operatorname{tr}\bigl((R(z) \sqrt{R(z)^* R(z)})^* D\bigr)\,dz, and for every rectangular matrix RR and entry (i,j)(i, j), |(RR*R)ij|≤∥rowiR∥2∥coljR∥2|(R\sqrt{R^* R})_{ij}| \le \|\mathrm{row}_i R\|_2 \, \|\mathrm{col}_j R\|_2. Rather than an integral formula on rectangular matrices, the theorem gives the Fréchet and segment derivatives (6) and (7) and the jump bound (8) for complex square matrices, and states (9) for rectangular matrices.

What Is Checked

The differentiability claims (6) and (7) are stated over all complex square matrices indexed by a finite set ι\iota and every real number tt, while (8) is stated over complex square matrices AA, BB, and MM satisfying |Aij|≤|Mij||A_{ij}| \le |M_{ij}| and |Bij|≤|Mij||B_{ij}| \le |M_{ij}| entrywise. The entrywise gradient bound (9) is stated over all complex rectangular matrices indexed by finite sets ι\iota and κ\kappa and all indices i∈ιi \in \iota and j∈κj \in \kappa.

The real trace A↦Re⁡tr⁡(A)A \mapsto \operatorname{Re}\operatorname{tr}(A) is additive and homogeneous under real scalars, defining a continuous real-linear functional on complex square matrices indexed by a finite set ι\iota.✦