[2605.06938] A Nonlinear Singular Worth Principle for Neural Networks

View a PDF of the paper titled A Nonlinear Singular Worth Principle for Neural Networks, by Brian Charles Brown and 4 different authors

View PDF
HTML (experimental)

Summary:Lately Brown et al. [2025] established a singular worth decomposition (SVD) for maps (particularly nonlinear) satisfying sure norm circumstances. We show that the majority fashionable neural architectures admit this nonlinear SVD (NLSVD) representation—with no change in input–output behavior—and enumerate the lessons lined. On this factorization the community is a left-invertible nonlinear map adopted by a remaining linear layer. Furthermore, the left-invertible issue is norm-preserving, so distances within the embedding (activations earlier than the ultimate linear layer) calibrate on to distances in enter area. We introduce a versatile structure that yields an specific decomposition at coaching time, a data-driven algorithm for estimating the illustration from educated fashions, and the mathematical foundations for nonlinear analogues of row and null areas in neural networks. Empirical case research illustrate makes use of of the idea for latent-space pullback (visualization and information technology), bias detection, and membership-inference robustness beneath coaching. Altogether, these foundations assist new approaches to core issues in neural-network evaluation.