Seminars

Designing appropriate neural network architectures remains a central open challenge indeep learning, with performance depending critically on depth, width, and connectivity. Cur-rent approaches rely largely on trial and error or computationally prohibitive search algorithms. This dissertation develops principled, mathematically rigorous methods for neural network architecture adaptation, drawing on ideas from topology optimization and finite element-inspired error estimation.
In the first part of this presentation, we address adaptive architecture design for regressionand classification tasks, tackling three fundamental questions: when to add capacity duringtraining phase (add a new layer in a network), where to add it, and how to initialize it. A topo-logical derivative framework quantifies the sensitivity of a shape functional to the insertion ofa new layer, yielding both the optimal insertion location and a principled initialization strategy for the added layer. Complementing this, a dual weighted residual approach from finiteelement theory decomposes approximation error across layers to guide adaptive depth refinement, forging formal connections between neural network approximation theory and classicalnumerical analysis.
The second part of the presentation introduces the Linear Latent Network (LiLaN), a novelarchitecture designed for learning solution operators of stiff nonlinear differential equationswith significant computational speedup over traditional solvers. LiLaN maps states into a la-tent space where dynamics admit closed-form solutions, eliminating numerical integration atinference time. Universal approximation guarantees are established theoretically, and numer-ical experiments in chemical kinetics, plasma physics, and phase-field modeling demonstrateup to three orders of magnitude speedup over traditional solvers. Together, these contributions provide a suite of computational mathematics tools for adaptive neural network architecture design and accelerated scientific machine learning.