For small covariate dimension, shape constrained inference is a well-established topic within nonparametric statistics. For large covariate dimensions, one naturally wants to introduce machine learning based methods. Input convex neural networks (ICNNs) were designed as a network architecture to learn convex functions. In this talk, we introduce Hyper Input Convex Neural Networks (HyCNNs). HyCNNs combine the principles of Maxout networks with ICNNs to create a neural network that is always convex in the input, theoretically capable of leveraging depth, and performs reliable when trained at scale compared to ICNNs. Concretely, we prove that HyCNNs require exponentially fewer parameters than ICNNs to approximate quadratic functions up to a given precision. Throughout a series of synthetic experiments, we demonstrate that HyCNNs outperform existing ICNNs and MLPs in terms of predictive performance for convex regression and interpolation tasks. We further apply HyCNNs to learn high-dimensional optimal transport maps for synthetic examples and for single-cell RNA sequencing data, where they oftentimes outperform ICNN-based neural optimal transport methods and other baselines across a wide range of settings.

This is joint work with Shayan Hundrieser and Insung Kong. For more details, see arxiv.org/pdf/2604.26942.