A PDE-Based Convolutional Neural Network with Variational Information Bottleneck: Experimental Evaluation and Generalization Analysis

Abstract

We present a hybrid convolutional architecture that combines trainable PDE-based preprocessing with a Variational Information Bottleneck (VIB) to improve generalization in image classification. The PDE stage applies a small number of discretized Laplacian steps with learnable step size and depthwise coupling, injecting physics-inspired inductive bias into early feature maps. A tensor-wise VIB module then parameterizes a Gaussian latent (μ, log σ2) via 1×1 convolutions and enforces information compression through a KL penalty to a unit prior, encouraging retention of task-relevant features while discarding nuisance variability. The compressed representation feeds a ResNet-18 backbone adapted for CIFAR-10 inputs. On CIFAR-10, systematic variation of the VIB weight β shows that moderate compression yields improved test performance and training stability relative to both a baseline CNN and a PDE-only variant. Qualitative analysis indicates smoother activations and reduced sensitivity to input noise, consistent with the information-theoretic objective. The results suggest that PDE priors and variational compression act complementarily, offering a principled path to robust and generalizable convolutional models.