IDA Machine Learning Seminars - Spring 2025
Wednesday, February 12 at 13:30, 2025
Investigating Batch Inference in an Sequential Monte Carlo Framework for Deep Learning
Andrew Millard, University of Liverpool
Abstract: Bayesian Inference provides a principled framework for utilising probabilistic methods to estimate the underlying posterior distribution of various models, including neural networks. Bayesian neural networks provide uncertainty quantification while still performing competitively in terms of accuracy and optimal loss compared to traditional methods. However, direct computation of neural network posterior distributions is often intractable. Mini-batch stochastic gradient descent is often used in deep learning settings in order to speed up training and reduce computational load. We investigate extending the use of mini-batches in a Bayesian framework, specifically in sequential Monte Carlo samplers to speed up inference. We also investigate the use of data annealing schemes and demonstrate that gradually increasing the batch size dramatically reduces training time with no drop-off in loss or accuracy.
Location: Alan Turing
Organizer: Louis Ohl
Wednesday, April 9 at 13:30, 2025
An introduction to Topological Data Analysis
Mathieu Carrière, Centre Inria d’Université Côte d’Azur
Abstract: Topological Data Analysis (TDA) is a growing field of research at the intersection of data science and computational geometry and topology. It has encountered key successes in several different applications (ranging from cancer subtype identification in bioinformatics to shape recognition in computer vision, just to name a few), and become the landmark product of several companies in the recent years. Indeed, many data sets nowadays come in the form of point clouds embedded in very large dimensional spaces, yet concentrated around low-dimensional geometric structures that need to be uncovered. Unraveling these structures is precisely the goal of TDA, which can build descriptors that can reliably capture geometric and topological information (connectivity, loops, holes, curvature, etc.) from the data sets without the need for an explicit mapping to lower-dimensional space. This is extremely useful since the hidden, non-trivial topology of many data sets can make it very challenging to perform well for classical techniques in data science and machine learning, such as dimensionality reduction.
Location: Alan Turing
Organizer: Louis Ohl
CANCELED Wednesday, May 7 at 13:30, 2025
Efficient Probabilistic Machine Learning via Surrogates and Amortization
[Luigi Acerbi](https://lacerbi.github.io/), University of Helsinki
Thursday, May 15 at 15:15, 2025
Lifted Networks, their Smoothness Properties and Beyond
Christopher Zach, Chalmers University of Technology
Abstract: Today error back-propagation is the main tool underneath most training methods for artificial neural networks. Unfortunately, it is also an algorithm that is highly implausible to be realized in our brains, which e.g. raises the question of alignment between natural intelligence and current AI. Consequently, there is an active search for biologically more plausible alternatives to back-propagation taking place. One set of such neuroscience-inspired methods builds on the old idea of contrastive Hebbian learning, where network activities are not computed in a forward pass but inferred by minimizing an appropriate “network energy.” We first show how various recent flavors of contrastive Hebbian learning naturally emerge by extending suitable bilevel minimization programs to deeper nesting levels, which also leads to algorithms that are nicely suited for certain neuromorphic and analog computing hardware. Further, we discuss the intrinsic adversarial robustness properties featured in some of the resulting neural network models. If time permits, the presentation concludes with more speculative thoughts on “how to liberate connectionism from error back-propagation.”
Location: Ada Lovelace
Organizer: Fredrik Lindsten
Monday, May 19 at 14:00, 2025
This seminar comprises two talks.
Misspecification in Gaussian process modelling
Toni Karvonen, Lappeenranta-Lahti University of Technology LUT
Abstract: Gaussian processes provide both predictions at unseen data locations and associated quantification of uncertainty. It is difficult to select a Gaussian process model that correctly encodes some of the most important properties of the latent function. In this talk I discuss smoothness misspecification, which occurs when a Gaussian process model is either smoother or rougher than the truth. I review a number of recent results that demonstrate that smoothness misspecification need not be a problem when it comes to the rate of convergence of the posterior and the reliability of uncertainty quantification.
Bayesian inference for rough feature reconstructions
Lassi Roininen, Lappeenranta-Lahti University of Technology LUT
Abstract: Edges in imaging, that is sharp discontinuities in intensity, pose a significant challenge for inverse problems algorithms that often rely on Gaussian assumptions. Non-Gaussian heavy-tailed priors, which can better model the sparsity and sharp transitions inherent in edges, offer an alternative for edge-preserving image reconstructions. We consider the inherent difficulties in handling edges and highlight the potential of heavy-tailed prior models to transform this problem into a practical engineering solution.
Location: John von Neumann
Organizer: Zheng Zhao