Helmholtz AI Researchers @ ICML 2026
Researchers from Helmholtz AI were represented at this year's International Conference on Machine Learning (ICML 2026), with contributions spanning LLM safety evaluation, mechanistic interpretability, vision transformers, time series modelling, and single-cell genomics.
Zeynep Akata and the Akata team are presenting multiple papers at the main conference, including a Spotlight presentation. Dissecting Multimodal In-Context Learning: Modality Asymmetries and Circuit Dynamics in Modern Transformers, with Yiran Huang, Karsten Roth, Quentin Bouniot, and Wenjia Xu, investigates how transformers learn to associate information across modalities from in-context examples. The team also contributes Beyond the Final Layer: Attentive Multi-Layer Fusion for Vision Transformers, with Laure Ciernik, Marco Morik, Lukas Thede, Luca Eyring, Shinichi Nakajima, and Lukas Muttenthaler, and Sparse Autoencoders are Topic Models, with Leander Girrbach, which reframes sparse autoencoder features as thematic components rather than steerable directions.
Our newest PI, Leo Schwinn (Schwinn team), has three papers accepted at the main conference. A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness, with Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, and Stephan Günnemann, shows that automated LLM-as-a-Judge frameworks can degrade to near-random performance under the distribution shifts inherent to red-teaming. A companion position paper, LLM-Safety Evaluations Lack Robustness, with Tim Beyer, Sophie Xhonneux, Simon Geisler, and Gauthier Gidel, analyses noise and inconsistency across the LLM safety evaluation pipeline and proposes guidelines to address it. Schwinn's team also contributes Byte Pair Encoding for Efficient Time Series Forecasting, with Leon Götz and Marcel Kollovieh, introducing the first pattern-centric tokenizer for time series data.
Stefan Bauer (Bauer team) contributes two papers. From Growing to Looping: A Unified View of Iterative Computation in LLMs, with Ferdinand Kapl, Emmanouil Angelis, Kaitlin Maile, and Johannes von Oswald, provides a mechanistic unification of two techniques linked to stronger reasoning in language models. The team also presents "Are Object-Centric Representations Better at Compositional Generalization?", with Kapl, Amir Mohammad Karimi-Mamaghan, Maximilian Seitzer, Karl Henrik Johansson, Carsten Marr, and Andrea Dittadi.
Fabian Theis, Scientific Director of Helmholtz AI, co-authors Flow-Based Density Ratio Estimation for Intractable Distributions with Applications in Genomics, with Egor Antipov, Alessandro Palma, Lorenzo Consoli, Stephan Günnemann, and Andrea Dittadi. The work introduces scRatio, a tool for scoring individual cells for differential abundance across experimental conditions, detecting batch effects, and identifying synergistic drug combinations in single-cell genomics data.
The Fortuin team is represented at the workshop level: Vincent Fortuin is co-organizing a workshop on philosophical foundations of machine learning concepts alongside Junhyung Park, Fanny Yang, Bernhard Schölkopf, Konstantin Genin, Thomas Icard, and Jaesik Choi, and co-authors the position paper "Agentic AI Orchestration Should Be Bayes-Consistent" together with a large international author group.
ICML 2026 took place in Seoul, South Korea, July 6 to 11, 2026.