Reflections on ICML 2026

Connect with us: Valence is constantly seeking talented individuals with diverse backgrounds and expertise to join our team. Explore open roles here.

On July 6-11th, the ICML 2026 conference brought over 22,500 (in-person and virtual) attendees to Seoul to discuss the latest breakthroughs in the AI community. AI for biology and health was far from just another application this year, coming in 5th place across top subject areas and making up 2.5% of all accepted papers (an increase from last year). Three workshops on biology and health (Generative and Agentic AI for Biology, Multi-modal Foundation Models and LLMs for Life Sciences, Structured Data for Health) plus a keynote by Aviv Regev further demonstrated the importance of AI4Bio in the overall AI community. Multiple themes emerged from the meeting, some specific to AI for drug discovery and some spanning the entire scope of the conference, with a considerable overlap between them.

In the era of LLMs firmly gripping the advances in the ML/AI community, efficiency (both at training and inference time) continues to be a central problem with considerable efforts spent to address it. A good illustration of it came with Cerebras, the wafer-scale engine company for fast inference, which had a large presence at ICML and boasted long queues to join their party, celebrating a recent deal with OpenAI as well as their own IPO.

In the AI for biology community, practitioners were thinking about novel strategies for tokenization to improve scalability or better match the properties of biological data. dnaHnet, a scalable model for genetic sequence modeling, learned which k-mers to compress based on where the sequence was less predictable. scBIG partitioned semantically related genes into modules and attended between them to generate representations for latent flow matching. STACK, a foundation model (FM) for single-cell biology (preprinted prior to ICML and highlighted by Yusuf Roohani in his invited talk at GenBio), also grouped multiple genes into tokens, yet which genes to combine was learned during training without relying on prior information. ST-JEPA tokenized spatial transcriptomics samples into hierarchical representations across cell neighbourhood, cell, and gene resolutions. A consistent theme here was the view that specialized tokenizers should be a part of the model or done away with entirely when working with genomic data; the standard approaches from language modeling are not enough.

Studies also emphasized scalability gains as models scale into larger parameter regimes. However, the assumption that FMs for biology will follow the same scaling trajectory as LLMs, where more parameters and more data yield monotonic improvements, remains uncertain. Recent work found that transcriptomic FMs show no clear data scaling laws, with smaller models outperforming counterparts trained on larger datasets and with more parameters. This suggests that the bottleneck in biological data modeling may not be compute efficiency or data scale, but rather the sparsity of learnable signal in the raw input. It will thus be important for future work to evaluate the practical value of the proposed compute efficiency gains, especially in comparison to richer supervision or data curation strategies that increase data quality and diversity.

A large theme coming from both academic and industry labs was perturbation prediction for transcriptomics, which signals the importance of this task for drug discovery. PerturbDiff used the flow matching framework to learn a family of plausible response distributions to a perturbation, instead of a single static one. scBIG (mentioned above) also used flow matching to model single-cell perturbation response with module-inductive representations. As both methods reported state-of-the-art performance, future work will be important to identify and build on winning strategies as the community continues to evaluate newly proposed methods (including our recently published TxPert). Another interesting method for perturbation prediction, PerturbODE, instead relied on neural ODEs and additionally inferred gene regulatory networks, which has the potential to advance interpretability and as such be especially valuable for drug discovery.

Model evaluation was also front-of-mind for many authors at ICML as we saw many benchmarking studies and position papers asking: “what do models actually learn?”. An interesting submission on the representation learning topic investigated the differences between patterns learnable by nonlinear versus linear autoencoders (AEs). The authors designed a dataset with parts of the latent structure invisible to PCA but recoverable by minimal nonlinear AEs. Interestingly, they observed that, despite this, linear AEs achieved a lower test loss compared to their nonlinear counterparts. The conclusion was that reconstruction error alone can give a misleading picture of what a model has learned and that, in self-supervised learning, downstream benchmark performance is increasingly important.

In the AI for biology community, one study took a closer look at what is now a mainstream finding causing much debate in the field: the mean baselines outperform large FMs for biology on perturbation prediction. The authors proposed signal dilution as an explanation: only a small number of genes change between perturbed and other cells, making detecting them difficult and rewarding the simple mean baseline. Their proposed solution augmented the regular MSE loss with gene weights derived from perturbation-specific DEGs. This is a timely recommendation and it will be interesting to see how training with this loss affects model performance or whether it could benefit from computing DEGs across many datasets given reproducibility considerations around DEGs.

A similar line of questioning was extended across many biological domains, from biomedical imaging to drug response or binding affinity prediction. Multiple studies presented evidence of naive baselines performing competitively with the existing complex models tailored to specific tasks in drug discovery. The authors advocated for including a suite of simple baselines into any new model’s evaluation to assess ceiling performance and identify failure modes, and provided useful tips for how to define these for a specific task. A resounding conclusion from this body of work is: we will always need better benchmarks, and some of them better be really simple.

The AI community clearly has a strong interest in multimodality, with many submissions looking at the vision-language or video-audio modality intersection. In AI4Bio, the multimodal efforts revolved around modeling spatial transcriptomics (ST) and linking it to histopathology data. MoLF, a conditional flow matching framework with a mixture-of-experts velocity parameterization, predicted ST from histology images in a pan-cancer regime, reportedly achieving state-of-the art performance. Previously mentioned ST-JEPA jointly modeled the expression and spatial components of ST, converting them into multi-scale tokens representing genes, cells, and cell contexts. Though not published at ICML, Aviv Regev highlighted their preprint SCHAF, a method for inference of spatially-resolved scRNA-seq from histology images. As the field increasingly focuses on histopathology data, much easier to access than sequencing data, we can expect future multimodal efforts to combine histopathology data with genomic, transcriptomic, and protein FMs, in the pursuit of a truly virtual cell.

Another angle to multimodal modeling came from combining transcriptomics with high-content imaging. Regev presented PerturbPair (currently a preprint), an experimental platform for parallel Perturb-seq and optical pooled screening, allowing measurement of perturbation-specific transcriptomic and morphological phenotypes. The authors developed a contrastive latent variable model to leverage cross-modal relationships between these datasets, improving transcriptomic perturbation prediction by using morphological features and enabling imputation from one modality to another. This signals the field’s sustained interest in phenomics, which will likely remain a key data layer in multimodal efforts for drug discovery.

This year has also seen a sharp increase in interest in agentic and autonomous discovery. Indeed, the GenBio workshop included “agentic AI” into its title for the first time, while the main conference featured at least two keynotes explicitly focused on agents (and, one could argue, two more were heavily related). Looking at applications the authors sought to agent-ify, the community’s interest in perturbation prediction was evident once again. One work approached chemical perturbation prediction from bulk sequencing data in a multi-agent framework, combining specialized knowledge agents with a synthesis agent and judges. Another study focused on the lab-in-the-loop setting, aiming to select genes to perturb looking at the resulting morphological features and using simulated experimental feedback. Related work, though not agentic, proposed a novel acquisition function for active hit discovery in perturbation experiments. OpenAI and Anthropic demoed GPT-Rosalind and Claude Science in their invited talks at the GenBio workshop, and Aviv Regev talked about lab-in-the-loop efforts at Genentech in her keynote. While it is exciting to see early progress on formalizing what autonomous discovery can look like, the current readout from published work is: marginal improvements without clear game changer in heavily constrained setups – still a few steps away from a true lab-in-the-loop in all of its messy glory.

It was fitting for ICML 2026 to take place in a stunning, energetic, and densely populated Seoul, reflecting the amount of talent, resources, and innovation brewing in our ever rapidly-expanding AI community. What research will we see more of in the coming months? If I had to bet, I would put money on precise model evaluations, interpretable and mechanistic AI for everyone’s favourite drug discovery task – perturbation prediction, multimodal efforts leveraging any considerably abundant biological modality, and of course, more agentic discovery and lab-in-the-loop efforts (soon to be turned into human-in-the-loop-of-agents).

And lastly, a note on logistics (verging on the existential), which came up as a discussion point in the Town Hall and many workshop panels. How must we change conference processes to cope with the sheer scale of submissions that will likely continue to increase? We saw the community, including new players such as Anthropic, turn its attention to agentic and autonomous discovery. Any such successful process culminating in a worthwhile, even if modest, discovery, will likely include autonomous manuscript generation at the end. This means that, this time next year, we may have a whole new stream of AI-generated papers to contend with. For comparison, the year-on-year growth in ICML submissions was 32% in 2025 and a whopping 98% in 2026, with the conference basically doubling in size this year (with ~24k submissions and ~6k accepted papers). What will 2027 bring? Are we entering the new era of agents reading agent-written papers? And if so, where do we, human researchers fit in? Ask me again next year.

This post is part of “Inside Valence”, a series where you’ll get a behind-the-scenes look at our research, exploring new ways to predict, explain, and ultimately decode biology. If this resonates, consider subscribing!