Robust and trustworthy artificial intelligence and computer vision research and engineering
I spent the past few years working on AI for health. Starting with custom multimodal encoders, post-training, and sophisticated multi-agent architecturs, I now see modeling work becoming less and less important for domain-specific applications. I believe that researchers should embrace this change and focus on owning the application, workflows, and evals instead. The post Domain-Specific AI Should…
Having worked on robustness problems during my PhD, I see many of the characteristics appearing in the evaluation of LLMs and AI systems. Adversarial attacks such as jailbreaks are becoming more relevant, edge cases finally become relevant, test sets and benchmarks are not static anymore but introduce distribution shift, and more sources of randomness make properly estimating accuracy difficult.…
Recently, I attended the Responsible AI for Social and Ethical Healthcare 2025 “2.0” Symposium organized by, among others, Harvard Medical School. The symposium featured various panels on topics surrounding generative AI, in particular multimodal LLMs, in healthcare. I was invited to speak on a panel about aligning AI to clinical values. In this article, I want to share a brief statement I…
Writing and responding to reviews is the bread and butter of any academic and especially in AI research, PhD students are confronted with both rather early compared to other displicines. Unfortunately, I found that drafting reviews and rebuttals is a skill that is rarely properly taught as part of PhD programs. Thus, in this article, I want to share some of the lessons I learned throughout the…
Watermarking AI-generated content has the potential to address various problems that generative AI threatens to aggravate misinformation, impersonation, copyright infringement, web pollution, etc. However, it is also controversial with many researchers and users worrying about reduced quality and questioning whether watermarking actually works and helps. In this article, I want to share some of my…
With the goal of deploying generative AI systems, rater studies are becoming increasingly common and important. This means more and more researchers and engineers face the challenge of actually planning and conducting rater studies for AI systems. In this article, I want to share some lessons I learned over the past year. The post Thoughts and Lessons for Planning Rater Studies in AI appeared…
Dealing with rater disagreement is becoming more important in AI, especially for LLMs and in specialized domains such as health. In the past year, I helped open source two datasets allowing to study rater disagreement in the health domain: a relabeling of MedQA, a key benchmark for evaluating medical LLMs, and a dataset including differential diagnosis ratings for skin condition classification.…
In this article, I want to share some thoughts on the difference between research ideas and technology, particularly in machine learning. This distinction is have been contemplating since starting my PhD. After joining Google DeepMind and being involved in product releases such as SynthID , I realized that it can be useful to distinguish between research ideas and technology in many projects, both…
Engineering and running experiments are a key component of most PhDs in AI. While there are plenty of more theoretical topics that are often limited to smaller scale experimentation, the trend has definitely been to scale up models, datasets and experiments. Thus, being an effective engineer and setting up experiments in an efficient way can be a crucial element of being successful during a PhD.…
Over the past months, I have given several talks about Monte Carlo conformal prediction and the problem of calibrating with uncertain ground truth, for example, stemming from annotator disagreement. Each time, the audience had great questions and ideas for extensions and interesting applications. In this article, I want to provide a sort of FAQ for our work. The post FAQ for our Monte Carlo…