Recent content in Everything Flo's on Floris den Hengst's Homepage
Our paper on ``Addressing Incomplete Data in Survival and Quality of Life Prediction: a Pancreatic Cancer Case Study" won the best paper award at AIME 2026 (AR ~20%) ! 
 In this work, we address the joint prediction of survival and quality-of-life for patients suffering from pancreatic cancer, in the face of various forms of missing data distributed heterogeneously over time.
Focusing on…
Throwback to this post about ICT.OPEN! 
 This research with Cas Hoekstra has by now been published by Transactions on Machine Learning Research . 
 We introduce Symbolic Quantile Regression (SQR), a framework that bridges the gap between high-performance numerical prediction and human-level interpretability: 
 
 SQR generates concise, closed-form mathematical expressions that…
Today I gave a guest lecture for the course Human-interactive Agent
Learning .
I presented SHARPIE , our tool for controlled human-RL experiments. 
 During the lecture, I showed how SHARPIE can be used to implement a simple evaluative feedback
experiment using
 TAMEr in the MountainCar environment.
Nothing beats putting the pressure on as a bit of live coding does, but then…
Small post to celebrate that I obtained my University Teaching
Qualification . 
 This has been useful in designing teaching material, presenting it in a way to reach an audience
with a diverse background, supervising students, creating tests and evaluation material, and to
understand how to continuously improve on all these aspects.
We contributed a tutorial on doing real-world experiments involving humans and sequential decision
making agents to a national PhD course on Reinforcement Learning for Adaptive Hybrid
Intelligence . This two-day
course is about making AI agents adapt to humans using reinforcement learning. 
 We set two goals for our tutorial: 
 
 Hands-on experience with hybrid RL example…
I spoke at ICT.OPEN 2025 about a joint work with Cas Oude Hoekstra on predicting conditional percentiles with fully transparent models using the pinball loss and symbolic regression. 
 More to follow soon hopefully! 

Recently, I needed a logo for a project I am working. I love SVG images for their versatility, but have the ability nor interest in creating on myself from scratch.
So why not use an LLM for this? 
 I tried to create an SVG directly with all the current (2024-12-17) easily available offerings 1 .
They all failed miserably. 
 
 
	 
 
Figure 1: SVG 'icon depicting…
Today I joined Bluesky , a social media platform that aims to put
users front-and-centre again. 
 See here for my profile and use this list of starter packs: 
 
 Neurosymbolic AI starter pack 
 RL 
 AI in healthcare 
 AI 

I will attend the 2024 BNAIC/BeNeLearn to present our work on
 guideline-reinforcement informed reinforcement learning in critical
care . 
 See the poster here (PDF) and slides
 here (PDF).
I visited Verena Wolf’s Neuro-Mechanistic Modeling in Saarbrücken and met with her many interesting group members there to discuss RL, safety, formal methods and RL, and causality. 
 The visit was a great opportunity to discuss recent trends and share ideas.
We have contributed papers to the health care workshop and demo track of the Hybrid Human AI
Conference . Our contributions target the development of a
lifestyle support chatbot for diabetes patients based on knowledge, semantic user modeling and
active information gathering.
Teaching staff who are active in one of our English programmes must prove that their English level is at EFSET C1 . 
 I am happy to share that, based on an oral assessment with two examiners from the VU language centre , my English level has been assessed as C2 . 
 I am still allowed to teach Msc. courses!
Our work on hashtag#task-oriented dialogue systems has been accepted at ACL’s 2024 NAACL Findings. 
 Task-oriented dialogue systems help users solve tasks in their daily lives.
In order to do so, these systems have to identify the users intent. 
 Dialogue systems may be uncertain about the user intent. In these cases, the system can ask the user to clarify, i.e. select from the…
I contributed a talk on guideline-informed reinforcement learning with Annette ten Teije to the AI & Health Winter School organized by the VU Campus Center Artificial Intelligence & Health. 
 The winter school was attended by over 100 in-person and virtual participants interested in the intersection of AI & Health. 
 You may find the slides which focus on how the knowledge-driven AI…
Today I succesfully defended my PhD. thesis Learning to
Behave - Reinforcement Learning in Human Contexts.
I am now a doctor of philosophy! 
 In the Netherlands it is customary to give a 10 minute talk prior to the actual defence. This talk
is aimed at a general audience because the defense is a public affair and because it is nice to
tell friends and family what the thesis is…
For the past couple of years, I have always loved working with Msc. and Bsc. students. Usually, I
have been involved in the role of supervisor for the thesis.
In doing so, I have learnt a lot about what makes a good AI research project and what I can do to
improve these projects. 
 The following is a distillation of some of that knowledge. Some of it applies only to theses,…
Last week I attended the International Joint Conference on Artificial
Intelligence (IJCAI) in Vienna. Being `co-hosted’ with the European Conference
on AI, the official name of this years’ edition was IJCAI-ECAI 2022.
It was a great week with too many impressions and learnings to list.
Here are some personal highlights: 
 
 presented our paper on Option Machines…
Past week, I attended and assisted at the Reinforcement Learning Summer
School organized by Vincent François-Lavet. We had
some great lectures on a wide variety of topics, including pure exploration in
bandits, the exploration-exploitation tradeoff in RL, MCTS, symmetries and
state similarities, world models and hierarchical and distributational RL. 
 The summer school was…

 Gaussian Processes are a fascinating tool for usage in RL due to modelling uncertainty and data efficiency 
 I briefly introduced GP’s and shown how/why they are used in RL 
 
 Gaussian processes (GPs) are a fascinating tool in the machine learning toolbelt. They stand out
for a couple of reasons: some people will like them for their data efficiency, others love…
A paper on Deep Reinforcement Learning (DRL) for strategic workforce planning co-authored with Yannick Smit, Sandjai Bhulai and Ehsan Mehdad is accepted as a long paper at the LOD conference. 
 In this paper, we model strategic workforce planning as a stochastic nonlinear optimization problem, learn a generative model from data and use it as a simulator in a simulation-optimization approach.…
The paper Reinforcement Learning with Option
Machines co-authored by me, Vincent
François-Lavet, Mark Hoogendoorn and Frank van Harmelen is accepted as a long
presentation (~3% acceptance rate) at IJCAI . 
 Stay tuned for details! 
 Update 2022-08-01 : I presented my paper at IJCAI, it was great fun ! 
 Update 2022-09-23 : The IJCAI proceedings are now available .
Reinforcement Learning has proven to be capable of outperforming humans on
various tasks by interacting with and experimenting some environment. This
makes it one of the most interesting and promising AI solutions to problems
that require complex behaviors which we are unable to fully define upfront but
can assign an objective score to. 
 In many settings of interest such as in…
Upcoming weekend will be a virtual conference on reinforcement learning (RL) in real-life. 
 It is of great interest to me primarily due to its program but also due to its organisation. 
 The virtual conference consists of two panel sessions and a virtual `poster’ session with
pre-recorded videos. There is a slack workspace for discussion, and questions to the panelists…
Update 2020-06-27 : this paper was presented as a `poster’(video) at the RL for Real Life virtual
Conference . Read more
 here . 
 
 Reinforcement learning (RL) is becoming an increasingly popular tool to tackle hairy problems
using data. A nice example of such a hairy problem is personalization . Personalization
refers to a task central to many applications of data…
Mickey, a master student I was co-supervising, recently published a paper based on his thesis work in ACM’s Conference for Human Information Interaction and Retrieval. His work focused on interfaces for collecting high-quality user satisfaction ratings for dialogue systems. 
 User satisfaction is an important indicator in the design, evaluation and adaptation of dialogue…
Today, my work is – very briefly – mentioned in a post by Nick Kivits. 
 The post is part of a blog series on innovation in the media and it is hosted on Villamedia, a
Dutch website on journalism.
In a previous post I explained how reinforcement
learning (RL) can be used to make chatbots better from its experience with users. RL allows chatbots to learn what to say by interacting with users. This allows for chatbots to tailor their behavior to preferences of groups of individuals, e.g. to personalize the interaction. I recently presented our paper on
this topic, specifically on…
I was recently asked to help organise a ‘music bingo’ for students of horseriding association
 BLOK . To be completely honest with you, I feel that bingo is
probably one of the most boring party games out there as the only ‘skills’ involved are paying attention
and bookkeeping. 
 Music bingo, however, puts a nice twist to the original game that makes it…
Just returned from a lovely stay in Chania, Crete for the Advanced Course on Artificial
Intelligence , a yearly summer school by EurAI .
Besides the beautiful scenery and lovely people, there were lots of interesting talks to enjoy.
Most talks stuck to the theme ‘AI for multi-agent worlds’ quite well. 
 Although I learned a lot in general, there were some key take-aways…
A couple of days ago, RL founding father Rich Sutton posted a blog
post . My reading is as follows: 
 
 Moore’s Law has consistently made general, compute-based methods outperform task-specific,
domain knowledge-based methods for tasks in the AI `sphere of interest' 
 This has served as a bitter lesson for many (all?) researchers that have focused on developing
such…
The value offering of most contemporary chatbot platforms consists of packaging state-of-art
Automated Speech Recognition (ASR, or ‘speech-to-text’), Natural Language Understanding (NLU) and
Voice Synthesis into a comprehensive API. The API typically also includes some programming
model for dialog control such as DialogFlows’ Contexts and follow-up
Intents and…
I completed the SIKS course ‘Research Methods and Methodology in IKS’,
spanning a wide range of topics such as research methods, Design Research, many issues involved
with evaluation in ML and how the field of IR has a strong history of rigourous evaluation (much
to learn there for ML/RL researchers!). 
 To top it all off, the course was held in the lovely location (see…
I just completed the course ‘Writing a Scientific Article’ at the VU Language
Centre .
Although I joined the course thinking I knew a fair share of writing in English, I learned a great
deal about scientific writing and picked up some neat tricks to improve writing.
As a bonus, I got some great feedback on an survey paper on Reinforcement Learning…
Today I completed the HPC course at VU University. The course was well
organized and had some interesting courses. Some of the courses contained very little new
information for me, though. I would recommend researchers/PhD. Students with some knowledge of
programming/bash, Linux and clusters but with no experience working with SurfSARAs offerings to
follow the following courses:…
Little under three years ago, in March 2016, a Reinforcement Learning (RL) algorithm beat Lee
Sedol, a pro
player in a match of Go. This game had been considered too hard for algorithms due to the
 astronomical 
number of board game configurations and the AlphaGo team therefore baffled experts in AI with their
accomplishments. Nobody seemed to be aware that techniques were…
You will be automatically redirected to the tool at http://rl4personalization.fht800.ops.few.vu.nl/ …