Two Tales of Reward Design
At the end of August, 2025, I gave a talk at the University of Alberta. The talk was titled “Two Tales of Reward Design”.
Craig Sherstan
Write an awesome description for your new site here. You can edit this line in _config.yml. It will appear in your document head meta (for Google search results) and in your feed.xml site description.
Latest posts
First Tokyo AI RL Talk
I recently gave a talk at the first Reinforcement Learning focused session of Tokyo AI Talks.I gave a 10 minute introduction to RL and then talked about GT Sophy - the RL-based agent we trained to outrace the top human racers in Gran Turismo
ALA 2020: Temporally Extended Auxiliary Tasks
My paper “Work in Progress: Temporally Extended Auxiliary Tasks” has been accepted to the Adaptive and Learning Agents workshop at AAMAS 2020.This work was done in collaboration with Bilal Kartal, Pablo Hernandez-Leal and Matt Taylor while I was intern at Borealis AI last summer. It was a pleasure to work with all of them.The paper focuses on the question, what effect does prediction timescale of…
Gamma-Nets: Generalizing Value Estimation over Timescale
I’m happy to announce that my paper: “Gamma-Nets: Generalizing Value Estimation over Timescale” has been accepted for oral presentation at AAAI 2020!You can find the arXiv copy here: https://arxiv.org/abs/1911.07794This is work that I started as an intern with Cogitai in 2017.
Candidacy Passed!
This is a bit old, but I successfully passed my PhD candidacy exam back in November, 2018. So all I have left to graduate is to finish my research (which is nearly complete), write my thesis and defend.On that note, I’m officially on the market for a real adult job. I’m looking for a research scientist position in any of the following areas: robots, human augmentation, neural interfaces,…
Borealis AI Internship!
Good news (for me at least)! I’ll be starting an internship with Borealis AI, Edmonton in April. I’ll get to work with some great researchers including Matt Taylor.
Generalizing Value Estimation over Timescale
Generalizing Value Estimation over TimescaleSherstan, C., MacGlashan, J., Pilarski, P. M. (2018) Generalizing Value Estimation over Timescale. FAIM Workshop: Prediction and Generative Modeling in Reinforcement Learning (PGMRL). Stockholm, Sweden. July 15.General value functions (GVFs) are an approach to representing models of an agent’s world as a collection of predictive questions. A GVF is…
Successor Representation Literature
This is a list of literature related to the successor representation. It is not exhaustive and I have not read it all. Right now it is just a list; if I have time I’ll add summaries for those papers I’ve read.Learning the SRDayan, P. (1993). Improving Generalization for Temporal Difference Learning: The Successor Representation. Neural Computation, 5(4), 613–624.Gehring, C. A. (2015). Approximate…
Accelerating Learning in Constructive Predictive Frameworks with the Successor Representation
Accelerating Learning in Constructive Predictive Frameworks with the Successor RepresentationI’ve got a new paper with Marlos C. Machado and Patrick M. Pilarski on using the successor representation to speed up learning of incrementally added general value functions.
Directly Estimating the Variance of the lambda-Return Using Temporal-Difference Methods
Directly Estimating the Variance of the lambda-Return Using Temporal-Difference MethodsI’ve got a new paper on arXiv on using TD methods to directly estimate the variance of the lambda-return.