Table of Contents What's the difference? A thought experiment Making a choice under uncertainty Predicting someone else's choice Language models are uncertain simulators How to mitigate the simulator-uncertainty gap Simulating a less-certain persona Simulating a persona that has access to external tools Starring: A Large Language Model As "A Large Language Model" Using the full distribution of…
Note to present-day readers: The use of the word "transformer" in this post, and in the corresponding ICLR paper, is unrelated to the now-popular Transformer architecture introduced in the paper "Attention is all you need", which was released shortly after this post. Recently, I submitted a paper titled "Learning Graphical State Transitions" to the ICLR conference. (Update: My paper was accepted!…
I had some free time recently, and felt like making a new watchface for my Pebble Time. The basic idea is that the outer spiral makes one revolution every hour, but the previous path of the spiral remains. Over time, the whole watchface fills up with the growing spiral. As with my previous watchface, the code is available on GitHub . You can also download it from the Pebble App Store if you have a…
Last semester, I was part of the Clay Wolkin Fellowship at Harvey Mudd. The fellowship consists of a group of students (mostly Engineering majors, but some CS also) who work on interesting electrical-engineering-focused projects. The project I worked on was the "LEG Processor", an open-source pipelined processor that implements the ARMv5 instruction set and can boot the Linux kernel (3.19) in…
This summer, I had the chance to do research at Mudd as part of the Intelligent Music Software team. Every year, under the advisement of Prof. Robert Keller, members of the Intelligent Music Software team work on computational jazz improvisation, usually in connection with the Impro-Visor software tool. Currently, Impro-Visor uses a grammar-based generation approach for most of its generation…
At Harvey Mudd, I'm taking a cool class called "Discrete and Computational Geometry", a special topics course taught by Professor Satyan Devadoss . It's a very interesting class. In lieu of normal problem sets, we instead do a bunch of group projects, each one very freeform. The basic instructions are "go make something related to this class". Here are a couple of the projects my group made: Read…
Story time: So a few years ago, after playing Portal and loving it, and after making my iOS word game Wordchemy, I was kind of in a game-design zone. I brainstormed a ton of ideas: from memory games, to arcade-style games, to physics-based puzzle games, and many more. One idea, however, stuck with me. The basic idea was this: what if you could go backwards in time? Not as in traditional time…
In my previous post, I described my music-composing neural network. In some of the time since then, I extracted some interesting images to visualize the neural network's internal state. And this is what I got: Read more… (8 min remaining to read)
(Update: A paper based on this work has been accepted at EvoMusArt 2017! See here for more details.) It's hard not to be blown away by the surprising power of neural networks these days. With enough training, so called "deep neural networks", with many nodes and hidden layers, can do impressively well on modeling and predicting all kinds of data. (If you don't know what I'm talking about, I…
For a few years, I've had a Pebble, and I recently upgraded to a Pebble Time. So far I really like it: its more comfortable, it looks cooler, and it has a color display! However, my favorite watchface for Pebble ( Squared by lastfuture) was only black-and-white, and it seemed like a waste to use a non-color watchface on my color Pebble. So, naturally, since the watchface was open-source and Pebble…