In this article we’ll learn how to train a neural network to play Tic-Tac-Toe using reinforcement learning in Jax . This article will aim to be more pedagogical, so the code we’ll end up with won’t be super optimized, but it will be fast enough to train a model to perfect play in about 15 seconds on a laptop. Code from this page can be found at this Github repo as well as in a Colab notebook…
A few years ago I outlined the so-called “ triviality argument ” that consciousness cannot be reduced to the process of computing a particular kind of computer program. At the core of the argument is the recognition that there is no unique, objective way to define computational states in the physical world. Consequently, it is always possible to find some (perhaps complicated) mapping of arbitrary…
Here in the northern hemisphere the vernal equinox just passed and the days are quickly getting longer. One of my colleagues lives in Stavanger, Norway. Our team’s semi-weekly standup is at 6:30pm his time, so I’ve been accustomed to seeing the window in his background be pitch black for the past six months. But from one meeting to the next, his window went from pitch black to bright. This led me…
Spend enough time in the tech world and you will sooner or later find that it is a common article of faith that computers will one day gain consciousness. With powerful enough computers and sophisticated enough AI, the reasoning goes, it is inevitable that computers will eventually become conscious just as we are. In fact, some are even bolder and claim that this isn’t just something that will…
The paradox Stein’s paradox is among the most surprising results in statistics. The basic idea is easily stated, but it is difficult to understand how it could possibly be true. The premise is this: suppose that I have a Gaussian distribution with a variance of unity and some mean which I don’t tell you. I then draw a single sample from this distribution, give it to you, and ask you to guess the…
I’ve been using Purism’s Librem 13 laptop as my primary personal laptop for just over a year now. Given that Purism laptops are not widely used I thought I would write up my experience with it for anyone who is considering a Librem laptop. (The Librem 13 is no longer available, but Purism recently released the Librem 14 instead.) tl;dr : As much as I wanted to like it, the build quality issues are…
Voting is an odd thing. We all in some sense know that our individual votes are exceedingly unlikely to impact an election, but many of us find ourselves highly motivated to vote anyway. There are, of course, many reasons to vote — a sense of civic duty, an expression of political solidarity, or just as something to do brighten up a gloomy Tuesday morning. But let us set those aside for the time…
Sunday, February 10, 2019 began as a wonderful day. I showered in the morning and I distinctly remember feeling well rested and all around quite good. But as I was drying off I felt a sharp pain in my upper back like I had pulled a muscle or pinched a nerve. I started to suspect that something was amiss as my wife and I walked to mass that morning. Our church is down a pretty steep hill from our…
About a year ago I finished up a year of doing machine learning research at Google as a Google AI Resident. Technically when I started I was a Google Brain Resident, but midway through the powers that be decided to rebrand the program and thereafter I was supposed to refer to myself as a Google AI Resident. I was in the second cohort so the program as a whole was still having a few teething…
Tensorflow Estimators handle much of the boilerplate of training a neural network like saving checkpoints and summaries, running the training loop, periodically evaluating on the validation set, and so on. If you’re already using Estimators, it’s easy to move from training on a single GPU to synchronous multi-GPU training with only three new lines of code. First, wrap your model_fn with…
In the last post we built a static C++ Tensorflow library on Windows. Here we’ll write a small Tensorflow program in Visual Studio independent from the Tensorflow repository and link to the Tensorflow library. The tutorials I have been able to find about writing a new Tensorflow C++ program all seem to require that the new C++ project live within the Tensorflow repository itself. But this is not…
Tensorflow was built first and foremost as a Python API in a Unix-like environment. But there are some projects where using Windows and C++ is unavoidable. This post will show how to write a simple C++ program in Visual Studio 2015 that links to Tensorflow. This build was done on Windows 8 without GPU support. Building Tensorflow in Visual Studio with CMake The first step is to build Tensorflow…
Suppose we have a pair of stars orbiting each other in a binary system. Each star has a Roche lobe , beyond which matter is not gravitationally bound to the star. What happens if one of the stars grows larger than its Roche lobe? Since this material is no longer bound to the original star it is clear that mass will be lost from the original star and it is likely that the secondary will accrete…
I discussed in a previous article one of the most important discoveries in all of astronomy — the discovery of dark energy. Key to this is an accurate measurement of the distances and recession velocities of distant galaxies. For this, cosmologists have used Type Ia supernovae. Type Ia supernovae are ideal for cosmological measurements. They are extremely bright, so they can be viewed at very…
There are many ways to give bad talks, but one guaranteed way to make your talk worse is to add a “Thank You” slide at the end. You know the one — the big slide that says nothing but “Thank you” that you leave up after you’ve finished your talk. Don’t do it. There’s no point. It is a complete waste of a slide. You have just spent 15, 20, 30 minutes, maybe even an hour, describing to your audience…
Although there is strong evidence that nearly all stars are born in open clusters, it is clear that open clusters do exist for a very long time. After all, most stars are not seen in open clusters so they must spend a small fraction of their lives in them. It’s also possible to measure the ages of open clusters directly by measuring the masses of stars just starting to evolve off the main sequence…
Among the most important developments in cosmology was the discovery at the tail end of the twentieth century of the accelerating expansion of the universe. This result implied that around 70% of the energy density of the universe was composed of some exotic substance given the placeholder name “dark energy.” This discovery was announced in two papers that were published nearly simultaneously:…
In our look at Jeans’s 1919 paper in this post we derived the eccentricity distribution of a population of thermalized binaries. Recall that we assumed that the binaries obeyed equipartition of energy and then showed that the eccentricity distribution must follow the law \[f(e) = 2e \: de.\] After deriving this law, Jeans compared this to the eccentricity distribution of known binaries and did not…
Scott Tremaine’s contribution to a retrospective series on the work of the French dynamicist Michel Hénon inspired me to do a review of an interesting problem in dynamics. The puzzle Suppose we are in a spacecraft orbiting the Earth in a circular orbit. The spacecraft is rotating such that the same side of the spacecraft always points toward the Earth. That is to say its rotation period is equal…
One of the most beautiful results in dynamics is the thermal eccentricity distribution. Suppose we have a population of binary stars which is, in some sense, “thermalized.” That is, these binaries have all interacted with each other and exchanged energy many times and have reached statistical equilibrium. We would like to know what these binaries look like. In particular we would like to know the…