Recent content on Moss' blog
Back pressure for agents 
 You might notice a pattern in the most successful applications of agents over the last year. Projects that are able to
setup structure around the agent itself, to provide it with automated feedback on quality and correctness, have been able
to push them to work on longer horizon tasks. 
 This back pressure helps the agent identify mistakes as it…
Introduction 
 Recently I’ve been motivated to investigate generative models, of which the
most popular is currently GANs. The best way I could think of learning about
GANs in deeper detail (since I’m interested in tweaking them later in some
applied cases) is to implement them myself, and solve the bugs and issues that
arise in practice myself. 
 In this…
Introduction 
 Orderbooks are tables that show the immediate and public supply and demand of
instruments that trade on financial exchanges. They collate orders that specify
the direction, price and volume at which a party wants to trade an instrument. 
 I have collected several months of orderbook snapshots from the perpetual XBTUSD
contract on the cryptocurrency derivatives…
Introduction 
 In this article we’ll review Q-learning and walk through a subtle improvement
that leads to Double Q-learning and better policies. We’ll then look at this in
action, and compare the two methods on a toy problem. 
 Reinforcement learning refresher 
 In this article I assume a familiarity with reinforcement learning and the
standard Q-learning…
Introduction 
 Recently I came across the need for an easy UI to display information coming
from a websocket. A niche option but one I’ve had a pleasant experience with
previously is Elm, a functional language designed specifically for web
applications. 
 Previously, the standard approach for connecting to a websocket was through the
 elm/websocket package, has been…
Introduction 
 A few months ago I read this paper from DeepMind that addressed a simple choice
of architecture to encourage sensible weights in neural networks when solving
problems that at the core are simple arithmetic. Despite the continued hype
surrounding neural networks and deep learning in general, some simple problems
like this are difficult to generalise past the…
Introduction 
 In this article we will explain the minimax algorithm. We’ll cover game trees, the minimax algorithm itself and a simple implementation in Python. We’ll also review some popular extensions that speed up or improve upon the actions taken by minimax. 
 Game trees 
 For games with perfect information, we can model the entire play-space using a directed graph…
Introduction 
 In this post we’ll go over auto-regressive time series. What they are, what they
look like and some properties they exhibit. Throughout the post we’ll use small
snippets of R to plot processes and visualisations. 
 What are autoregressive time series? 
 An auto-regressive time series is a stochastic process in which future values
are modelled by a…
Introduction 
 One of the most important steps in developing a model for machine learning is tuning hyperparameters to ensure it generalises to unseen data. A model that fits the training set well but performs poorly on anything else is useless, so care should be taken in ensuring that in-sample performance characteristics of a model are representative of real world performance also. The most…
Introduction 
 In this post we’ll look at the problem of keyword matching including a number of approaches, applications and the Aho-Corasick algorithm. 
 Statement of problem 
 To begin, let’s define the keyword searching problem. 
 Given a list of strings \(K = [s_1, \ldots, s_n]\) (called keywords ) and a (usually) much longer string \(C\) (called the corpus ) count…