I'm interested in answers to the question: how do we make it as easy as possible for users of a package (PyPI package, conda package, npm package, etc) to use the skills for that package? This post explains my best idea so far: an environment variable that specifies paths to …
With access to AI tools I can create, work on and contribute to (software) projects that I have no business creating, working on or contributing to. This is fantastic! It also creates a dilemma. In the days before AI tools, adding a feature to a browser extension would be something …
TL;DR: If you have GPU code in your project, setup a GitHub hosted GPU runner today. It is fairly quick to do and will free you from having to run tests manually. Writing automated tests for your code base and certainly for the more complex parts of it has …
What is a TIL? Today I learnt how to encode data as multipart/form-data in Python. This is useful if you want to construct the body of a POST request yourself. Once you find the answer it is surprisingly simple: encode_multipart_formdata() in urllib3 . fields = { "foo" : "bar" , "somefile" : ( "somefile.txt" , "contents …
We humans communicate a lot. Increasingly so in writing. This post is exists because I am trying to find books, guides, courses, lessons and expert advice on how to get better at communicating, in writing. I am looking for ways to improve my (and others) communication in the context of …
What is a TIL? Today I learnt how to make a Python module from which you can import anything. Yuvi asked if you can write a Python module that will allow anything to be imported from it. A bit like from idealmock import whatever, does_not_exist . I thought "how hard can …
What is a TIL? Today I learnt how to configure traefik as reverse proxy for a service running on "bare metal". I have a webserver listening on http://localhost:8080 and would like it to be reachable from the internet. With HTTPS. I setup traefik v2.4.8 by downloading …
I started posting "TILs" and you might be wondering what they are and why I am posting them. TIL stands for Today I Learnt. The posts are short summaries of something I learnt. I learnt about TILs from Simon Willison who picked up the idea from Josh Branchaud . Josh's description …
What is a TIL? Today I learnt how to limit clamd's memory consumption. The clamd process uses a lot of memory (about 1GB) because it loads the complete database of virus definitions into memory. This allows it to be super fast. When deploying this to a kubernetes cluster we want …
What is a TIL? Today I learnt how to use PDF.js to add a PDF preview to a HTML form without having to upload the file to a server. I needed this to build a multi-step form that shows a preview before we have reached the last step in …
What is a TIL? Hotwire's Turbo Streams can use WebSockets to stream updates. How to do this is not well documented in the handbook. This post contains my notes on building a prototype to figure out how to do it. I used Tornado as webserver but the technique is completely …
This post introduces a Jupyter notebook extension that I created to help you save your notebooks as PDFs. It is available as notebook-as-pdf and the source is on betatim/notebook-as-pdf . Try it without having to install anything: This extension does a few things differently, it will: produce a PDF with …
This post is about a tool to create a web of knowledge for your team. A tool to help you to think things through together. A tool that helps keep everyone in the loop. This tool is your issue tracker. What features do you look for in an issue tracker …
Eventually you will get stuck when trying to do something. This post is about how to get help with getting unstuck. People love helping others. Below some tips on how to, in my experience, improve your chances of getting a good answer. The answers you get will depend very much …
In this post a computer will learn basic algebra, from scratch! Not excited? Keep reading. This is an introduction to basic sequence-to-sequence learning using a Long short term memory (LSTM) module. Sequences of varying length are everywhere around us. Yet most machine-learning problems are presented as "here are five features …
Spend an afternoon understanding these seemingly magical algorithms. This post covers a few probabilistic data structures in python. Each of them deals with counting things in one form or another (how many unique items, is an item present, how many of each have we seen, etc). The idea behind these …
This post collects code snippets I created while learning about genome assembly. I understand complicated things through code. I build simulations or little tools and they help me immensely to understand how something works. Probably because it uses vocabulary I am familiar with (python!) instead of domain specific jargon. For …
This is an interactive blog post , you can modify and run the code directly from your browser. To see any of the output you have to run each of the cells. When building an ensemble of trees (a Random Forest or via gradient boosting) one question keeps coming up: how …
This is the story of how I put my idea for an openscienceprize entry on the internet, nearly gave up on it, and then met a ton of incredible people. It all ended with: everpub - reusable research, 21st century style. Everpub is a prototype to make reproducibility a first-class citizen …
This is an interactive blog post , you can modify and run the code directly from your browser. To see any of the output you have to run each of the cells. In particle physics applications (like the flavour of physics competition on kaggle) we often optimise the decision threshold of …
Update (13 Jan 2016) : Added links to the DIY section. A few days ago I started making my blog posts interactive . It was cool, but required you to surf to a different page for the interactive experience, while the original post was still non-interactive. Alex pointed out that really you …
This post is out of date. Interactivity on a different page was cool, but Alex was "meh, same page!", so now there are really interactive posts ! Interactivity on the same page as the post. From today on all my blog posts based on jupyter notebooks will be interactive!! The combined …
Last week I attended the second half of NIPS 2015 . It was great! I am an experimental particle physicist, not a professional machine-learning guy. This was the first time I went to NIPS and it was very different from big conferences I have experienced in particle physics. This post is …
This post explains how you can setup jupyterhub to serve notebooks from a computer owned by your visitors. Sounds confusing? Bare with me. In a traditional setup the person hosting the jupyterhub instance has to provide the compute power for all users. This means you have to have enough resources …
Take interesting code for a spin, in your browser! Easily spin up the code from a git repository in a custom docker container, in order to quickly test out and play with something you are curious about. Like nbviewer , but executable. A super easy way to run a notebook (with …
Fitting a single classifier does not take long, fitting hundreds takes a while. To find the best hyperparameters you need to fit a lot of classifiers. What to do? This post explores the inner workings of an algorithm you can use to reduce the number of hyperparameter sets you need …
Gaussian process regression is one of the techniques used by Bayesian optimisation to find the best hyperparameters of a machine learning algorithms and other optimisation problems with very expensive cost functions. Rasmussen and Williams have written a very good introduction to Gaussian processes in general. This blog post is about …
Yes, this is about machine learning and not some weird fetish. This post is totally safe for work, promise. With that out of the way: What is bumping? Bumping is a simple algorithm that can help your classifier escape from a local minimum. Huh? Read on, after a few imports …
Powerful features (or variables) are often expensive to compute. Maybe they require a lot of CPU power to compute or require human interaction. As a result the size of the datasets for which the features are evaluated are smaller than they should be. This post explores a possible way around …
Every time you want to use machine learning to solve a challenge you face a problem: Choosing the input features, classifier type, ... and then estimating your new model's performance. Simultaneously it is one of the most important steps as the difference in performance can be quite different for different models …
Particle physics is full of machine learning. It powered the discovery of the Higgs boson and it is used to decide which data an experiment should record. As well as in many other analyses. This is a follow up to my previous post scikit-learn for TMVA users which covers how …
Particle physics is full of machine learning. It powered the discovery of the Higgs boson and it is used to decide which data an experiment should record. As well as in many other analyses. There is only one drawback: most of it is based on one library: TMVA . It was …
A large number of scientific articles are published on the arXiv every day. Most of them are not that interesting. This post is about how you can improve your chances of picking an article that is interesting by a factor of about three! The dataset was collected for my previous …
Inspired by cubism and various projects using genetic algorithms to paint the Mona Lisa here a method for teaching your computer to be an artist! I bring you the artificial artist! The idea is to use regression to "learn" what an image looks like and then draw the learned image …