Break's Forge World
This post talks about the policy gradient theorem, which summarizes my attempt to derive it in an intuive approach. This is based on the derivation from Richard Sutton's RL book, with a focus on its application in deep reinforcement learning. This assumes basic knowledge about Markov decision process (MDP), reinforcement learning (RL) and neural networks. You should be familar with the…
Variational Autoencoder, also known as VAE, is an elegant algorithm in machine learning. This post summarizes my attempt to teach the math behind VAE in an intuitive way. Maximum Likelihood Estimation (MLE) A common problem (arguably the central problem) in machine learning is learning the underlying distribution of a dataset $X_{\text{Data}}$. This dataset contains $n$ samples: $$ X_{\text{Data}}…
Motivation To use rsample or not to use rsample , that is a question. If you ever come across the above when implementing a deep learning algorithm, for example, a policy gradient algorithm for reinforcement learning, this post is about that. Disclaimer : Please note that I am more interested in making the math intuitive rather than strict here. The concepts Transformation of Random Variables Our…
Why am I writing this? As I worked on upgrading my home network with a NixOS router, I found myself once again needing to update the VLAN configuration on my Aruba Instant On 1930 PoE switch. However, I felt hesitant to do so due to my previous struggles in grasping the concept of VLAN despite reading multiple online articles. Fortunately, my friend Hao recommended an informative post on the topic…
For quite some time, I have been leading software engineering teams. This is not a piece of advice for current or prospective tech leads. In fact, I believe that most of you are better at managing a group than I am. Nonethelss, I wanted to convey what I have learned from this incredible journey in the hopes of inspiring some of the readers. There are no rules that you can follow to reach the…
Background As promised, in this post we will be deriving the multi-variate version of Kalman Filter. It will be a bit more math intensive because we are focusing on derivation , but similar to the previous post I will try my best to make the equations intuitive and easily understandable. Bayes Filter Kalman filter is actually a special form of Bayes filter. This means that Bayes filter is actually…
Motivation State estimation has many applications in general robotics, for example autonomous driving localization and environment prediction. Kalman filter is a classical yet powerful algorithm that tackles such problem beautifully. Although there are already many articles, textbooks and papers on how to derive the algorithm, I found most of them too heavy on the theoretical side and might be…
Important Update 2020.05.24 After upgrading to 20.03 version of NixOS, the docker container starts to use the container's actual name instead of its systemd service's name to address the container. This means that to specify the database container from the filerun web server's container, you need to change the value of FR_DB_HOST from docker-filerun-mariadb.service to…