Studying Artificial Intelligence, from backbone to application.
To those unfamiliar with the topic, math equations can be confusing. If not explained properly, it is difficult to understand the meaning behind the equation without scratching one’s head a few times. Some nice way to make such equations more “digestible” is to color different components, like this explanation of Fourier Transform : Each component is color-coded, allowing the reader to easily…
For the last few days, I have been re-organizing my personal collections. One of my collections is a list of websites, saved as bookmarks on Chrome. I have about 1700 bookmarks saved. Some of them are outdated and may no longer be of use, but I found many of them are still helpful. Although Chrome bookmarks does its job, since I was planning to re-organize bookmarks, I decided to take a look at…
Dear readers, Last week, I received a couple of emails indicating that one issue every two weeks would work best for them. A big thank you to those who sent feedback! I am experimenting with the bi-weekly issues, and I hope to make a decision in January. If you have any suggestions or feedback, please email me ! In this issue, we look at using human demonstrations to help robots learn and…
Dear readers, Yesterday, DeepMind’s AlphaFold 2 made a major breakthrough for the protein folding problem . It is an exciting work that shows just how much impact AI can have in the real world. Reinforcement learning also has great potential to help make such scientific discoveries, and I hope to see such results in the next few years. You may have noticed that RL Weekly went on a long hiatus.…
Prerequisites Human Level Control Through Deep Reinforcement Learning [Paper] Rainbow: Combining Improvements in Deep Reinforcement Learning [ArXiv] Download Original Paper Annotated Paper 1-2 Introduction and Background Deep Q-Network (DQN) (Mnih et al., 2015) is one of the most popular deep reinforcement learning (RL) algorithms and has been augmented by various improvements since its…
Introduction Swift for TensorFlow (S4TF) is Google’s product to bring machine learning to Swift. Python has been the primary language for prototyping and developing machine learning models, with the two most popular machine learning libraries (TensorFlow and PyTorch) both being Python libraries. Python has a great benefit of being easy to use. However, native Python is very slow and relies on…
Introduction Proximal Policy Optimization (PPO) is a reinforcement learning algorithm published by OpenAI (Schulman et al.) in 2017. It is a policy gradient algorithm that succeeded Trust Region Policy Optimization (TRPO) with the benefit of being easier to implement without a drop in performance. It has become one of the most commonly used baselines for new reinforcement learning tasks, and its…
Ngrok exposes local servers behind NATs and firewalls to the public internet over secure tunnels. Using ngrok, we can expose our laptop or server to be connected via SSH from the public internet. Prerequisites To use ngrok , you must have an account. To register a new account, visit the signup page . You can also use your Google or GitHub account for easy signup. You should have sudo privilege in…
Python is one of the most popular languages, known for being simple and easy to read. It is general-purposed and has been used in machine learning (TensorFlow, PyTorch), server (Flask, Django), web scraping (BeautifulSoup4), and more. PyPI (Python Package Index) is a repository of software for Python. Nearly all Python packages are distributed through PyPI, allowing for the package to be installed…
Introduction Deep Q-Network (DQN) is a reinforcement learning algorithm published in Nature in 2015. It was the first deep reinforcement learning algorithm to use the Atari 2600 environments from Arcade Learning Environment, which became the standard benchmark for reinforcement learning agents. DQN achieved superhuman performance in many of the games, which led to its popularity. Swift for…