Project Repository: https://github.com/workofart/harness-training So I recently wanted to see whether an AI agent could self-improve a harness to solve terminal bench tasks. To align on the definitions, “harness” means the system (e.g. Claude Code, Codex, ChatGPT web interface etc…) wrapping around the model (e.g. GPT-5.5, Claude Opus 4.7 etc…) that...
Project Repository: https://github.com/workofart/harness-experiment Update (July 2026): The follow-up, Training Self-Improving AI Agent Harnesses, turns these lessons into a deterministic training framework that allows for model-agnostic and task-environment-agnostic capability improvements. So I recently wanted to see whether an AI agent could self-improve a harness to solve terminal bench tasks.…
1. The Familiar Training Step 2. Why Call loss.backward() At All 3. Why An Update Actually Helps 4. Values Remember Where They Came From 5. What One Operation Does During Backward 6. How loss.backward() Walks the Whole Graph 7. So We Have All the Gradients in the Graph. What’s Next?...
Motivation and Goals Abstraction layers of Machine Learning Libraries Comparing this project to PyTorch for the same functionality Efficiency vs Learning Technical Design Tensor Class Tensor-level Operations Function class Computational Graph Tensor.backward() NN Module Functional Module Optimizer Additional Thoughts Project link: https://github.com/workofart/ml-by-hand I recently started working…
Video games were very attractive to me when I was kid. This is because the possibilities in the virtual world is vastly greater than the real world, as least during my teenage years where most of my life was filled with school work that can’t be related to in the...
Code 1. Goal 2. Environment State Action Reward Termination Condition 3. Approach 3.1 Reward Function Design 3.2 Learning Algorithm 3.3 Hyperparameters 4. Experiment & Findings 4.1 Training Evaluation 4.2 Optimal Policy Discussion Experiment - Play two copies of the same agent against each other, by following the trained policy Experiment...
Code 1. Goal 2. Environment State: Action: Reward: Terminal Condition: Solved Condition: 3. Approach 3.1 Algorithm Comparison 3.2 Problem Parameterization 3.3 Policy Key consideration: 3.4 Neural Network Architecture 3.5 Hyperparameters 4. Experiment & Findings 4.1 Performance 4.2 Training Duration 4.3 Neural Network Complexity 4.4 Discount Rate 5. Next Steps Code...
Code 1. Goal 2. Environment State Actions Reward Starting State Episode Termination Solved Condition 3. Approach 3.1 Discretization 3.2 Exploration vs Exploitation 3.3 Gaussian Policy 3.4 Linear Value Function 3.5 Training 3.6 Hyperparameters 4. Experiment & Findings 5. Next Steps Code Here 1. Goal The problem setting is to solve...
Code 1. Goal 2. Environment State Actions Reward Starting State Episode Termination Solved Requirements 3. Approach 3.1 Discretization 3.2 Exploration vs Exploitation 3.3 Training 3.4 Hyperparameters 4. Experiment & Findings 4.1 Introducing baseline to reduce variance 4.2 Discrete vs Continuous Actions 4.3 Performance 5. Next Steps Code Here 1. Goal...
1. Perception 1.1 Current Player Position Green Circle Player Name 1.2 Stars (Reward) Player Stars Team Stars 2. Planning 2.1 Agent 2.2 Brain 3. Error Analysis 4. Challenges & Future Steps First and foremost, I must say, perception is harder than planning. This part, I will be attempting to apply...
1. Motivation 2. Goals 3. Starting Point 3. Related Work 4. Project Focus/Scope 5. Showcase 6. Supervised Learning 6.1 Creating training data 6.2 Features 6.2.1 Raw Pixels as Features 6.2.2 Using MobileNet as the feature extractor 6.3 Action Determination 6.3.1 AlexNet (Convolutional Neural Network) 6.3.2 Long short-term memory 7. Challenges...
Policy Gradient Problem Setting Agent Environment Reward Technical Details Policy Network Design Training Key Considerations Key Challenges Challenge 1 Challenge 2 Results Next Steps This is the first post that’s part of the series for teaching an agent to trade. I will evaluate different reinforcement learning (RL) approaches and share...
Previous Knowledge Required Goals Showcase Training Set Test Set Input Data Neural Network Architecture (All 3 Versions) Version 1 Version 2 Version 3 Previous Knowledge Required Understand what is a neural network (NN) and how it works conceptually. Python Basic understanding of what derivatives/gradients are Goals In this tutorial, I...
The A3C method in Reinforcement Learning (RL) combines both a critic’s value function (how good a state is) and an actor’s policy (a set of action probability for a given state). I promise this explanation doesn’t not contain greek letters or calculus. It only contains English alphabets and subtraction in...
I’ve been bugged by the native state management system in React that I finally had to take a stab at Redux. Here are some notes I took along the way to understand what Redux is and why we need it. First off, why do we need Redux when we already...
This post is adapted from my answer on Reddit. I think there are various career paths that fall under “Data Science/Machine Learning”. This doesn’t mean there are 4 roles, it’s just how I see this industry; nevertheless, the more skills you possess, the better. Below are my personal thoughts based...
It’s been a while since my last post. There’s just too much going on in my life that I didn’t get time to write down. Anyways, here goes nothing. The past 1.5 year has been really, really rough without any exaggeration. After switching majors, I’ve learnt a lot of new...
When choosing a career path, the most important thing is to do the research and plan ahead. When we are talking about doing research for career cruising, we need to understand its industry. Its past, present, future. You don’t just wanna look at the present condition, because who knows what...
How I ever wonder, how does so many people have so many different answers to a simple “1+1” problem. To put the question into perspective, we have all heard of the “Standard Deviation” term from basic statistics, if not, this measures the variability of the sample or population data points....
It has been approximately 24 days since classes have started and I have gotten used to the work and life style in business school, the case method and socializing all the way. Most of the courses here are taught in the form of case studies, where the learning, supposedly comes...