While applying logit attribution analysis to transformer outputs, I have noticed that in many cases the generated token can be attributed to the output of a single neuron. One way to analyze neurons activations is to collect activations from a dataset of text snippets, like in “Exploring Llama-3-8B MLP Neurons” [1]. This does show that some of the neurons are strongly activated by a specific token…
It has not been a very good year. I personally can’t complain about anything. But the war in Ukraine continues, and the war in Israel has started. Yes, a single person can't do much about it, but at the very least we can: vote for sane and competent people be well informed, not fall for propaganda, fight disinformation donate to the right causes Let’s start the year with good and charitable deeds.…
I have described the problem of analogy-making interpretability in the previous post : given the examples of transformed sequences of numbers, what’s the mechanism behind figuring this transformation out, and applying it correctly to the incomplete (test) sequence? prompt: "0 1 2 to 2 1 0, 1 2 3 to 3 2 1, 4 5 6 to ", output: “6 5 4” It was easy to check on which layer the correct answer appears,…
I have been using llama.cpp for learning about transformers and experimenting with LLM visualizations and mechanistic interpretability . Initially I’ve just inserted bunch of code all over the ggml compute framework. That code was not thread safe, used lots of hardcoded values, and communicated using global variables. Now I have refactored it a bit, moving most of the code into hooks/callbacks.…
Can LLM make analogies? Yes, according to tests done by Melanie Mitchell a few years back, GPT-3 is quite decent at “Copycat” letter-string analogy-making problems. Copycat was invented by Douglas Hofstadter in the 80s, to be a very simple “microworld”, that would capture some key aspects of human analogy reasoning. An example of a Copycat problem: ”If the string abc changes to the string abd ,…
I’m starting to learn about mechanistic interpretability , and I’m seeing lots of great visualizations of transformer internals, but somehow I’ve never seen the whole large model’s internal state shown at once, on one image. So I made this visualization, for Llama-2-7B. Attention matrices are on the left, in 32 rows for 32 blocks, top to bottom. To the right, there are 64 rows: residual stream…
The next iteration of my experiments with robot simulation is a demo of a simulator for Petoi Bittle , a neat palm-sized robot dog. Its size and price make it great for democratizing robotics research and education. Thanks to Rongzhong Li of Petoi for inspiring this demo and providing the data for creating the robot model. The new robot model is used with the quadruped simulator which Is based on…
There are many apps for blocking distracting websites: freedom.to, leechblock , selfcontrol , coldturkey , just to name a few. They are useful for maintaining focus, avoiding procrastination, and curbing addictive web surfing. They work well for blocking a list of a few distracting websites. For me, this is not enough, because I’m spending a large portion of my time on a large number of websites,…
disclaimer: for now, this is just a proof of concept prototype, it is not ready to be used as a sole parental control application Parental control apps usually rely on blacklists, which is not flexible enough for sites that are a mixed bag of useful and disturbing content, like Wikipedia. Some parental control apps (and Google’s SafeSearch) would use AI to filter content that’s “nudity, sexually…
Previously, I have implemented a controller for a quadruped robot for stepping in place . It is a good starting point for implementing a controller for walking. targets for placing feet The only difference between stepping in place and walking — is selecting target points for lifting and planting feet. An idea that is frequently used in procedural animation is to set this target points relative to…
My previous post was about implementing a simulator for a quadruped robot. I got it to work, including inverse kinematics for controlling robot’s feet and torso poses. Next step is making the simulated model move and walk. How hard can it be? procedural animation The simplest way to generate walking motion is very similar to procedural animation . It imitates motion by setting poses of some of the…
I’m continuing to learn about robot simulation , and the next step is simulating a real robot, rather than a simplified platform. I’ve chosen a quadruped (Unitree A1) for the next demo. First step is to load robot description from the official repository , using urdf-loaders library . Next, robot’s links and joints can be created in rapier.js physics engine for simulation and added to a three.js…
While learning more about embodied AI and robot simulation i’m building a simple simulator, based on Three.js and Rapier physics engine . The robot model is a mobile platform with a 6DOF manipulator and a 2-finger gripper. Each part of the robot is a three.js mesh, kept in sync with the Rapier rigid body. Parts are connected into a kinematic chain with Rapier joints. Arm pose can be set by…