Recent content on br0x
TL;DR: Part 2 gave the owl a confidence governor: when timing is noisy, stop learning. But the owl still can’t tell a catch from a miss. For that it needs a reward signal — and that signal isn’t what you think. Dopamine doesn’t flood the brain. It barely makes it past the neighbors. If we’re building a spiking neural network that spans servers, the reward signal has to work…
TL;DR: Part 2 gave the owl a plasticity governor: when timing is noisy, stop learning. But timing alone can’t tell a catch from a miss. The missing piece is reward—and in biology reward isn’t a global broadcast. Dopamine is fast because it’s local. If we’re going to build a spiking neural network that spans servers, reward has to behave the same way: local propagation, spatial decay, and a scope…
TL;DR: Making reward local prevents interference, but it also breaks deep credit assignment: reward can’t reach early layers before eligibility traces evaporate. The fix doesn’t need backprop or a coordinator. It needs a relay. When a synapse consolidates, it emits a short-range retrograde “credit happened downstream” signal. Chain those signals and credit can walk backward…
TL;DR: Noisy timing makes STDP thrash. In Part 1 we added uncertainty intervals. This post takes it to the next level and uses uncertainty as a signal to determine how much each neuron is willing to learn. When things are noisy, neurons freeze up. When the signal is good, they keep learning. No global tuning required. The Barn Owl Problem A barn owl hunts in darkness. It finds a mouse by comparing…
Extending it to a Time Coded Ripple Carry Adder Let’s stretch this into a time coded ripple carry adder . In a clocked adder the entire pipeline dances to a drum that must be slow enough to accommodate the worst‑case carry marching all the way through. Most of the time reality is nicer than the worst case, but the clock never rewards you for that. In our timing‑coded adder the next stage…
The amber liquid swirls in the glass. If you read the previous essay, you know the pitch: a World Brain built from spiking neurons, distributed across machines, fed by sensors, learning by experience. You also know the immediate objection: If it’s open, people will try to poison it. Every communication network eventually meets its adversary. The telegraph, the internet, smoke signals—someone…
In the last post on neural logic gates , I treated a spike like a yes or no question. If a neuron crossed the threshold in time, we called that a one. If it didn’t, we called that a zero. Linear separability does the heavy lifting, we put the threshold here get an AND, move it there and get an OR. We add a scheduled nudge and get NAND and NOR. XOR stood off to the side like Sonic tapping his…
As you may have read , I’ve been thinking about some crazy ideas related to neurons lately. So I thought I should dig in a bit more and get comfortable with the basic behavior of how they function and process information. As someone with a background in computer science, mapping neurons to logic gates is a natural way to get started. So, my goal with this post is to simply get a feel for how…
iuSTDP is designed to handle order uncertainty. However, if you update weights before being sure you’ve seen all relevant spikes for that period, late arrivals can invalidate learning. The next problem is order completeness. We avoid global barriers and heavy, Kafka-like checkpoints. Global synchronization becomes a bottleneck at scale; instead, we use local coordination. Like biological circuits,…
When you’re thinking about how to build a world brain , you end up with a huge bag of shifting unsolved problems. One of the big ones is how to deal with pesky latency between groups of neurons. Neurons are very sensitive to when they receive spikes (boolean signals) from other neurons. If a spike arrives too late or too early, it may result in incorrect signal output by triggering or…
The amber liquid swirls in the glass, rye catches the light like neurons firing. In 1938, Wells proposed something he called the “World Brain” , a permanent encyclopedia that would grow and evolve, connecting all human knowledge into a living whole. “This is no remote dream,” he wrote, sitting in his study without the benefit of either computers or good bourbon. We can…
There’s an announement for a new stream processing system tonight. Details below are copied from the meetup page and LinkedIn Event page : Northguard is a log storage system developed at LinkedIn with a focus on scalability and operability. To achieve high scalability, Northguard shards its data and metadata, keeps minimal global state, and adopts a decentralized group membership protocol.…
Introduction Although Terraform seems to be the current front-runner in the infrastructure-as-code department, I've recently run across Pulumi as an excellent alternative. In this post, I'd like to run through how to integrate Pulumi's Python with Poetry (a PEP-517 compatible packaging manager). Once you have everything integrated together, you can benefit from the code quality tools available in…
Here’s a picture that summarizes what I’ve been obsessing over for the past 7 months: As you can see, I’ve learned to take pictures with better lighting (and lost a bit of weight). [10:05] Nice Work!!! [10:05] hey you kinda look like Wash from firefly [10:05] stay away from reavers [10:05] and space ships What I’ve been doing is following a 1400-1500 kcal a day diet (about…
Introduction Apache Kafka is a distributed and decentralized publish and describe system. To reduce contention, Kafka uses the concept of partitions to allow minimal coordination between the production and consumption of messages. However, due to the nature of partition growth, an unbalanced state may be created during normal use. This can lead to performance loss and underutilization. In…
I've recently done some work on a very rough prototype of a Mesos scheduler for Samza . While going through the paces to get this working, I've noticed a few similarities between Mesos and Yarn that might be worth talking about. The high-level architecture of both Yarn and Mesos are basically the same. Both resource managers have a master-slave architecture (both support a leader election via…
This is a repost of an article I wrote for LinkedIn's engineering blog. To see the original, please visit here . At LinkedIn, we use a log-centric system called Apache Kafka to move tons of data around. If you're not familiar with Kafka, you can think of it as a publish-subscribe system that models each message as a log entry. However, it's difficult for a developer to spend their time thinking…
Note: You may want to read the the Paxos paper , the Raft paper , the Chubby paper , the Spanner paper , and the F1 SQL layer paper before trying to read this -- it won't make sense unless you're familiar with the concept of multiple consistency domains. The following is a simple algorithm for bootstrapping a Raft consensus quorum in a multiple consistency domain system (which utilizes a location…
Bitcoin? How does that work. They’re like coins, but they’re made of bits! Many people have rambled on how bitcoins may or may not change the world. I’m not here to talk about that (not much anyway). I’m going to talk about the details of how the math works. Mmmm, sweet, sweet details. We’re going for a ride into the world of cryptography. Everyone knows that Satoshi…
A side project I’ve been working on for some time now is a decentralized-distributed file copy tool in the spirit of the typical cp command you can find on just about any unix style system out there. When I start a project like this, I tend to concentrate on what the most difficult problem is – once I solve the most difficult problem, I can then easily finish designing the rest of the…
I haven’t been able to find much information on how space filling curves, like Hilbert curves, are used in network interconnection topology in supercomputers. So, this post is my attempt at putting some of the information in one place. We’ll start off with an introduction to gray encoding and how it relates to Hilbert curves. So, gray codes are the concept of the day here. Gray codes…
Thicket: What Happens When You Let a Spiking Network Grow TL;DR We built a minimal spiking neural network from scratch (NumPy + NetworkX, no frameworks) and ran 13 experiments testing whether biologically-motivated learning rules – reward-modulated STDP, inhibitory homeostasis, and a developmental growth controller – can learn, avoid catastrophic forgetting, and recover from damage.…
Timing‑Coded Half Adder — Single Token (Arrival‑Controlled) Timing‑Coded Half Adder (Single Token) Adjust A and B arrival times or disable either input. The earliest enabled arrival becomes t₀ . Inputs inside the green window ( W c ) are early (bit 1); otherwise they’re late (bit 0). Pattern: both early → Carry early; exactly one early → Sum early; none early → both late at t₀ + Δ . Everything…
Timing‑Coded Ripple‑Carry Adder — Clean Bars, Aligned Inputs 0]_abs = Cout[i-1]_abs --> Timing‑Coded Ripple‑Carry Adder Bars (not hairlines) for every event remove pixel artifacts. Rows share one absolute time axis. Pink Cout in one row is gray Cin in the next row at the exact same time.