The Kernel Between the Roles

This page did not load. You can still read it on the original site — the toolbar below keeps your place in the directory.

The QK post in this series ended on a construction it refused to build: keep the query and key roles, but replace the bilinear-then-exponential score with a genuine Mercer kernel between them. This post builds it, with the kernel's full form: a per-head learned bias inside the square, the term the universality theorem requires, and a per-head learned softening. Because the kernel is nonnegative by…

Read on tahabouhsine.com ↗