How Many Random Neurons Buy a Trained One?

This page did not load. You can still read it on the original site — the toolbar below keeps your place in the directory.

Freeze a bank of Yat units at random initialization, noise prototypes that will never move, and train nothing but the linear readout on top. This should not work, and the reason it does is one of the oldest facts in the kernel literature wearing a new activation: a frozen random layer is a Monte Carlo estimate of a kernel, one neuron per sample, converging at the square-root rate. This post…

Read on tahabouhsine.com ↗