Neural Networks ExplAIned, pt. 1

In these 90 seconds you can learn a large portion of how neural networks actually work. It’s a very simplified picture, but let me show you the actual math.

Every neural network has three kinds of layers: an input layer, one or more hidden layers, and an output layer. Each layer is made of neurons (the dots), connected by edges. Every edge carries its own weight, a number.

Your input word or text input goes through a transformation into number form to enter the AI model - check out my “AI is math” video. That transformation is called an embedding, and I have an explAIned video on those as well.

In real models, an embedding can be hundreds or even thousands of numbers long; here I will use just two.

Say the input is 5 and 2.

Now give each edge a weight. To reach one neuron in the hidden layer, two edges feed into it: one with weight 10, one with weight 2. The neuron’s value is 5 x 10 + 2 x 2, which is 50 + 4, so 54 (remember PEMDAS, my friends!!).

The neuron beside it has its own edges, say weights 20 and 4. Its value is 5 x 20 + 2 times 4, which is 100 + 8, so 108.

Multiply each input by the weight on its edge, add the results, and pass the number forward. Do it again for the next layer, and the next. Basically every generative AI model you use runs on this calculation.

Btw - we don’t go over them in this first video, but we’re missing the *nonlinear* functions in this demonstration. I can make another video on those & why they are crucial to the neural networks. Also, the output layer works a little differently from the hidden layers, and I will cover that in Part 2. I can also break down how these weights are actually learned, through backpropagation. If that is of interest to you, or if you like this format, or if I can do better somehow, please let me know in the comments!

Welcome!! I’m Harper. I’ve spent a decade building AI as a Stanford computer scientist, and now I hope to share the intuition I’ve developed with you. For a complete AI basics guide, click here.