2022-08-12

[public] 35.0K views, 56.5K likes, dislikes audio only

Sebastian Lague

Exploring how neural networks learn by programming one from scratch in C#, and then attempting to teach it to recognize various doodles and images.

Source code: https://github.com/SebLague/Neural-Network-Experiments

Demo: https://sebastian.itch.io/neural-network-experiment

If you'd like to support me in creating more videos (and get early access to new stuff), you can join my patreon here: https://www.patreon.com/SebastianLague

The data is from:

Digits http://yann.lecun.com/exdb/mnist/

Fashion https://github.com/zalandoresearch/fashion-mnist

Doodles https://github.com/googlecreativelab/quickdraw-dataset

Cifar10 https://www.cs.toronto.edu/~kriz/cifar.html

Chapters

0:00 Introduction

2:39 The decision boundary

3:49 Weights

5:42 Biases

6:45 Hidden layers

7:45 Programming the network

9:57 Activation functions

12:42 Cost

15:07 Gradient descent example

18:22 The cost landscape

19:55 Programming gradient descent

21:10 It's learning! (slowly)

23:21 Calculus example

27:34 The chain rule

29:50 Some partial derivatives

33:14 Backpropagation

39:25 Digit recognition

43:56 Drawing our own digits

47:37 Fashion

48:25 Doodles

52:00 The final challenge

Music:

Cosmic Waves - Michael FK

Amber - The Stolen Orchestra

Beyond the Horizon - Sounds Like Sander

Air - Assaf Ayalon

Purest Form - Sounds Like Sander

Hear Wide Open - Sounds Like Sander

Universal Wonder - Moments

Roman P - Moments

All In Good Time - Shimmer

It Will Come Back - The Stolen Orchestra

Frontier - Shimmer

New Moon - Cloud Wave

Sunflower - Cody Martin

Inner Peace - Moments

Enchanted - Cody Martin

Just Around The Corner - Shimmer