Handwritten Digit Recognizer

Interactive Neural Network in a Shader

Dive into the world of machine learning with this unique, interactive project built entirely within a LÖVE 2D shader. Draw a number with your mouse, and watch a small, pre-trained neural network run in real-time, attempting to recognize your handwriting.

This is a compact neural network with 784 input neurons, a hidden layer of 128 neurons, and 10 output neurons, and it runs entirely on the client side. To make the entire network run on your GPU, a unique approach was taken: the model's 100,490 weights and biases were saved as image files (.exr format) during the training process. These images are then loaded into the shader as textures, allowing the GPU to perform all the calculations in real-time. The model was specifically trained on the MNIST dataset, using techniques like dropout and data augmentation (random affine transformations, Gaussian noise) to improve its robustness against variations in handwriting. The final trained model achieved an accuracy of over 90% on the test set, demonstrating its ability to generalize to new handwriting examples.

How to Use:

• Left Mouse Button: Draw on the canvas.
• Right Mouse Button: Erase what you've drawn.
• Mouse Wheel: Adjust the canvas size.
• Hold Middle Mouse Button: Drag to move the canvas.
• Press 'C': Clear the entire screen.
• Press 'Esc': Quit the application.

Experience the fascinating intersection of graphics programming and machine learning firsthand!