This is a set of introductions to the core components of neural networks.
nn_by_scalarintroduces you to a neural network via individual scalar values that back up a single "neuron" within a layer within a network.nn_by_tensoruses numpy to introduce mechanisms of batching data and a demonstration of the slight updates that are required to set weights, biases, and gradients within a tensor.
As you go through, pay close attention to the derivative descriptions in nn_by_scalar.scalar and nn_by_tensor.layer. It is also great to consider how a neuron is made up of a set of simple operations: multiplcation of weights and input data, addition with the biases, and then the slightly- more-complicated activation functions.
These are largely based on projects from Andrej Karpathy and Joel Grus whose excellent demos I use in my classes. In this case, they have been updated to use consistent nomenclature with each other and with my teaching.