In our previous tutorial, we discussed LeNet, a seminal Convolutional Neural Network in the deep learning and computer vision literature. VGGNet, (sometimes referred to as simply VGG), was first introduced by Simonyan and Zisserman in their 2014 paper, Very Deep…

One concept we have not discussed yet is architecture visualization, the process of constructing a graph of nodes and associated connections in a network and saving the graph to disk as an image (i.e., PNG, JPG, etc.). Nodes in the…

In this tutorial, you will implement a CNN using Python and Keras. We’ll start with a quick review of Keras configurations you should keep in mind when constructing and training your own CNNs. We’ll then implement ShallowNet, which as the…

Now that we have implemented neural networks in pure Python, let’s move on to the preferred implementation method — using a dedicated (highly optimized) neural network library such as Keras. Today, I will discuss how to implement feedforward, multi-layer networks…

We’ll study the fundamentals of neural networks in depth. We’ll start with a discussion of artificial neural networks and how they are inspired by the real-life biological neural networks in our own bodies. From there, we’ll review the classic Perceptron…

First introduced by Rosenblatt in 1958, The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain is arguably the oldest and most simple of the ANN algorithms. Following this publication, Perceptron-based techniques were all the rage in…

Backpropagation is arguably the most important algorithm in neural network history — without (efficient) backpropagation, it would be impossible to train deep learning networks to the depths that we see today. Backpropagation can be considered the cornerstone of modern neural…

You might have started to notice a pattern in our Python code examples when training neural networks. There are four main ingredients you need to put together in your own neural network and deep learning algorithm: a dataset, a model/architecture,…

In this tutorial, you will learn: What gradient descent is How gradient descent enables us to train neural networks Variations of gradient descent, including Stochastic Gradient Descent (SGD) How SGD can be improved using momentum and Nesterov acceleration To learn…