Deep Learning

A common question I get asked is: Are Convolutional Neural Networks invariant to changes in translation, rotation, and scaling? Is that why they are such powerful image classifiers? To answer this question, we first need to discriminate between the individual…

We’ll study the fundamentals of neural networks in depth. We’ll start with a discussion of artificial neural networks and how they are inspired by the real-life biological neural networks in our own bodies. From there, we’ll review the classic Perceptron…

First introduced by Rosenblatt in 1958, The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain is arguably the oldest and most simple of the ANN algorithms. Following this publication, Perceptron-based techniques were all the rage in…

Backpropagation is arguably the most important algorithm in neural network history — without (efficient) backpropagation, it would be impossible to train deep learning networks to the depths that we see today. Backpropagation can be considered the cornerstone of modern neural…

You might have started to notice a pattern in our Python code examples when training neural networks. There are four main ingredients you need to put together in your own neural network and deep learning algorithm: a dataset, a model/architecture,…

In this tutorial, we will discuss the concept of weight initialization, or more simply, how we initialize our weight matrices and bias vectors. This tutorial is not meant to be a comprehensive initialization technique; however, it does highlight popular methods,…

In this tutorial, you will learn: What gradient descent is How gradient descent enables us to train neural networks Variations of gradient descent, including Stochastic Gradient Descent (SGD) How SGD can be improved using momentum and Nesterov acceleration To learn…

Revised and updated 4/10/2023 Table of Contents What Is Deep Learning? Topic Description A Brief Overview of Machine Learning The Basics of Neural Networks Deep Learning vs. Traditional Machine Learning Challenges and Limitations of Deep Learning Data Requirements Interpretability Overfitting…

Based on our previous two sections on image classification and types of learning algorithms, you might be starting to feel a bit steamrolled with new terms, considerations, and what looks to be an insurmountable amount of variation in building an…