Getting Started with Docker for Machine Learning

---

Table of Contents

---

Getting Started with Docker for Machine Learning

In this tutorial, you will learn what Containers are, how they differ from Virtual Machines (VMs), and what Docker is. Finally, we will top it off by installing Docker on our local machine with simple and easy-to-follow steps.

This lesson is the 1st of a 3-part series on Docker for Machine Learning:

1. Getting Started with Docker for Machine Learning (this tutorial)
2. Lesson 2
3. Lesson 3

---

Overview: Why the Need?

Envision yourself as an ML Engineer at one of the world’s largest companies. You make a Machine Learning (ML) pipeline that does everything, from gathering and preparing data to making predictions. All the individual blocks are in the proper order, and you ensure everything works like a well-oiled machine.

After verifying everything, you hand over this pipeline to a colleague to get a review before your pipeline is deployed to perform real-time streaming predictions. But, alas! The pipeline broke down when moved from your computer to your colleague’s computer.

Switching gears, imagine yourself being part of a high-tech research lab working with Machine Learning algorithms. You quickly iterate with several different things daily, each with additional requirements and dependencies. One day, you are working with TPUs to run JAX code. Next, you’re trying to make your app available to the world as an API.

In these cases, you’d also encounter annoying times when your entire environment is polluted by one dirty install. Repairing such issues becomes time-consuming, especially in research iteration where time is crucial.

---

What Are Containers?

In the cases we just saw, the problem patterns stem from the tight coupling between the software and hardware we develop.

This can be a major impediment to developing software that “just works” on all systems. When we look at this problem in the Machine Learning context, this can bring quick iteration to a halt.

What if we decouple the dependencies of the software we write from the hardware it works on? Could our software development environment be wrapped into a neat and simple package that can be shifted to any machine we want, irrespective of hardware or other dependency issues?

Enter the concept of Containers. Containers are lightweight packages that act as portable, sandboxed environments for use on any hardware that supports them. This would include code, dependencies, files, operating systems, runtime, and system settings.

Let’s break that down. Containers are lightweight since they do not store Operating System (OS) images within them while using a snapshot instead. They virtualize the resources available to the OS for a specific process concerning CPU usage, memory, and storage. This allows your process to be portable and sandboxed from all other processes running on the host.

Container runtimes are consistent, meaning they would work precisely the same whether you’re on a Dell laptop with an AMD CPU, a top-notch MacBook Pro, or an old Intel Lenovo ThinkPad from 2015.

One of the most popular methods of creating and working with containers is by using Docker. It is a platform that provides a batteries-included experience for creating, deploying, and working with containers.

Throughout this tutorial, we will use Docker as the software for manipulating and working with containers. Some other alternatives to Docker include LXC (Linux Container Runtime) and Podman.

Note: Something of this sort would be useful to an ML Engineer such as yourself because it allows you to expose services from isolated runtimes and ensure horizontal scalability without any hassle.

---

How Do Containers Differ from Virtual Machines?

A question that may have popped up by now should be how containers differ from traditional Virtual Machines. Let’s see how they diverge.

The most notable difference between a virtual machine (VM) and a container is the level of virtualization. This refers to the introduction of an abstraction layer over the hardware at a level where the resources are allocated and utilized separately from the original OS on the machine. This abstraction layer is also referred to commonly as a hypervisor. They tend to have several functions (e.g., creating and deleting VMs, managing, and scheduling them). VMs work to virtualize everything up from the hardware layers on a machine with this layer on top.

On the other hand, containers only virtualize up to the software layers of the container engine that handles all the containers present in a system. It does not go down to the hypervisor layer and instead hands over its own control to the original OS on the machine, which is controlled by the container runtime. The container runtime is like a hypervisor but made only on a software level for containers. It does not have direct access to the hardware of the machine.

Virtual machines typically provide higher isolation and security by working directly with the hardware (depending on the type of hypervisor used). In contrast, Containers tend to simply isolate the specific process and filesystem while delegating the grunt work of allocating resources to the host OS itself via the container runtime.

When working with different Operating Systems vis-à-vis Containers, they tend only to have a snapshot of an OS instead of the full OS image, making them significantly lighter than traditional VMs. But, this brings about a problem wherein we must ensure that the container’s OS matches the same kernel as the original OS. That means the container’s OS must have the same format for requesting machine resources as that of the host OS. For that reason, one would see that a Windows container cannot be used on Linux machines. In comparison, Linux containers may be created on Windows machines with the advent of WSL2.

---

Most Popular Container Images

Today, popular software applications have container images available for direct use on any hardware that the user desires. Some notable examples include the following.

---

TensorFlow

The container runtime for TensorFlow acts as a simple method to quickly experiment or iterate with TensorFlow in a throwaway fashion. These images also support interfacing with the GPU, meaning you can leverage it for training your Deep Learning networks written in TensorFlow. Note that, using a GPU typically refers to using NVIDIA CUDA-enabled GPUs (Alternative NVIDIA NGC Container Image here).

---

PyTorch

The container runtime for PyTorch brings torch and torchvision to your runtime, along with all dependencies pre-installed. This allows you to begin your development directly and provides a platform for quick iteration. Similar to TensorFlow, this container image also provides a GPU-compatible version that uses Compute Unified Device Architecture (CUDA) to speed up Tensor processing. (Alternative NVIDIA NGC Container Image here)

---

Python

The container runtime for Python sets up a Debian Linux instance with Python pre-installed. This is suitable for making a variety of Python applications with other dependencies being added to it at the user’s convenience. This container is commonly used for simpler tasks (e.g., developing web services, APIs, running tests, etc.). Depending on their host platform, one can choose between using Debian Linux, Alpine Linux, or Windows Server Core versions of this image.

---

NGINX

The container runtime for NGINX sets up a Debian Linux instance with NGINX pre-installed. This allows you to use the popular reverse proxy server with load-balancers and HTTP caches without the hassle of setting it up across different machines. This image is also available in various styles (e.g., the standard Debian Linux base or with an Alpine Linux base).

---

Install Docker

With this information on why Docker containers are so popular, it’s time we install it on our local machine and start making our containers for Machine Learning.

To later work with all the required components of Docker, we install Docker Desktop. It includes a standalone Desktop GUI (graphical user interface) application that simplifies your experience with Docker.

If you are using Ubuntu, Debian, Fedora, Red Hat Enterprise Linux (RHEL), CentOS, or other common Linux Distributions, you can use the official convenience script from Docker by executing the command below.

```bash
curl https://get.docker.com | sh \
  && sudo systemctl --now enable docker
```

This should set up Docker completely on your system! Skip over the next section to see how you can install Docker Desktop for a GUI-based interface.

If you want to improve your Docker experience for later, you can follow some convenience-enhancing steps for Linux systems. Note that these are optional and are geared toward superusers.

---

Install Docker Manually

Now, let’s look at how to install Docker on a Linux system manually. For this example, we will look at a Fedora Linux setup with GNOME. Note, if you are using Ubuntu or any other Linux distribution, you can simply replace most of these commands with the distribution’s respective package manager (“apt” for Ubuntu, “pacman” for Arch Linux, “brew” for Mac, etc.).

• Ensure you have gnome-terminal installed (or the appropriate terminal as required). To do so, use the following command:

```bash 
>> sudo dnf install gnome-terminal
```

• Set up the Docker package repositories

```bash
>> sudo dnf -y install dnf-plugins-core
>> sudo dnf config-manager --add-repo https://download.docker.com/linux/fedora/docker-ce.repo
```

• Install Docker Engine, containerd, Docker build kit, Docker Compose, and related command line interfaces (CLIs)

```bash
>> sudo dnf install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```

Once this process is complete, you will have a working installation of Docker on your system!

---

Install Docker Desktop

Now, let’s look at how to install Docker Desktop on your system. Follow along!

• Download the RPM (Red Hat Package Management system) file for Docker Desktop (Note: This link may change in the future. To get the latest link, please look here.)
• Use the following command to install it:

```bash
>> sudo dnf install ./docker-desktop-<version>-<arch>.rpm
```

• Start Docker Desktop

```bash
systemctl --user start docker-desktop
```

The last command should launch a GUI window for Docker Desktop. You should now have a full installation of all the components required to work with Docker and a GUI application to handle it too!

Installation of Docker Desktop can be performed for other Linux distributions or Windows by following the instructions given on their official website as well.

---

Install NVIDIA Container Toolkit

If you have a CUDA-enabled NVIDIA GPU, you can also install the NVIDIA Container Toolkit to use the GPU for your workflows. To do so, follow the instructions here.

Note that to install the toolkit and have it work correctly, you must have previously installed the appropriate CUDA drivers for your GPU.

• Set up the package repository and the GPG (GNU Privacy Guard) Key

```bash
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
      && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
      && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
            sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
            sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

```

• Update your system

```bash
sudo apt-get update
```

• Install nvidia-container-toolkit as a package

```bash
sudo apt-get install -y nvidia-container-toolkit-base nvidia-container-toolkit
```

• Configure Docker such that it can recognize nvidia-container-toolkit and work with it

```bash
sudo nvidia-ctk runtime configure --runtime=docker
```

• Restart the daemon to activate changes

```bash
sudo systemctl restart docker
```

• Now, let’s try running a base CUDA container. Enter the following command:

```bash
sudo docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
```

• You should see a table giving information about the GPU you have on your device, along with information on the CUDA driver version.

---

---

Summary

In this blog post, we demystify Docker, highlighting its role in streamlining Machine Learning (ML) workflows. We introduce Docker as a lightweight, efficient tool that packs, ships, and runs applications as portable, isolated containers, contrasting this with the bulkier design of traditional Virtual Machines.

We guide you through setting up Docker on your system, explaining its significance in ML. Docker ensures consistent environments from development to deployment, eliminating compatibility issues and facilitating reproducible research.

Our post underlines Docker’s use cases (e.g., enhancing collaboration, easing cloud deployments, and scaling ML applications), thus demonstrating its value in simplifying and enhancing ML projects.

---

Citation Information

Mukherjee, S. “Getting Started with Docker for Machine Learning,” PyImageSearch, P. Chugh, A. R. Gosthipaty, S. Huot, K. Kidriavsteva, and R. Raha, eds., 2023, https://pyimg.co/k3xw6

@incollection{Mukherjee_2023_Docker4ML,
  author = {Suvaditya Mukherjee},
  title = {Getting Started with Docker for Machine Learning},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Aritra Roy Gosthipaty and Susan Huot and Kseniia Kidriavsteva and Ritwik Raha},
  year = {2023},
  url = {https://pyimg.co/k3xw6},
}

---

---