If you’re serious about doing any type of deep learning, you should be utilizing your GPU rather than your CPU. And the more GPUs you have, the better off you are.
If you already have an NVIDIA supported GPU, then the next logical step is to install two important libraries:
- The NVIDIA CUDA Toolkit: A development environment for building GPU-accelerated applications. This toolkit includes a compiler specifically designed for NVIDIA GPUs and associated math libraries + optimization routines.
- The cuDNN library: A GPU-accelerated library of primitives for deep neural networks. Using the cuDNN package, you can increase training speeds by upwards of 44%, with over 6x speedups in Torch and Caffe.
In the remainder of this blog post, I’ll demonstrate how to install both the NVIDIA CUDA Toolkit and the cuDNN library for deep learning.
Specifically, I’ll be using an Amazon EC2 g2.2xlarge machine running Ubuntu 14.04. Feel free to spin up an instance of your own and follow along.
By the time you’re finished this tutorial, you’ll have a brand new system ready for deep learning.
How to install CUDA Toolkit and cuDNN for deep learning
As I mentioned in an earlier blog post, Amazon offers an EC2 instance that provides access to the GPU for computation purposes.
This instance is named the g2.2xlarge instance and costs approximately $0.65 per hour. The GPU included on the system is a K520 with 4GB of memory and 1,536 cores.
You can also upgrade to the g2.8xlarge instance ($2.60 per hour) to obtain four K520 GPUs (for a grand total of 16GB of memory).
For most of us, the g2.8xlarge is a bit expensive, especially if you’re only doing deep learning as a hobby. On the other hand, the g2.2xlarge instance is a totally reasonable option, allowing you to forgo your afternoon Starbucks coffee and trade a caffeine jolt for a bit of deep learning fun and education.
Insider the remainder of this blog post, I’ll detail how to install the NVIDIA CUDA Toolkit v7.5 along with cuDNN v5 in a g2.2xlarge GPU instance on Amazon EC2.
If you’re interested in deep learning, I highly encourage you to setup your own EC2 system using the instructions detailed in this blog post — you’ll be able to use your GPU instance to follow along with future deep learning tutorials on the PyImageSearch blog (and trust me, there will be a lot of them).
Note: Are you new to Amazon AWS and EC2? You might want to read Deep learning on Amazon EC2 GPU with Python and nolearn before continuing. This blog post provides step-by-step instructions (with tons of screenshots) on how to spin up your first EC2 instance and use it for deep learning.
Installing the CUDA Toolkit
Assuming you have either (1) an EC2 system spun up with GPU support or (2) your own NVIDIA-enabled GPU hardware, the next step is to install the CUDA Toolkit.
But before we can do that, we need to install a few required packages first:
$ sudo apt-get update $ sudo apt-get upgrade $ sudo apt-get install build-essential cmake git unzip pkg-config $ sudo apt-get install libopenblas-dev liblapack-dev $ sudo apt-get install linux-image-generic linux-image-extra-virtual $ sudo apt-get install linux-source linux-headers-generic
One issue that I’ve encountered on Amazon EC2 GPU instances is that we need to disable the Nouveau kernel driver since it conflicts with the NVIDIA kernel module that we’re about to install.
Note: I’ve only had to disable the Nouveau kernel driver on Amazon EC2 GPU instances — I’m not sure if this needs to be done on standard, desktop installations of Ubuntu. Depending on your own hardware and setup, you can potentially skip this step.
To disable the Nouveau kernel driver, first create a new file:
$ sudo nano /etc/modprobe.d/blacklist-nouveau.conf
And then add the following lines to the file:
blacklist nouveau blacklist lbm-nouveau options nouveau modeset=0 alias nouveau off alias lbm-nouveau off
Save this file, exit your editor, and then update the initial RAM filesystem, followed by rebooting your machine:
$ echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf $ sudo update-initramfs -u $ sudo reboot
After reboot, the Nouveau kernel driver should be disabled.
The next step is to install the CUDA Toolkit. We’ll be installing CUDA Toolkit v7.5 for Ubuntu 14.04. Installing CUDA is actually a fairly simple process:
- Download the installation archive and unpack it.
- Run the associated scripts.
- Select the default options/install directories when prompted.
To start, let’s first download the .run file for CUDA 7.5:
$ wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda_7.5.18_linux.run
With the super fast EC2 connection, I was able to download the entire 1.1GB file in less than 30 seconds:
Next, we need to make the .run file executable:
$ chmod +x cuda_7.5.18_linux.run
Followed by extracting the individual installation scripts into an installers directory:
$ mkdir installers $ sudo ./cuda_7.5.18_linux.run -extract=`pwd`/installers
Your installers directory should now look like this:
Notice how we have three separate .run files — we’ll need to execute each of these individually and in the correct order:
NVIDIA-Linux-x86_64-352.39.runcuda-linux64-rel-7.5.18-19867135.runcuda-samples-linux-7.5.18-19867135.run
The following set of commands will take care of actually installing the CUDA Toolkit:
$ sudo ./NVIDIA-Linux-x86_64-352.39.run $ modprobe nvidia $ sudo ./cuda-linux64-rel-7.5.18-19867135.run $ sudo ./cuda-samples-linux-7.5.18-19867135.run
Again, make sure you select the default options and directories when prompted.
To verify that the CUDA Toolkit is installed, you should examine your /usr/local directory which should contain a sub-directory named cuda-7.5 , followed by a sym-link named cuda which points to it:
Now that the CUDA Toolkit is installed, we need to update our ~/.bashrc configuration:
$ nano ~/.bashrc
And then append the following lines to define the CUDA Toolkit PATH variables:
# CUDA Toolkit
export CUDA_HOME=/usr/local/cuda-7.5
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
export PATH=${CUDA_HOME}/bin:${PATH}
Your .bashrc file is automatically source ‘d each time you login or open up a new terminal, but since we just modified it, we need to manually source it:
$ source ~/.bashrc
Next, let’s install cuDNN!
Installing cuDNN
We are now ready to install the NVIDIA CUDA Deep Neural Network library, a GPU-accelerated library for deep neural networks. Packages such as Caffe and Keras (and at a lower level, Theano) use cuDNN to dramatically speedup the networking training process.
To obtain the cuDNN library, you first need to create a (free) account with NVIDIA. From there, you can download cuDNN.
For this tutorial, we’ll be using cuDNN v5:
Make sure you download the cuDNN v5 Library for Linux:
This is a small, 75MB download which you should save to your local machine (i.e., the laptop/desktop you are using to read this tutorial) and then upload to your EC2 instance. To accomplish this, simply use scp , replacing the paths and IP address as necessary:
$ scp -i EC2KeyPair.pem ~/Downloads/cudnn-7.5-linux-x64-v5.0-ga.tgz ubuntu@<ip_address>:~
Installing cuDNN is quite simple — all we need to do is copy the files in the lib64 and include directories to their appropriate locations on our EC2 machine:
$ cd ~ $ tar -zxf cudnn-7.5-linux-x64-v5.0-ga.tgz $ cd cuda $ sudo cp lib64/* /usr/local/cuda/lib64/ $ sudo cp include/* /usr/local/cuda/include/
Congratulations, cuDNN is now installed!
Doing a bit of cleanup
Now that we have (1) installed the NVIDIA CUDA Toolkit and (2) installed cuDNN, let’s do a bit of cleanup to reclaim disk space:
$ cd ~ $ rm -rf cuda installers $ rm -f cuda_7.5.18_linux.run cudnn-7.5-linux-x64-v5.0-ga.tgz
In future tutorials, I’ll be demonstrating how to use both CUDA and cuDNN to facilitate faster training of deep neural networks.
What's next? We recommend PyImageSearch University.
86 total classes • 115+ hours of on-demand code walkthrough videos • Last updated: October 2024
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 86 courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 86 Certificates of Completion
- ✓ 115+ hours of on-demand video
- ✓ Brand new courses released regularly, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 540+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
In today’s blog post, I demonstrated how to install the CUDA Toolkit and the cuDNN library for deep learning. If you’re interested in working with deep learning, I highly recommend that you setup a GPU-enabled machine.
If you don’t already have an NVIDIA-compatible GPU, no worries — Amazon EC2 offers the g2.2xlarge ($0.65/hour) and the g2.8xlarge ($2.60/hour) instances, both of which can be used for deep learning.
The steps detailed in this blog post will work on both the g2.2xlarge and g2.8xlarge instances for Ubuntu 14.04 — feel free to choose an instance and setup your own deep learning development environment (in fact, I encourage you to do just that!)
The entire process should only take you 1-2 hours to complete if you are familiar with the command line and Linux systems (and have a small amount of experience in the EC2 ecosystem).
Best of all, you can use this EC2 instance to follow along with future deep learning tutorials on the PyImageSearch blog.
Be sure to signup for the PyImageSearch Newsletter using the form below to be notified when new deep learning articles are published!
Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF
Enter your email address below to join the PyImageSearch Newsletter and download my FREE 17-page Resource Guide PDF on Computer Vision, OpenCV, and Deep Learning.
Cant wait to try this out! Thanks Adrian
Unfortunately, nouveau is PITA on all Linux distros since its kernel module is installed by default. The fact it persists on AWS should be addressed in official HVM instances.
excelent topic i always find a lot of interesting topics in your blog.
Thanks for another excelent
Thanks Andres 🙂
Really great topic.
I follow your blog as reference for best practices and new approach.
I’m having a problem of memory leak after the installation, did you have any similar problem?
Hey Hugo — I personally haven’t ran into a memory leak issue. What process is causing the memory leak?
Hi Adrian,
I finished the installation process, and I tried to execute some tests.I had the same leak in each of these commands.
nvidia-smi
deviceQuerys (from cuda samples)
make runtest (from caffe library)
I executed the command and my screen froze, after a minute I had this leak into kern.log (http://pastebin.com/bc66wrM0)
Hey Adrian, every process that uses the driver causes the leak, I had tried nvidia-smi, caffe make runtest, and deviceQuery from cuda samples.
Thanks for sharing. Again, I haven’t seen this behavior before — but I’ll be on the lookout. And it’s good for to other PyImageSearch readers to be aware of as well.
Hey Adrian, I have window 10.
What is the process of installing deep learning?
Windows is not a good choice for Deep Learning. I would suggest setting up an Amazon EC2 GPU instance running Ubuntu for deep learning, especially if you’re just getting started.
If you have an Intel CPU and installed Intel MKL library (it is not free), you could do it in Windows. You will find a way to tune up OpenCV DNN source and compile it with Intel MKL library linking. Then it will speed up 440x times than original OpenCV DNN.
What you will need could be
1. Intel based workstation with at least 32 GB ram,
2. Intel MKL library (not free),
3. Caffe for Windows (optional),
4. A custum built OpenCV 3.1.0 tuned up with MKL (you need to change replace OpenCV function with MKL function),
5. Nvidia GPUs,
6. Caffe model and prototxt files(the official bvlc deploy.prototxt file has a bug; you need to fix it).
Train procedure:
1. Gather images (about at least 2000 ) and make train and evaluation list in text files
2. Load Caffe model and prototxt
3. Load images with OpenCV.
4. Train them with OpenCV::ml library (e.g. SVM); it will give you a SVM yml file.
5. Evaluate your test to check costs and error rate (lower is better)
6. Predict a new image with the SVM yml file.
I found this repo (https://github.com/philferriere/dlwin) to very clear and helpful for setting up Deep Learning in Windows.
Sir,
Isnt there anything for amd gpu’s I have amd radeon M series ? What to do ?
Hi. I’ve tried these instructions but nvidia-smi hangs the terminal. Here is an NVIDIA thread, there is potentially a fix there. I ended up using 7.0. https://devtalk.nvidia.com/default/topic/880246/cuda-7-5-unstable-on-ec2-/
I’ve had issues installing CUDA 7.5 myself, but that was 6 months ago on an EC2 instance. I wish I could remember what the exact issue was. Thanks for sharing the thread!
Thanks for sharing, Martin. I ran into the same issue. When I tried your suggestion of using Cuda 7.0, that still didn’t work. So I ended up following the suggestion of Silvain in that same thread who mentioned to installed the latest NVidia driver:
NVIDIA-Linux-x86_64-361.45.11.run
That worked for me.
Thaks Adrian, this page is very helpful.
I’m glad you found the tutorial helpful Yoshi! 🙂
to disable we have to create “blacklist-nouveau.conf” file ….what after installation of CUDA ….do we need to delete this file ..?
After installation I would keep this file to ensure the Nouveau driver is disabled and the CUDA driver is always used.
Hello Adrian, Thanks for your posting ! It is very nice.
Can I have a question?
I do a project about number recognition. and I’m a beginner…
Other people says SVM is good method for number recognition.
(And Your web site looks perpect for it! 🙂 )
But I have a question.
Your posting keep talking about GPU and It looks focusing on Mac PC(?)
But I want to do a project with raspberry pi 3 as a processing machine.
Is raspberry pi 3 not good for machine learning?
or How do I Read your postings?
(Pls give me “To Read List” for ‘Raspberry pi + SVM’)
If you’re just getting started with computer vision and machine learning, then number recognition is a great first project. I would also suggest starting with k-NN, Logistic Regression, and SVMs as these are good “starter” algorithms.
I do indeed use a Mac; however, I SSH into my GPU instance. If could certainly use your Pi for machine learning, but keep in mind that you’re limited my RAM. Many machine learning algorithms are quite memory hungry and the Pi 3 only has 1GB of RAM.
May I ask why we use nano to write
“options nouveau modeset=0”
into the nouveau-kms.conf file and after that append it a 2nd time to the same file with
“$ echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf”?
It’s not unlikely I’ just overlooking something, so excuse the question.
just i want to say, thanks so much!, i read this page many times in the last months, and i learned a lot!, thank you for all the times you have given us an answer to our problems!!!
thanks from CHile
While installing cuDNN, adding -P will keep the symbolic links intact:
$ sudo cp -P lib64/* /usr/local/cuda/lib64
$ sudo cp -P include/* /usr/local/cuda/include/
For me, this took care of the message: /sbin/ldconfig.real: /usr/local/cuda-8.0/targets/x86_64-linux/lib/libcudnn.so.5 is not a symbolic link
Hello Adrian,
Just to let you know I followed the steps with CUDA 8.0 and CUDNN 5.1 and apparently it works, I have now to install TensorFlow to be totally sure
Cheers,
mph
Congrats on getting CUDA and TensorFlow installed, that’s awesome 🙂
Hi Adrian.. thank you for such a wonderful tutorial.. i got stuck in the step where i have to install cuda , exactly after this ((After reboot, the Nouveau kernel driver should be disabled.)) i always got this message that i am running X server.. but after rebooting and before login i just pressed Ctrl+Alt+F1 and then i stopped the lightdm , and the tried again to install cuda and it worked just fine.. again thanks for this tutorial
Thank you for sharing Naif!
Hello,
I was trying to follow this tutorial (it was referenced here https://github.com/agent-jay/dl_setup_metaguide ) but I got stuck at the following point when installing CUDA:
$ sudo ./NVIDIA-Linux-x86_64-352.39.run
$ modprobe nvidia
$ sudo ./cuda-linux64-rel-7.5.18-19867135.run
$ sudo ./cuda-samples-linux-7.5.18-19867135.run
Specifically, when I say modprobe nvidia, the terminal says: “modprobe: ERROR: could not insert ‘nvidia_375’: Operation not permitted”
I’m new to ubuntu (just installed it for the first time yesterday) so I’m not entirely sure how to proceed. Any help? I have ubuntu 16.04.2, my gpu is the quadro k2200 (the driver I chose in software and updates was the 375.39 one, which according to the nvidia site seems to be the right one).
What could be the problem and what are some possible solutions?
Thanks,
Ana
Hi
did you try updating linux kernel?
sudo modprobe nvidia
https://xkcd.com/149/
Hi Adrian,
Having installed both CUDA-8.0 and cuDNN, What setup will be needed to use torch/caffe to speed up training and testing time?.
CUDA and cuDNN are the main optimization libraries when it comes to deep learning. Other than that, I’m not sure what other speedups you are looking for?
Hi Adrian,
I followed your instruction step by step, but i get stack on: $ sudo ./NVIDIA-Linux-x86_64-352.39.run
it shows: You do not appear to have an NVIDIA GPU supported by the 352.39
NVIDIA Linux graphics driver installed in this system. For further
details, please see the appendix SUPPORTED NVIDIA GRAPHICS CHIPS in
the README available on the Linux driver download page at
http://www.nvidia.com.
Do you know what the problem is? I am new to ubuntu and I have gtx970 in my computer.
Thanks!
Hi Summer, thanks for the comment. Unfortunately, I haven’t used the GTX 970 before so I’m not sure what the exact error is. The latest GPU driver released by NVIDIA is 375.51 at the time of this writing, so I would suggest upgrading your drivers.
I wasn’t able to install CUDA with runfile method, the .deb method worked however:
Installation Instructions:
`sudo dpkg -i cuda-repo-ubuntu1604_8.0.61-1_amd64.deb`
`sudo apt-get update`
`sudo apt-get install cuda`
Thanks a lot .. this was really helpful
Hey Adrian, thank you for another detailed tutorial. I’ve been testing multiple frameworks and they happen to use different cuDNN versions. So, I had to change the version of cuDNN often to switch to another framework, which becomes very frustrating in a short time. For this reason, I wrote a simple script called cuDNNv, which can be used to clean install cuDNN and change the version with a click of a button. I would be very pleased if you could check it and give some feedback, and share it with others if you’d like. It is at:
https://github.com/dnzzcn/cuDNNv
Thank you again for amazing tutorials you share for free, I learned many things from you 🙂
Thanks for sharing Deniz! Earlier today I was fighting with some cuDNN + TensorFlow compatibly issues. This script would have been real handy then 🙂
oh no, nvidia
With this installation, CUDA and CUDNN could be used for dlib with GPU support??
You need to install dlib explicitly with GPU support. That said, this is an old tutorial. You should be be following this tutorial instead.