Considerations when setting up deep learning hardware

In last week’s blog post, I discussed my investment in an NVIDIA DIGITS DevBox for deep learning.

It was quite the investment, weighing in at a staggering (and wallet breaking) $15,000 — more than I ever thought I would spend on a computer that lives in my office (I normally like hardware that exists in the cloud where I don’t need to physically interact with it).

That said, this is an investment that will drive the PyImageSearch blog over the coming months. I’ve made the commitment do to more more deep learning tutorials — specifically tutorials that involve Convolutional Neural Networks and image classification.

In fact, over Memorial Day weekend, I spent two hours inside Trello brainstorming and planning the next year’s worth of tutorials on the PyImageSearch blog.

Most of them involve deep learning.

So, with that said, let’s take a look at some considerations you should keep in mind if you decide to purchase your own DevBox or build your own system for deep learning.

Considerations when setting up deep learning hardware

The NVIDIA DevBox isn’t a “normal” piece of hardware like your laptop or desktop. While you can simply plug it into the wall and boot it up, you really shouldn’t.

Before you even boot your system up for the first time, you need to consider how you are going to handle power surges, battery backup, and where the box is going to “live” in your home or office (this is critical for keeping the system cool).

While I’m writing this blog post in context of the NVIDIA DIGITS DevBox, these same suggestions will apply to you if you decide to build your own deep learning system from the ground up.

Get a UPS — but not just any UPS

First of, you need to get an Uninterruptible Power Supply (UPS). As the name suggests, a UPS provides emergency power to your system(s) when your main power source fails — you can think of a UPS as a large, (very) heavy battery pack to keep your system running when the power goes out. A UPS can even initiate a shutdown sequence once the battery backup levels are critically low.

But before you run down to your local electronics store or hop on Amazon.com, you need to ask yourself one critical question — how much power will my system be drawing?

The microwave test

At max load, the NVIDIA DevBox power consumption can reach 1350W (this is due to the 4 Titan X GPUs in the system).

In the United States, 1350W is comparable to higher-end microwaves, so you should perform what I like to call “the microwave test”.

Check the wattage of your microwave, and if it’s similar to the wattage draw of the deep learning system you’re looking to build, haul the microwave down to the room where you plan on hooking up your deep learning system, turn it on, and let it run for a few minutes.

If you don’t overload the circuit after a few minutes, then you’re probably in a good shape.

I stress the term probably because this isn’t a perfect test. Running a microwave of similar wattage for a few minutes is not the same as running a system for days or even weeks on end at high load. When running your deep learning system, you’ll be drawing more power for longer periods of time.

Therefore, it’s critical to do further research and ensure your home/office is wired properly for this sustained power draw. In some cases, you may need to consult an electrician.

Death by RAID

You know what happens when you have a RAID array (Redundant Array of Independent Disks) running and the power abruptly shuts off and your system immediately shuts down?

Bad things. Bad things will happen.

Your data can be left in an inconsistent, “dirty” state, and your system may need to create the entire array (if it can). As this Why power failures are bad for your data article suggestions, RAID 0 can (potentially) be completed destroyed in a power failure.

Luckily, most modern Unix kernels and RAID tools have built-in precautions to help prevent catastrophic loss during a power failure.

But why take the chance?

Invest in a UPS that can initiate a safe, clean shutdown in the event power goes out and the battery levels are critically low.

Note: Why am I talking about RAID? Because the NVIDIA DIGITS DevBox ships with 3 x 3TB RAID5 disks with a separate SSD cache. The last thing I would want would be to (1) rebuild that array or (2) lose my data due to power failure.

The land hurricane

It’s defined as “a widespread, long-lived, straight-line wind storm that is associated with a land-based, fast-moving group of thunderstorms” [Wikipedia].

Derechos are extremely damaging thunderstorms capable of producing hurricane force winds, tornadoes, heavy downpours, and flash floods. Some people even refer to derechos as land hurricanes. These types of storms happen primarily in North America, but have occurred elsewhere in the world with substantially less frequency.

It’s important to understand that these storms develop quickly. In some cases, you may not even hear the thunder as the storm approaches — it’s moving that rapidly. Keep in mind that the storm itself is moving at 60 MPH or higher — most thunderstorms don’t even have gusts that reach 60 MPH.

During my first year of graduate work at UMBC, my home state of Maryland was hit by a historic derecho on Friday, June 29th, 2012. This storm was one of the most destructive, deadly, and fast-moving storms in North American History.

I was alone in my apartment and the time. And to tell you the truth, it was one of the most frightening experiences of my life.

Within minutes, extremely high winds (> 60 MPH with gusts approaching 100 MPH) knocked down power lines and toppled large trees.

Roads flooded.

Brutal ground lighting strikes flashed across the dark, ominous sky, followed by nearly instantaneous, deafening thunder claps that relentlessly continued from 10PM at night to 4AM the next morning.

28 people died during the storm as large trees fell onto their homes, crushing them underneath.

When the storm was over, most of my home city of Catonsville, MD didn’t have power for over 6 days. Some parts of Maryland didn’t have power for weeks — an unheard of time living in the modern day United States.

So, why am I telling you this?

During the storm, one of the electrical lines outside my apartment building was struck by lighting. The surge traveled through the line, to the apartment, and into my UPS.

My APC UPS died that night…

…but my electronics didn’t.

My laptops and hard drives were safe.

Ever since that night, I’ve always, always bought APC — and I’ve made damn sure that any type of sensitive equipment or important data are behind a UPS.

Sure, if the surge is strong enough, no surge protector can save you. But you might as well do your damnedest to ensure you don’t lose your equipment (or more importantly, the data) during a thunderstorm (or any other freak act of nature).

In this case, I only lost my UPS — I could have lost a lot more.

But not just any UPS will do

So, as I mentioned above, you should be utilizing a UPS that has:

1. Battery backup
2. Surge protection

That’s all fine and good, but you also need to consider the wattage that you’re drawing.

Most consumer level UPS’ cap out at 900W — in fact, I have this exact UPS that I use for my laptops and external hard drives that caps out a 865W.

Considering that I would need to draw up to 1350W at maximum load, I clearly need an industrial, server-level UPS.

After doing a bunch of research, I found a bunch of UPS’ that can easily handle higher wattage and amps. I ended up going with the APC SMT2200RM2U UPS, which is actually 2x the price of other UPS’ I was looking at — but as I said, I have a trust and affinity with APC, so I decided to spend the extra money.

This rack mount UPS can handle 1980W and 2200VA, more than enough for my NVIDIA DevBox.

Racking it all up

After I received my UPS, I performed a quick test to ensure that the NVIDIA DevBox would work with it. Sure enough, it did (although this image is almost embarrassing to post):

When I posted this image in a Slack channel with a few of my other entrepreneur colleagues, my friend Nick Cook of SysDoc.io nearly shit a brick.

Nick quickly pointed out the problems of having a UPS sitting directly on top of carpet, followed by the DevBox sitting on top of the UPS. He was absolutely right — I needed to rack it all up to help with cooling and to keep the equipment safe.

So again, I went to do my research and ended up purchasing the Tripp Lite 18U SmartRack. I could have gone with a smaller rack, but I decided to:

1. Allow myself to expand and include new systems in the future, if I so desire.
2. Give the NVIDIA DevBox a little more “breathing room” for cooling.

Note: The NVIDIA DIGITS DevBox is not made to be rack mountable. Instead, I bought a heavy-duty shelf, racked the shelf, and then set the DevBox on top of the shelf.

The rack itself was delivered in quite the large cardboard container:

Compared to my UPS and DevBox, the rack could easily include both, leaving room for growth:

I then spent the afternoon racking the UPS and DevBox:

Which looks quite nice in the space:

The DevBox will also serve as a nice heater during the winter months:

What's next? We recommend PyImageSearch University.

Course information:
86 total classes • 115+ hours of on-demand code walkthrough videos • Last updated: October 2024
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled

I strongly believe that if you had the right teacher you could master computer vision and deep learning.

Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?

That’s not the case.

All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.

If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.

Inside PyImageSearch University you'll find:

• ✓ 86 courses on essential computer vision, deep learning, and OpenCV topics
• ✓ 86 Certificates of Completion
• ✓ 115+ hours of on-demand video
• ✓ Brand new courses released regularly, ensuring you can keep up with state-of-the-art techniques
• ✓ Pre-configured Jupyter Notebooks in Google Colab
• ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
• ✓ Access to centralized code repos for all 540+ tutorials on PyImageSearch
• ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
• ✓ Access on mobile, laptop, desktop, etc.

Click here to join PyImageSearch University

Summary

In this blog post, I detailed considerations and suggestions you should keep in mind when setting up deep learning hardware. Regardless if you are using the NVIDIA DIGITS DevBox, or building a custom solution, make sure you:

1. Determine if your house, apartment, or office is correctly wired to draw the required amps/wattage.
2. Invest in a UPS to protect your investment, especially if you’re using RAID configured drives.
3. If necessary, rack it all up.

Now that the deep learning environment is all ready to go, I’ll spend next week detailing how to install the CUDA Toolkit for deep learning and cuDNN so you can take advantage of GPUs for speedy training.

Note: The NVIDIA DevBox already comes with CUDA and cuDNN installed, so I’ll be setting up an Amazon EC2 instance from scratch. You’ll be able to use these instructions to stand up your own deep learning machine so you can follow along with future tutorials.

Be sure to sign up for the PyImageSearch Newsletter using the form below to be notified when future blog posts are published!

Join the PyImageSearch Newsletter and Grab My FREE 17-page Resource Guide PDF

Enter your email address below to join the PyImageSearch Newsletter and download my FREE 17-page Resource Guide PDF on Computer Vision, OpenCV, and Deep Learning.

Join the Newsletter!

Website