In last week’s blog post we got our feet wet by implementing a simple object tracking algorithm called “centroid tracking”.
Today, we are going to take the next step and look at eight separate object tracking algorithms built right into OpenCV!
You see, while our centroid tracker worked well, it required us to run an actual object detector on each frame of the input video. For the vast majority of circumstances, having to run the detection phase on each and every frame is undesirable and potentially computationally limiting.
Instead, we would like to apply object detection only once and then have the object tracker be able to handle every subsequent frame, leading to a faster, more efficient object tracking pipeline.
The question is — can OpenCV help us achieve such object tracking?
The answer is undoubtedly a resounding “Yes”.
To learn how to apply object tracking using OpenCV’s built-in object trackers, just keep reading.
OpenCV Object Tracking
In the first part of today’s blog post, we are going to briefly review the eight object tracking algorithms built-in to OpenCV.
From there I’ll demonstrate how we can use each of these object trackers in real-time.
Finally, we’ll review the results of each of OpenCV’s object trackers, noting which ones worked under what situations and which ones didn’t.
Let’s go ahead and get started tracking objects with OpenCV!
8 OpenCV Object Tracking Implementations
You might be surprised to know that OpenCV includes eight (yes, eight!) separate object tracking implementations that you can use in your own computer vision applications.
I’ve included a brief highlight of each object tracker below:
- BOOSTING Tracker: Based on the same algorithm used to power the machine learning behind Haar cascades (AdaBoost), but like Haar cascades, is over a decade old. This tracker is slow and doesn’t work very well. Interesting only for legacy reasons and comparing other algorithms. (minimum OpenCV 3.0.0)
- MIL Tracker: Better accuracy than BOOSTING tracker but does a poor job of reporting failure. (minimum OpenCV 3.0.0)
- KCF Tracker: Kernelized Correlation Filters. Faster than BOOSTING and MIL. Similar to MIL and KCF, does not handle full occlusion well. (minimum OpenCV 3.1.0)
- CSRT Tracker: Discriminative Correlation Filter (with Channel and Spatial Reliability). Tends to be more accurate than KCF but slightly slower. (minimum OpenCV 3.4.2)
- MedianFlow Tracker: Does a nice job reporting failures; however, if there is too large of a jump in motion, such as fast moving objects, or objects that change quickly in their appearance, the model will fail. (minimum OpenCV 3.0.0)
- TLD Tracker: I’m not sure if there is a problem with the OpenCV implementation of the TLD tracker or the actual algorithm itself, but the TLD tracker was incredibly prone to false-positives. I do not recommend using this OpenCV object tracker. (minimum OpenCV 3.0.0)
- MOSSE Tracker: Very, very fast. Not as accurate as CSRT or KCF but a good choice if you need pure speed. (minimum OpenCV 3.4.1)
- GOTURN Tracker: The only deep learning-based object detector included in OpenCV. It requires additional model files to run (will not be covered in this post). My initial experiments showed it was a bit of a pain to use even though it reportedly handles viewing changes well (my initial experiments didn’t confirm this though). I’ll try to cover it in a future post, but in the meantime, take a look at Satya’s writeup. (minimum OpenCV 3.2.0)
My personal suggestion is to:
- Use CSRT when you need higher object tracking accuracy and can tolerate slower FPS throughput
- Use KCF when you need faster FPS throughput but can handle slightly lower object tracking accuracy
- Use MOSSE when you need pure speed
Satya Mallick also provides some additional information on these object trackers in his article as well.
Object Trackers have been in active development in OpenCV 3. Here is a brief summary of which versions of OpenCV the trackers appear in:
Note: Despite following the instructions in this issue on GitHub and turning off precompiled headers, I was not able to get OpenCV 3.1 to compile.
Now that you’ve had a brief overview of each of the object trackers, let’s get down to business!
Object Tracking with OpenCV
To perform object tracking using OpenCV, open up a new file, name it opencv_object_tracker.py , and insert the following code:
# import the necessary packages from imutils.video import VideoStream from imutils.video import FPS import argparse import imutils import time import cv2
We begin by importing our required packages. Ensure that you have OpenCV installed (I recommend OpenCV 3.4+) and that you have imutils installed:
$ pip install --upgrade imutils
Now that our packages are imported, let’s parse two command line arguments:
# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-v", "--video", type=str,
help="path to input video file")
ap.add_argument("-t", "--tracker", type=str, default="kcf",
help="OpenCV object tracker type")
args = vars(ap.parse_args())
Our command line arguments include:
--video: The optional path to the input video file. If this argument is left off, then the script will use your webcam.--tracker: The OpenCV object tracker to use. By default, it is set tokcf(Kernelized Correlation Filters). For a full list of possible tracker code strings refer to the next code block or to the section below, “Object Tracking Results”.
Let’s handle the different types of trackers:
# extract the OpenCV version info
(major, minor) = cv2.__version__.split(".")[:2]
# if we are using OpenCV 3.2 OR BEFORE, we can use a special factory
# function to create our object tracker
if int(major) == 3 and int(minor) < 3:
tracker = cv2.Tracker_create(args["tracker"].upper())
# otherwise, for OpenCV 3.3 OR NEWER, we need to explicity call the
# approrpiate object tracker constructor:
else:
# initialize a dictionary that maps strings to their corresponding
# OpenCV object tracker implementations
OPENCV_OBJECT_TRACKERS = {
"csrt": cv2.TrackerCSRT_create,
"kcf": cv2.TrackerKCF_create,
"boosting": cv2.TrackerBoosting_create,
"mil": cv2.TrackerMIL_create,
"tld": cv2.TrackerTLD_create,
"medianflow": cv2.TrackerMedianFlow_create,
"mosse": cv2.TrackerMOSSE_create
}
# grab the appropriate object tracker using our dictionary of
# OpenCV object tracker objects
tracker = OPENCV_OBJECT_TRACKERS[args["tracker"]]()
# initialize the bounding box coordinates of the object we are going
# to track
initBB = None
As denoted in Figure 2 not all trackers are present in each minor version of OpenCV 3+.
There’s also an implementation change that occurs at OpenCV 3.3. Prior to OpenCV 3.3, tracker objects must be created with cv2.Tracker_create and passing an uppercase string of the tracker name (Lines 22 and 23).
For OpenCV 3.3+, each tracker can be created with their own respective function call such as cv2.TrackerKCF_create . The dictionary, OPENCV_OBJECT_TRACKERS , contains seven of the eight built-in OpenCV object trackers (Lines 30-38). It maps the object tracker command line argument string (key) with the actual OpenCV object tracker function (value).
On Line 42 the tracker object is instantiated based on the command line argument for the tracker and the associated key from OPENCV_OBJECT_TRACKERS .
Note: I am purposely leaving GOTURN out of the set of usable object trackers as it requires additional model files.
We also initialize initBB to None (Line 46). This variable will hold the bounding box coordinates of the object that we select with the mouse.
Next, let’s initialize our video stream and FPS counter:
# if a video path was not supplied, grab the reference to the web cam
if not args.get("video", False):
print("[INFO] starting video stream...")
vs = VideoStream(src=0).start()
time.sleep(1.0)
# otherwise, grab a reference to the video file
else:
vs = cv2.VideoCapture(args["video"])
# initialize the FPS throughput estimator
fps = None
Lines 49-52 handle the case in which we are accessing our webcam. Here we initialize the webcam video stream with a one-second pause for the camera sensor to “warm up”.
Otherwise, the --video command line argument was provided so we’ll initialize a video stream from a video file (Lines 55 and 56).
Let’s begin looping over frames from the video stream:
# loop over frames from the video stream
while True:
# grab the current frame, then handle if we are using a
# VideoStream or VideoCapture object
frame = vs.read()
frame = frame[1] if args.get("video", False) else frame
# check to see if we have reached the end of the stream
if frame is None:
break
# resize the frame (so we can process it faster) and grab the
# frame dimensions
frame = imutils.resize(frame, width=500)
(H, W) = frame.shape[:2]
We grab a frame on Lines 65 and 66 as well as handle the case where there are no frames left in a video file on Lines 69 and 70.
In order for our object tracking algorithms to process the frame faster, we resize the input frame to 50 pixels (Line 74) — the less data there is to process, the faster our object tracking pipeline will run.
We then grab the width and height of the frame as we’ll need the height later (Line 75).
Now let’s handle the case where an object has already been selected:
# check to see if we are currently tracking an object
if initBB is not None:
# grab the new bounding box coordinates of the object
(success, box) = tracker.update(frame)
# check to see if the tracking was a success
if success:
(x, y, w, h) = [int(v) for v in box]
cv2.rectangle(frame, (x, y), (x + w, y + h),
(0, 255, 0), 2)
# update the FPS counter
fps.update()
fps.stop()
# initialize the set of information we'll be displaying on
# the frame
info = [
("Tracker", args["tracker"]),
("Success", "Yes" if success else "No"),
("FPS", "{:.2f}".format(fps.fps())),
]
# loop over the info tuples and draw them on our frame
for (i, (k, v)) in enumerate(info):
text = "{}: {}".format(k, v)
cv2.putText(frame, text, (10, H - ((i * 20) + 20)),
cv2.FONT_HERSHEY_SIMPLEX, 0.6, (0, 0, 255), 2)
If an object has been selected, we need to update the location of the object. To do so, we call the update method on Line 80 passing only the frame to the function. The update method will locate the object’s new position and return a success boolean and the bounding box of the object.
If successful, we draw the new, updated bounding box location on the frame (Lines 83-86). Keep in mind that trackers can lose objects and report failure so the success boolean may not always be True .
Our FPS throughput estimator is updated on Lines 89 and 90.
On Lines 94-98 we construct a list of textual information to display on the frame . Subsequently, we draw the information on the frame on Lines 101-104.
From there, let’s show the frame and handle selecting an object with the mouse:
# show the output frame
cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF
# if the 's' key is selected, we are going to "select" a bounding
# box to track
if key == ord("s"):
# select the bounding box of the object we want to track (make
# sure you press ENTER or SPACE after selecting the ROI)
initBB = cv2.selectROI("Frame", frame, fromCenter=False,
showCrosshair=True)
# start OpenCV object tracker using the supplied bounding box
# coordinates, then start the FPS throughput estimator as well
tracker.init(frame, initBB)
fps = FPS().start()
We’ll display a frame and continue to loop unless a key is pressed.
When the “s” key is pressed, we’ll “select” an object ROI using cv2.selectROI . This function allows you to manually select an ROI with your mouse while the video stream is frozen on the frame.
The user must draw the bounding box and then press “ENTER” or “SPACE” to confirm the selection. If you need to reselect the region, simply press “ESCAPE”.
Using the bounding box info reported from the selection function, our tracker is initialized on Line 120. We also initialize our FPS counter on the subsequent Line 121.
Of course, we could also use an actual, real object detector in place of manual selection here as well.
Last week’s blog post already showed you how to combine a face detector with object tracking. For other objects, I would suggest referring to this blog post on real-time deep learning object detection to get you started. In a future blog post in this object tracking series, I’ll be showing you how to combine both the object detection and object tracking phase into a single script.
Lastly, let’s handle if the “q” key (for “quit”) is pressed or if there are no more frames in the video, thus exiting our loop:
# if the `q` key was pressed, break from the loop
elif key == ord("q"):
break
# if we are using a webcam, release the pointer
if not args.get("video", False):
vs.stop()
# otherwise, release the file pointer
else:
vs.release()
# close all windows
cv2.destroyAllWindows()
This last block simply handles the case where we have broken out of the loop. All pointers are released and windows are closed.
Object Tracking Results
In order to follow along and apply object tracking using OpenCV to the videos in this blog post, make sure you use the “Downloads” section to grab the code + videos.
From there, open up a terminal and execute the following command:
$ python opencv_object_tracking.py --video dashcam_boston.mp4 --tracker csrt
Be sure to edit the two command line arguments as desired: --video and --tracker .
If you have downloaded the source code + videos associated with this tutorial, the available arguments for --video include the following video files:
american_pharoah.mp4dashcam_boston.mp4drone.mp4nascar_01.mp4nascar_02.mp4race.mp4- …and feel free to experiment with your own videos or other videos you find online!
The available arguments for --tracker include:
csrtkcfboostingmiltldmedianflowmosse
Refer to the section, “8 OpenCV Object Tracking Implementations” above for more information about each tracker.
You can also use your computer’s webcam — simply leave off the video file argument:
$ python opencv_object_tracking.py --tracker csrt
In the following example video I have demonstrated how OpenCV’s object trackers can be used to track an object for an extended amount of time (i.e., an entire horse race) versus just short clips:
Video and Audio Credits
To create the examples for this blog post I needed to use clips from a number of different videos.
A massive “thank you” to Billy Higdon (Dashcam Boston), The New York Racing Association, Inc. (American Pharaoh), Tom Wadsworth (Multirotor UAV Tracking Cars for Aerial Filming), NASCAR (Danica Patrick leads NASCAR lap), GERrevolt (Usain Bolt 9.58 100m New World Record Berlin), Erich Schall, and Soularflair.
What's next? I recommend PyImageSearch University.
76 total classes • 90 hours of on-demand code walkthrough videos • Last updated: May 2023
★★★★★ 4.84 (128 Ratings) • 16,000+ Students Enrolled
I strongly believe that if you had the right teacher you could master computer vision and deep learning.
Do you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?
That’s not the case.
All you need to master computer vision and deep learning is for someone to explain things to you in simple, intuitive terms. And that’s exactly what I do. My mission is to change education and how complex Artificial Intelligence topics are taught.
If you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to successfully and confidently apply computer vision to your work, research, and projects. Join me in computer vision mastery.
Inside PyImageSearch University you'll find:
- ✓ 76 courses on essential computer vision, deep learning, and OpenCV topics
- ✓ 76 Certificates of Completion
- ✓ 90 hours of on-demand video
- ✓ Brand new courses released regularly, ensuring you can keep up with state-of-the-art techniques
- ✓ Pre-configured Jupyter Notebooks in Google Colab
- ✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)
- ✓ Access to centralized code repos for all 500+ tutorials on PyImageSearch
- ✓ Easy one-click downloads for code, datasets, pre-trained models, etc.
- ✓ Access on mobile, laptop, desktop, etc.
Summary
In today’s blog post you learned how to utilize OpenCV for object tracking. Specifically, we reviewed the eight object tracking algorithms (as of OpenCV 3.4) included in the OpenCV library:
- CSRT
- KCF
- Boosting
- MIL
- TLD
- MedianFlow
- MOSSE
- GOTURN
I recommend using either CSRT, KCF, or MOSSE for most object tracking applications:
- Use CSRT when you need higher object tracking accuracy and can tolerate slower FPS throughput
- Use KCF when you need faster FPS throughput but can handle slightly lower object tracking accuracy
- Use MOSSE when you need pure speed
From there, we applied each of OpenCV’s eight object trackers to various tasks, including sprinting, horse racing, auto racing, drone/UAV tracker, and dash cams for vehicles.
In next week’s blog post you’ll learn how to apply multi-object tracking using a special, built-in (but mostly unknown) OpenCV function.
To be notified when next week’s blog post on multi-object tracking goes live, just enter your email address in the form below!
Download the Source Code and FREE 17-page Resource Guide
Enter your email address below to get a .zip of the code and a FREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning. Inside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!
Adrain i hava seen your face detection video it was nice. But some time there were false positives occuring. then i heard about openface for face recognition. why dont you make a tutorial on openface it will be very helpful
Navaneeth — you requested that very topic last week. I am aware of your comment and have acknowledged it. I love taking requests regarding what readers want to learn more above, but I need to kindly ask you to please stop requesting the same topic on every post. If I can cover it in the future I certainly will.
Hi Adam, great post as always! I wonder if you can do a separate post that use a similar approach(semi-supervised) for pixel-wise tracking?
It’s Adrian actually, not Adam 😉 I’ll be doing a pixel-wise segmentation post soon.
Hi, Very nice ! I test on RC plane video footage. Don’t know if bug or now but when bounding box fail to capture objet (rc plane), you must restart the loop from scratch (with video from timestamp zero) which is quite annoying. Is there any solution to delete the memorize bounding box with keyboard shortcut ? If I look long footage, it let better chance to successful track plane or another plane. Thanks !!! I tested goturn and performance is really poor I find (12/15 fps). Native code seems to run on CPU not GPU. Will investigate !!!
I haven’t done enough investigate work with GOTURN but just like any other machine learning algorithm, if the model was not trained on data representative of what it will see in the “real world” then it’s not going to perform well. I suspect GOTURN was not trained on RC plane video footage or objects that may appear in such footage.
As far as restarting the video all you should have to do is re-instantiate the actual OpenCV object tracker inside the “if” statement that is used to catch the “c” key. Something like this should work:
if key == ord("s"): initBB = cv2.selectROI("Frame", frame, fromCenter=False, howCrosshair=True) tracker = OPENCV_OBJECT_TRACKERS[args["tracker"]]() tracker.init(frame, initBB)Hi Adrian,
I am getting the following error when I am trying to execute the program!
Traceback (most recent call last):
File “opencv_object_tracking.py”, line 35, in
“csrt”: cv2.TrackerCSRT_create,
AttributeError: module ‘cv2.cv2’ has no attribute ‘TrackerCSRT_create’
The only time I’ve seen a similar error is from this post on saliency detection. If you check out the comments section, in particular this comments thread you’ll find what should be the solution. It appears there is a problem with your OpenCV install and/or version. Reading the thread I linked you to should help resolve it.
I think your version is official and some private proprietaries cause they dropout several functionality. The way to deal with was install opencv-contrib-python on this helpful link:
https://stackoverflow.com/questions/44633378/attributeerror-module-cv2-cv2-has-no-attribute-createlbphfacerecognizer
Thanks so much for sharing the solution, KhanhPhamDinh!
You need to uninstall opencv before installing opencv-contrib
Make sure no console is running that has imported cv2 while you execute your installing process
Run the cmd in Administration Mode (Go to the .exe of the cmd program, right click and select ‘Run as Administrator’)
Thanks a ton!! 🙂
uninstalled opencv-pthon and opencv-contrib-python
installed cv and then contib. which worked.
I’ve also run into this problem. I’m working on an Ubuntu 16.04 virtual machine – following Adrian’s build instructions for OpenCV (v3.4) I also installed opencv_contrib_python without any luck. It appears only to be CSRT. The rest of the trackers are there.
And I’ve just discovered my issue. I’m on OpenCV 3.4.0, and the minimum as mentioned above is 3.4.2.
I also encountered the same error. Have you solved it yet?
same here :/
got a solution?
uninstalled opencv-pthon and opencv-contrib-python
installed cv and then contib.
eureka!
please install opencv-contrib-python
Hi Adrian,
Is it possible to get object position during traking operation (delta position in relation of the center of the screen for example) ? I would like to plug the camera on a motorized support to “re-centering” tracked object on real-time operation.
David
Hey David, I’m not sure I fully understand your question. If you know the center (x, y)-coordinates of the screen along with the centroid or corners of the predicted bounding box you can always compute the distance between them to obtain the delta. But again, I may not be understanding your question properly.
Dear fellow. I ‘m window user and although i follow your manual to drag the objective region but it’s unable to track (without any blue rectangle appear). Could you help me? Best regards!
If you select the bounding box and then the bounding box immediately disappears after selection then the “success” boolean is false. The object tracker cannot track your object. You should try a different object tracker, keeping in mind my pros/cons listed in the post.
I tried the code and I got this error
AttributeError: module ‘cv2.cv2’ has no attribute ‘TrackerCSRT_create’
I am using a Mac-OS Sierra + Python3 and OpenCV 3.4.1
What could be the problem?
Make sure you read my reply to “Singh” as I have addressed this question.
uninstall opencv-python and then only install opencv-contrib-python.
As you can see here https://github.com/skvark/opencv-python#installation-and-usage ,
opencv-python is the main package and opencv-contrib-python is main package and contrib modules so installing them together is wrong, just install contrib.
Mehmet is 100% correct. Thanks Mehmet!
Hi all,
I had the same problem as Singh.
I followed Mehmet suggestion and, after unistalling opencv-python and installing ONLY opencv-contrib-python, everything worked fine for me!!!
Thank you very much!!!
thanks Adrian for the tips for re-instantiate the object tracker: perfectly works !
About Goturn , even on horse race or dashcam_boscam video, tracking is lost or become larger and larger with Goturn… Something is probably not well configured. Really cool piece of code ! love it …
Awesome, I’m glad that fix worked! 😀
Excellent post! Do you have a recommendation for traffic lights recognition ?
I would suggest training your own custom object detector. Faster R-CNNs and SSDs would be a good start. Use this post to help you get started. From there I cover how to train your own custom deep learning-based object detectors inside Deep Learning for Computer Vision with Python. Two of the chapters even have a self-driving car theme, including front/rear view vehicle recognition and traffic sign recognition.
i had implemented something similar. using yolo once per second +object follwing from dlib.in the meantime, together with a matrix to calculate the matching. depending on the algo, if you have 50+ objects it can get complex.
Congrats on the successful project, Florian! As you noted it can get very complex depending on how many objects you’re tracking. In a future post in this series I’m going to be demonstrating more efficient multi-object tracking by distributing the trackers across multiple cores on a processor.
Very impressive. Love your work.
Thank you, I really appreciate it 🙂
Hi Adrian,
Is it possible to use in raspberry pi 3B+?
You are doing a great work in computer vision
Thanks for the tutorial.
Try it and see! 😉 But yes, the code will work on a Pi, just use a faster tracker like MOSSE as the Pi is slower than a normal laptop/desktop.
Thanks for the reply it works well with MOSSE. It also works with Picamera too.
Thanks for following up, Sudharsan 🙂
Hi Adrian,
(may be my question is not right place) in your posts on object detection I’ve read, the prototxt and model files are already available, attached for download. So can you make a post to show how to train a neural network using some well-known frameworks (I found you prefer caffe). Actually, I just want to see if I can experience “like-a-boss” feeling in making complete neural network: I capture (collect) images as many, throw them in the network I prepared, get a cup of coffee for waiting, export into model files and lastly apply these files to your post.
Say I’m insane if you like!
Hey Atom — I actually cover how to train your own custom models inside Deep Learning for Computer Vision with Python. That is by far the most comprehensive book to training your own deep learning models. Otherwise, I think you’ll have fun playing with this tutorial.
Thanks so much, Adrian, I truly appreciate that
Happy too, Atom 🙂
dear Adrian:
I am a Chinese studen.I am sorry that myEnglish is not good enough.I have a question that
how can i input my own vidio and where i should write my path to the vidio flie.
Because of my poor English,I think too much time.if you can answer me,i am very aprriciated
Markus
There are no modifications needed to the code. Just update the
--videoswitch to point to your video file when you execute the script:$ python opencv_object_tracking.py --video path/to/your/video.mp4amazing！thank you so much ,i have succeed in run the code with my local video !
i am very appreciated for your help!
Adrian;
Would this work on a raspberry pi with pi camera?
Yes, but you would want to use one of the faster detectors like MOSSE as the Raspberry Pi isn’t as fast as a standard laptop/desktop.
Hello Adrian,
What is the best algorithm to distinguish head covered (scarf, Cap, etc.) from uncovered head? For detection and tracking purpose.
There are a few ways to approach this:
1. Use a face detector to detect a face and then run a classifier on the face region to determine covered/uncovered
2. Train an object detector to detect covered/uncovered
The first method is easier to implement but will be more prone to missing faces altogether as sometimes covered heads/faces are significantly harder to detect if the model was never trained on those images.
Hey Adrian,
Thanks for the tutorial, I am trying to run the code and whether I give no video argument and just use my webcam or if I give a path to a video on my computer whenever I press ‘s’ to draw the bounding box I get this message:
Traceback (most recent call last):
File “opencv_object_tracker.py”, line 120, in
showCrosshair=True)
TypeError: Required argument ‘boundingBox’ (pos 3) not found
I’m not sure what I am doing wrong since I just used your source code. If you have any tips to fix this please let me know. Thanks!
Hi Gabe, just to confirm a few things:
1. You used the “Downloads” section of the blog post to download the code rather than copied + pasted, right? Sometimes readers accidentally introduce errors if they try to copy and paste which is why I always recommend using the “Downloads” section.
2. Which version of OpenCV are you using?
Hi Adrian,
my experience is that this happens for all the trackers I’ve tried you say will compile under 3.2 (my version of OpenCV). As soon as “s” is hit, the program terminates with the above error.
Python 3.5.2
OpenCV 3.2
Trackers tried: kcf, medianflow and mil
I am using your code (downloaded) and one of the .mp4s provided
Cheers,
Rob
Hey Rob — I’ve had a lot of issues with OpenCV 3.2 and the object tracking algorithms. I would really recommend you use at least OpenCV 3.3 and ideally OpenCV 3.4+.
Greetings!
This is one of the great websites for OpenCV and Python implementation. I’m thankful for Adrian. I also purchased the book package earlier.
I’m not a software engineer, but I’m trying to learn.
I prefer Zoneminder as I can control all data.
basically, I’m looking for a integration or creating a plugin that would in realtime (or near realtime) do face recognition using OpenCV. I would also like to incorporate other types of detection such as license plate detection, obj3ect counter etc.
I have a budget established for this. If anyone is familiar with Zomeminder and can develop something like this, pls let me know
Thank you for picking up a copy of my book, I hope you are enjoying it so far! Regarding your project send me an email and we can get your job listed on PyImageJobs.
Adrian
Thanks for the post. Very interesting. Came across an error on the construction of the argument parser…..
usage: __main__.py [-h] [-t TRACKER]
The error is related to your command line arguments not being supplied. If you’re new to command line arguments and how they work with Python, that’s okay, but make sure you read this post first so you can get up to speed.
I am unable to create a Frame and track the same on the video not sure why, am on ubuntu and installed OpenCv through Anaconda framework.. even during first run i got an Attribute Error which I resolved on reading KhanhPhamDinh (thanks).
Any help is most welcomed.
When you say you are unable to “create a frame” are you saying that the Python script does not open a window? That it crashes immediately with another error? If you can be more specific I can try to provide a recommendation.
thanks a lot Adrian. good job again. What is the differences between object tracker and particle filter in opencv?
You can use particle filtering as a form of object tracking; however, particle filtering methods have a number of other assumptions, including Monte Carlo-based algorithms, dynamic systems, and random perturbations.
Hi Adrian.tnx for your great post.I have a question and I hope you can help me.I want to build a people counter that installed at the entrance of store.what’s your best solution for doing this?face detection?remove background of video?frames subtraction?…
I’ll actually be covering how to build a person counter later in this object tracking series, stay tuned!
Thank you so much.Adrian how long does it take?less than one month or more?
The person counter post will be published this Monday.
unable to track the car using any of models. While executing the program the Boston dash cam video is launching but as mentioned i couldn’t able to track. need your support
Make sure you click the window opened by OpenCV (not your terminal window) and then press the “s” key on your keyboard to select the ROI.
Great post! Just wondering if you have any advice about how one might go about working with trackers in OpenCV on the Android platform.
Sorry, I do not have any experience with OpenCV on Android. I hope another PyImageSearch reader may be able to help you out!
For anyone else interested:
http://answers.opencv.org/question/197296/android-object-tracking/
Hi, Adrian, this is my first comment :), first of all thanks for the your web site, I’ve learnt many things about image processing.
I have question automatic tracking initilization, for this post(OpenCv Object Tracking)
You said “Of course, we could also use an actual, real object detector in place of manual selection here as well.”
What is your suggestions instead of selecting ROI? Is there a new post about it?
Thanks for all 🙂
This post shows you how an object detector can automatically detect ROIs and track them.
hi Adrian,
I am getting the following error while running the file object_tracker.py
error: the following arguments are required: -p/–prototxt, -m/–model
You need to supply the command line arguments to the script when you’re executing it (which you’re not doing). If you’re new to command line arguments I recommend you read this post to get up to speed. From there you’ll be able to execute the script.
Thank You v much sir,
The mistake was a very basic one and I am great full to you that you guide me in a good way. you are doing a great job. keep it up. Most of the time I was stuck in the same problem but now it will be solved.
Hello Adrian,
thanks for continuous nice work.
I tried CSRT tracker, tracking result seems good; however when the target disappears from the screen and then it shows again , it isn’t detected
Any support to control this situation ( losing target)
Thanks again.
Hey Sam — you can actually combine the techniques from this post with the previous post on centroid tracking. I’ll be providing an example of such an implementation in next week’s blog post on people tracking.
Thanks Adrian,
in your post it was face detection on a pertained model, for the tracking example the object am tracking is not pertained so how can i achieve it
If you have not trained an object detector to detect your object you will not be able to detect it which means you will not be able to track it. What object are you trying to detect?
the idea is to select any object and track it in the video ; can i train the system to detect a selected object on the runtime ? haar cascade or any other algorithm
No, the actual object detector needs to be trained ahead of time. Once you’ve detected the actual object you can track it.
Hi Adrian, thanks for such a good tutorial. But when I was trying to run the code, there’s nothing happen. What will be the possible error? I’m using python 3.6 with OpenCV version 3.4.1.
Thanks in advance!
Could you provide more details on what you mean by “nothing is happening”? Does the script automatically exit? Is an error thrown? Does the script run but no output displayed?
Hi Adrian,
thanks a lot for your tutorial. I have problems with detection of object in complicated conditions (dark object in dark region of the scene). I’m also trying to detect, when the target disappears the scene. I’ve noticed that together with the detected position the tracker also gives the ‘success’ information which in unfortunately always True (even for the case when tracking failed). In this thread http://answers.opencv.org/question/110756/how-to-reset-or-update-kcf-tracker-roi-when-it-lose-the-target/ is discussed that one of the solution is to modify opencv function to give response of the tracker instead of binarized True/False information. Unfortunately, I’m not proficient enough to make such changes of the code. Is there another possible way?
Thanks a lot.
Jindřich
Unfortunately there is not an easy solution to this problem. In fact, handling the case when object trackers lose track of an object is one of the hardest components of an object tracking algorithm — I would argue that it’s far from solved. It’s not an easy solution and unfortunately for any other level of granularity you would need to modify the OpenCV source code.
I was able to get it to work with the pi camera by changing line 51 from
vs = VideoStream(src=0).start()
to
vs = VideoStream(usePiCamera=1).start()
Yep, that’s all you need to do!
At the end of your post you say CSRF instead of CSRT a few times.
Thanks Jeff. Post updated 🙂
Traceback (most recent call last):
File “deep_learning_with_opencv.py”, line 114, in
showCrosshair=True)
TypeError: Required argument ‘boundingBox’ (pos 3) not found
how to solve this!!
Hey Vipul — which version of OpenCV are you using?
3.3.1
“csrt”: cv2.TrackerCSRT_create,
AttributeError: ‘module’ object has no attribute ‘TrackerCSRT_create’
how??
You should take a look at Figure 2, Vipul — you need at least OpenCV 3.4 for the CSRT tracker. Since you’re on OpenCV 3.3 you should comment out the CSRT tracker and the MOSSE tracker, then the code will work for you.
Any advice on tracking when the background colors are similar to the object being tracked? I have been experimenting with tracking a green ball. The CSRT tracker does a decent job indoors but when i move outside and roll it through the grass it fails. I can detect the ball but the tracker loses it and ends up locked onto a spot on the grass.
How are you detecting objects in the image? Is it via just simple color thresholding? If so, you should use a more advanced method like a dedicated object detector (HOG + Linear SVM, SSDs, Faster R-CNNs, etc.). I have an example of such an object detection + object tracking pipeline here.
Yes, I’m currently just using color thresholding to detect the green ball. To test the tracker I put the ball on a high contrast surface and the ball is detected. I then use the detection to start tracking. If I then roll the ball off the high contrast surface onto the grass, the object detector fails.
I mean the object tracker fails when the ball leaves the high contrast surface, not the detector.
That’s not too terribly surprising given depending on how of a difference the high contrast surface is (comparatively) and how fast the ball is moving (the faster it moves, the harder it will be for the trackers to work properly). The solution is to use a hybrid approach that balances tracking with detection — you can find the method in this post.
Hello Adrian,
Actually i am in python3 and when i execute the below code it is showing me the error.
File “”, line 8
(major, minor) = cv2.__version__.split(“.”)[:2]
^
SyntaxError: invalid syntax.
I am new to python and when i googled i found that “the tuple is deprecated and we need to split” something… bla bla..
Can you help me on this error?
Make sure you use the “Downloads” section of this blog post to download the source code rather than trying to copy and paste. My guess is that you introduced an error in the code when copying and pasting or trying to “code along” with the post.
Hello Adrian,
I am getting error when i try to run the python file with –video arguments. Can you please help me on this. I am on windows
You need to supply the command line arguments to the script. If you’re new to command line arguments, that’s okay, but make sure you read up on them first.
Thanks a lot Adrian
A couple of question,
1-How can I make the tracker return false when the object leaves the scene?
2-how can I deal with an object that changes its dimensions in frame?
Best Regards
1. The tracker itself will return “False” via the
successboolean. You can also track the bounding box and once it hits the boundaries of the frame you could mark it as “leaving the scene” as well.2. You would want to use an object tracking method like CSRT which works well in changes in scale.
Hi Adrian,
Is it possible that I follow the object I already defined? So when the camera is turned on, can I find the object I defined? If possible, how can it be done?
Hey Eyup — object tracking and object detection are two different types of algorithms. What type of object are you trying to detect?
Hi Adrian, i am using Open cv 3.4.2 but still getting below error-
tracker = cv2.TrackerBoosting_create
AttributeError: module ‘cv2.cv2’ has no attribute ‘TrackerBoosting_create’
Your question has been answered in this reply.
Hi Adrian, I followed your tutorial to install Opencv on Raspberry and your people counter as well, but I made some changes on the code: I used substraction background to detect movement on the video and FindContours for enclosing it in a rectangle. However I use CentroidTracker which you made in the previous tutorials
Though I made those changes, still is very slow on Raspberry PI 3B so I have to implement MOSSE to increase the speed. I saw many implementation about MOSSE algorithm but it always use ROI to track the object. My question is: What is the ROI output? Is a coordinate?
Because my plan is:
# select the bounding box of the object we want to track (make
# sure you press ENTER or SPACE after selecting the ROI)
#initBB = cv2.selectROI(“Frame”, frame, fromCenter=False,
# showCrosshair=True)
initBB =(x,y,x+w,x+h)
# start OpenCV object tracker using the supplied bounding box
# coordinates, then start the FPS throughput estimator as well
tracker.init(frame, initBB)
But it doesn’t work very well.
PS: initBB need to be an array to have all the coordinates of each contours. I wiil put again my question but i don’t know if my question has been deleted or it just doesn’t appear on the page.
I realize that I would have a problem if a use MOSSE algorithm with findcontours because in each frame the function FIndContours will find the same contour until it disappears and I will pass to MOSSE algorithm and it will count the same person many times. So I m considering to do people counter on C++ to be faster tahn python
Hi. I’ve getting the warning from “imutils.video import VideoStream” (line2), error: No module named imutils.video. But i’ve installed imutils as you did before. Can you tell me what it means? Thx btw. Keep work!
It sounds like you have not installed “imutils” correctly. Please make sure you double-check.
Hi Adrian, I am learning CV lfrom your blogs, they are brilliant.
However, a error came out when I run this script, it happened in line 120: tracker.init(frame, initBB), error is “AttributeError: ‘builtin_function_or_method’ object has no attribute ‘init'”, I am using opencv 3.4, could you please help me out ?
By the way, HAPPY WEDDING !
Hm, I’m not sure what may be causing that error. Are you using the code from the “Downloads” section of this blog post? Or did you copy and paste? Make sure you use the “Downloads” section to avoid any accidental errors.
this actually happened to me too, turns out i forgot to add ‘()’ after cv2.TrackerBlaBla_create, so you might want to look over your syntax again. Hope this helps, cheers!
Adrian,
Yet another awesome blog.
Question, if the object leaves the view panel, how does one get rid of the remaining bounding box that is left on the edge?
And let’s say, that we decide that after awhile we want to track a different object instead? The current setup allows me to open another panel and make a new selection, but it doesn’t seem to track the new selection, nor does it destroy the previous object being tracked.
Hey Rob, that’s partially a limitation of each object tracker algorithm. Some object tracking algorithms do a poor job of detecting if an object has left the scene of view. Instead, you should try to monitor the (x, y)-coordinates of the object and if they “hang around” the border of the frame for too long you can mark them as “disappeared”.
Is the issue that the tracker is not reporting failure, or that the tracker reports failure but the green box is still on the frame?
If the issue is that the CSRT tracker does not report failure, I just posted about that below.
Hi Adrian,
I would like to program Stable Multi-Target Tracking in Real-Time see
https://www.youtube.com/watch?v=InqV34BcheM
Any advice would be appreciated very much.
Hi，Adrian:
I use the script and try it,everything is OK, but if I re-select ROI, than the tracker.update() function always return False, so it can not modify ROI and re-track it.
And I also find even I modify the ROI, but sometime it still show origin ROI.
So i want to know how I can re-select ROI and make if effect immediately.
thanks.
I am using Python 3.7 with OpenCV 3.4.3. Everything is working, I tried the examples. I noticed CSRT is more accurate, as was mentioned in the blog post. However, when I used CSRT , the tracker never fails, even when the object is completely out of the frame, it does not report any failure.
When I use other algorithms like KCF, it works as expected and reports failures as expected.
So I am not sure why CSRT is not reporting any failure.
It’s all in the underlying mechanics of the algorithm itself. Some algorithms are very good at reporting failure, others are not. There is no true, perfect object tracker. We pick and choose which one to use knowing there are going to be failures.
I am just curious. How do visual tracking community handles(FP or FN??) nans for calculating recall vs success rate?
how can i run this code with example video as input
The blog post already shows you how, Hashim. Just supply the path to your video file via command line argument:
$ python opencv_object_tracking.py --video dashcam_boston.mp4 --tracker csrtHi Adrian,
I am unable to pass the path through command line. Can you please help me out, and also the camera opens a frame but it’s not capturing anything, please do help.
If you are new to command line arguments I would recommend reading this post first.
Hi Adrian,
Thanks for your useful tutorial.
Is there any solution for tracking object via multiple cameras?
Best regards,
That might depend on your use case. Are you tracking multiple a single object across multiple cameras with different viewing angles?
Hello Adrian,
Another fascinating video! Do you have instructions on using an IP PTZ camera connected to a Raspberry Pi(3) and opencv to grab video frames? Then control the PTZ for object tracking and other opencv tasks over the net. I have seen some code using urllib, but I do not know what would be the best way to get started? Thank you for any suggestions.
Sorry, I do not have any tutorials for an IP PTZ camera. If I do cover that in the future I’ll be sure to let you know!
thanks a lot!
Thanks for the video!
Is there a way to automate this? I’m thinking of using this on a portable device that would run as a loop, without a user input. So how would you recommend initializing the object recognition without drawing the box manually?
You would need to apply a dedicated object detector to first detect the object. From there you can track it. I provide an example of such a system in OpenCV People Counter blog post.
hi I have a problem currently using raspi and i get this error while running the code “error: the following arguments are required: -p/–prototxt, -m/–model” how do I fix it. Im new in Raspi btw.
It’s okay if you’re new to the Raspberry Pi, but you need to read my tutorial on command line arguments first. From there you’ll be able to execute the script.
Congratulations for another amazing tutorial!!!
Adrian, why do you use frame[1] after reading from a video file? In other tutorials from you, you just use frame = vs.read()
Thanks again!
Because I need to check if we’re working with (1) a video file stream or (2) a webcam stream. The two are handled differently.
Hi Adrian,
thanks for the tutorial, it’s working great but i need to ask the question that it is running slow on my PC, it take 4 to 5 second per frame so what should i do to increase the speed of yolo,
I will be very thankful for your advice
To achieve true real-time performance you would need to use your GPU. Unfortunately OpenCV’s “dnn” module does not yet work well with many GPUs, but more GPU support is coming.
you choose the ROI by pausing the video with S and then making a window on the object you want to track
what i need help in is :: (i hope you help me)
1- i want to choose the object without pausing the video nor selecting a window i mean the selection window is static and i just click left click to track
2- everytime i click right click on another object it deletes the previous one and track the new object
3- control the selection window size by the mouse wheel|| here is the code im using now
That is certainly possible but keep in mind that OpenCV’s GUI functionality is extremely limited. You should use a dedicated GUI library like Tkinter, QT, or Kivy.
Hi
What mechanism would you recommend to delete a tracker once the object moves out of the frame. I tried with CSRT and it the tracker is maintained and returns success whenever I update even if the object has moved out of the frame
That’s likely a limitation of the tracker itself. Once you detect an object at the end of the frame (based on the predicted coordinates and image dimensions) you can insert code logic to ignore the tracker using “if/else” statements. You could even use the
delPython keyword if necessary.Hey – love the tutorial – great for newbies like me.
A few questions:
– is there any benefit to forcing some contrast on the video feed before the tracking is applied?
– is there a way to save off the captured region so that it can be used again ?
Thanks!
1. That really depends on your exact use case. Typically we try to train object detectors with varying levels of contrast to help improve the detection, not adjust contrast at test time.
2. You mean save the ROI itself? Just use
cv2.imwriteCould you descript how to use tracking to obtain orientation of object? Thanks for help 🙂
We are neither able to fetch the local video nor access the webcam. Only the webcam frame is opening but no image is capturing. Please respond as soon as possible.
Can I know that which method (out of point-based, kernel-based and silhouette based) we are using for object tracking.
Awesome article and as usual Adrian your website is a constant go to reference on all things CV / OpenCV and python! 🙂
I am using the python api for tracking and mixing it with an SSD object detector to handle the initial detection of objects.
When the slower neural network gives me a box I match it with existing boxes and do some non-max suppression and then resume tracking those boxes or add new trackers for new ones.
However when I have a new box from the SSD which I consider a sort of new ground truth for the trackers I want to update them with this information.
As per this post it seems this isn’t possible in Python even though the C++ API has it.
http://answers.opencv.org/question/189105/tracker-update-with-rect-in-python/
Instead I have to delete the tracker and create / init a new one.
While this works fairly well there is quite a bit of CPU overhead in constantly creating new trackers which is causing problematic performance.
I am using CSRT and I tried a cheaper tracker like MOSSE but it seemed like the number of inits went up possibly because the tracker is poorer so theres less quality matches etc. Either way if I was able to update the box in an existing tracker rather than re-init im sure it would be faster.
Does anyone know about this? It seems ridiculous. I would consider making the changes myself if it’s as trivial as updating the python API in opencv and recompiling but surely its not there for a reason?
Thanks!
Thanks for the kind words, Madhava. Have you seen my previous tutorial on building an OpenCV people counter? In that post I provide my implementation of balancing object detection with tracking. It should be able to help you with your project.
I am wondering if we want to use the tracker for gray scaled images, do we need to set some parameters in ‘create’ method (which I couldn’t do or maybe I don’t know how to), or just supplying gray scale images to ‘init’ and ‘update’ methods are sufficient.
You would need to convert the image to grayscale first. From there you can supply the frames to the tracker.
Thanks for this wonderful article. I’ve been trying to use this program with an ipcamera. I can successfully connect, start the program, and draw a box around the object I want to track in real time. However, once I press enter, the live stream cannot keep up with the actual real-time video on my ipcamera. If possible, can you give me an advice on how to fix this issue? I’d greatly appreciate it!
Sorry, I don’t have any tutorials on IP camera streaming. I may consider covering them in the future. Best of luck with the project!
Hey Adrian,
Nice blog, will definitely follow your blog more and try some of your implementation out.
I got a question that I would like to asked, not sure if this problem had been answered already (if so please point me to the right direction). So my question is:
If I trained my object tracking algorithm to recognized my face, and let say then I implement the algorithm in a drone to follow me from behind while I ride around. Since I trained the algorithm to learn to recognised my face, but not the back of my head (and to be honest the back of the head of anyone with the same hair colour and length looks is highly similar), and let say the clothing I’m wearing has different colour schemes at the front and back (i.e. white in front, black at back), how can the algorithm be able to identify me from behind in real time and without the need to wait for it to re-train itself to learn what do I look like from behind?
I personally don’t like using “pure computer vision” for these types of problems. It would be much easier for your drone to track you in other ways, mainly physical sensors emitting a signal for the drone. Computer vision can be used to facilitate better tracking, such as applying a person detector every N frames and then correlating hat with sensor data, but I really wouldn’t use “pure computer vision” here.
I still cannot reset the bounding box by clicking on “escape” button. However, I could manage to reset a bounding box by adding another key. so if key == “r”, I will clear and initialize the tracker object:
tracker.clear()
tracker = OPENCV_OBJECT_TRACKERS[trackerName]()
then taking the bounding box by cv2.selectROI. But, I am still interested to know why escape doesn’t reset the bounding box. Any thoughts? Thanks!
I’m honestly not sure. What version of OpenCV are you using?
Thanks for the great tutorial! I also used your tutorial on installing OpenCV on raspberry pi. They were both excellent. I have a Raspberry Pi 3 B+. I am getting ~5-6 fps with Mosse with these videos and the default resolution of 500×500. Aside from reducing the resolution, do you have any tips on increasing the fps? Perhaps using black and white instead of color (I’m a noob so apologies if this is an idiotic suggestion). I have a Kingston Class 10 64 gb micro SD and I am also wondering if my SD card could be slowing things down a bit. I find the Raspberry Pi to be a bit laggy even when typing commands in the terminal. Is this normal?
The color or the object itself is not going to improve the FPS of the object tracking — the computation is all contained with the object tracking algorithm. You likely won’t be able to get more than that performance out of the Pi without reducing image resolution.
sincerely I want to thank you for the clear knowledge you are delivering
Thanks Abdullah!
Hi Adrian,
Many thanks for all great tutorials!
My question: is setting the trackers parameters available in python?
I am using opencv 4 now, I have installed it in python and c++
I need to set the psr_threshold in the CSRT tracker, but I only managed to do it in C++ not in python.
TrackerCSRT::Params param;
param.psr_threshold=0.090;
track = TrackerCSRT::create(param);
Can I do something equivalent in python with opencv 4 or any previous version?
You’ll want to take a look at OpenCV’s documentation. Open up a Python shell and type
help(cv2.TrackerCSRT_create)— that will give you the available methods on the object. From there you can explore the documentation.I am getting an error saying – – video not recognised…please help
That sounds like an issue with your command line arguments. If you are new to command line arguments you should read this tutorial.
hello , adrian , i hope that you are doing well , and let me thank you for sharing your knowledges . i am going to use kcf tracker for custom object tracking purpose , the detector is based on yolo object detection. i tried to use opencv built-in trackers but the trackers suffer from heavy occlusion, since my goal is to overcome high occlusion. i want to use “implemented kcf tracker ” instead of opencv built-in tracker. will the performance be improved if i do that ? your suggestion will be helpful . thanks in advance.
I would suggest trying with the KCF tracker. Without knowing more about your dataset/video/end goals it’s hard for me to definitively say. Give it a try and see, let your empirical results guide your decisions.
Hi Adrian,
Thanks for a great post.
Could you please indicate how I should do if I want to track multiple squares in the same video?
Found your other vidoes :p
🙂
how can we do that?
found it.
https://pyimagesearch.com/2018/08/06/tracking-multiple-objects-with-opencv/
Once again, great stuff!
So I want to detect a person, and/or an object, say a ball in a video stream. Once detected, I’m sending an alert to a third party platform. the issue I’m having is that if I’m running the recognition/detection at 15-20 FPS, I’m seeing about that many alerts per second, as I get alerted for every detected frame. I’m running this on a reasonably powered machine with Nvidia Quadro P3200, so I have some cuda power available. I’ll also have up to 5 cameras on this system attempting to detect at approx 15 FPS each
What is the best way to address this issue so that it alerts once and then perhaps tracks. No one will be able to draw boxes, it needs to recognize and then track. I don’t necessarily want to alert on the first detection because the confidence may not be high enough and maybe higher in subsequent frames Maybe it selects the best detection/recognition in a given time frame and chooses that?? Been struggling with this issue for a while. Any thoughts?
Hi Ray — have you seen this tutorial where I show you how to detect + track? That seems like it would be the best option for you.
Hi, Adrian.
First of all, thanks for the excellent tutorials. I’m running the script on a Raspiberry Pi, using Mosses and detecting objects by PeopleCascade. I’m not using the Raspiberry camera, I use a video file. As expected, even with the use of Mosses and Cascades, tracking is very slow. So in another tutorial, I saw that it uses threads to boost FPS through the VideoStream library. However, at line 56 of your script, you use vs = cv2.VideoCapture (args [“video”]) instead of vs = VideoStream (args [“video”]). start (), which is the threaded version. I modified line 56, using vs = VideoStream (args [“video”]). start (), but the “invalid number od channels is input image” error occurs.
Do you have any idea why this error occurred? Or, why did you use vs = cv2.VideoCapture (args [“video”]) instead of vs = VideoStream (args [“video”]). start () on line 56?
Thank you very much.
If you are using a video file as an input you should be using the “cv2.VideoCapture” function or the FileVideoStream class.
Hi, Adrian.
Thanks for your reply. But VideoStream() is threaded and cv2.VideoCapture() isn´t. In your tutorial “Increasing webcam FPS with Python and OpenCV”, in class WebcanVideostream (threaded), you have the parameter src and you say “if src is a string then it is assumed to be the path to a video file.”. In the constructor __init_-(sef, src=0) of WebcamVideoStreamClass you call cv2.VideoCaoture and capture threaded frames. And in your VideoStream class, in line 23, you instantiate WebcamVideoStream if a pi camera is not used. So, why is not possible use VideoStream (src=video file path)?
Thanks a lot
Thanks
The short answer is that with the threaded version it will run continuously and discard frames from the video file if they are not processed fast enough. The “FileVideoStream” class implements a queue/buffer to ensure each frame of the video file is processed.
Hi, Adrian.
Yesterday I left a message here. I have another question. Do you think that using PeopleCascade for people detection and Mosses for tracking is the best option when the hardware is a RaspIberry PI?
Thank you.
On computationally limited devices, such as the Pi, MOSSES is the most efficient tracker (at least in terms of computation). Haar cascades are also very fast and suitable for the Pi, provided they give you the accuracy you desire.
Hi, Adrian.
Sorry for another post here. The error issued when using vs= VideoStream(src= args[“video”]).start() is “Insufficient Memory” and it occurs in cv2.imshow(“Frame”, frame). I think it´s a problem in the thread when running in Raspiberry Pi. Any idea on correcting this error?
Thanks a lot.
Your Pi is running out of memory. Try resizing your input images/frames before processing them.
Hi Adrian, Thanks for the wonderful post. I’m trying the following steps,
1) Detect object using detector, 2, Track using tracker 3. After 15 frames repeat 1.
Meaning, i’m refreshing the bounding box from detector every 15 frames. The problem is that the tracker once initialized with a bounding box cannot be re-initialized with a new bounding box. The Tracker::init() function (tracker.cpp from opencv_contrib) itself is preventing re-initialization.
I’m using Opencv 4.0.1. Is this a limitation of opencv tracker itself? One idea I had was to delete the tracker instance every 15 frames and create a new instance of the tracker and initialize with a new bounding box from the detector. The problem is that deleting the class instance (in CPP) using “delete tracker” is crashing. I tried KCF, MOSSE, CSRT and all are behaving the same.
Based on my search, this seems to be a problem reported by many.
Any thoughts? I’d like to know if we have a solution before touching the OpenCV code.
Thank you.
Hey San — I think you need to follow this tutorial which covers your exact question.
Hi,
Where does Optical flow methods fit in like Lucas-Kanade ? is tracking via optical flow more efficient?
I’ll try to cover optical flow in a future tutorial.
Hi Adrian, thanks for the really cool and detailed post. I had great fun following this, and I actually learned a lot! What would you recommend as a next step? I have no background in object detection and tracking.
Hey Jessica — if you are brand new to the world of computer vision and image processing I would recommend reading Practical Python and OpenCV. That is the first book I’ve ever written and is designed to help you get up to speed with OpenCV, paving the way to building more advanced projects.
Great tutorial my friend! Keep up the good work! Thanks to you I was easily able to get my Raspberry Pi 3b+ up and running opencv and tracking objects. I was just wondering if it is possible to select a rectangular ROI from a center point and a defined box size. I.E. you use a mouse click to define the center and have a predefined height and width for the box. Many thanks!
It’s possible but you would need to implement that functionality yourself. See this post on how to work with OpenCV mouse events.
Basically you’ll want to detect when the mouse is clicked, capture the (x, y)-coordinates, and then from there you’ll be able to derive the starting and ending box coordinates based on the pre-defined width and height.
Hi Adrian,
Many thanks for these wonderful tutorials! They’ve been tremendously helpful.
I am trying to import a ROI from ImageJ (.roi file) and use it to track a feature in a video of mine. It doesn’t seem as though Tracker can handle non-rectangular bounding boxes in the input to tracker.init(). Do you have any suggestions as to how I can handle this?
You need a rectangular bounding box. If you don’t have one, compute the minimum enclosing bounding box for your coordinates.
Hi Adrian,
Thank you for your response! If my ROI is a quadrilateral and I know the four vertices, do you have any suggestions of other OpenCV functionalities that could handle this kind of input ROI?
The “cv2.boundingRect” function might work, but if you already know the four coordinates, you can just find the top-left, and bottom-right most coordinates and derive your bounding box.
Adrian,
I have been working on a school project for months now tracking a small drone in real time, I was curious to see what method you recommend on the best way to go about this. I have used many many different methods and each have their own ups and downs. Just reaching out to other sources to see what other methods I could explore. Thanks!
It’s really dependent on your exact project and what objects you are tracking. There is no “one-size-fits-all” solution. It sounds like you’re on the right track though, you need to experiment with each tracker and balance the pros/cons for your particular application.
Thank you for the response! One other question, we had decided to invest in a better camera and ran the camera calibration, there is a noticeable difference in the picture. What other benefits should/could we expect in a higher quality camera and using the camera calibration?
I’m not sure I understand the question. The higher quality camera you have, ideally the higher quality camera, less artifacts in the image, and better the resulting calibration.
I have a question. Is there any way to reselect a different ROI to track without restarting the program? I have tried to implement this but whatever I do I end up tracking the original object.
Hey Adrian,
Thank you for the blogs you write on OpenCV. It helps me alot to get proper touch with opencv and your way of explanation is excellent.
I am doing machine learning based project on ”conjestion detection in traffic”.
Adrian can you suggest me the best tracker for vehicle detection in traffic in which i can perform fine-tunning on the model according to traffic in my city.
Thanks!
You would want to try each tracker, visualize validate that its performing correctly, and then pick the one that works best for your project.
Really cool project! Its amazing what OpenCV can do in basically 100 lines of code.
Thanks Yoichiro!
Dear Adrian. This is my second reply. First one was lost..
What happens when I loose tracking? I cant reinitialize tracker. Should I create new? Does it affect memory? Can I discard the old one? I read a reference (for c) about tracker.release() but its not working here. I also read about IsInit flag ,but what about python? Thank you
Hi,
Are you aware of any optimized implementations of CSRT that would run raspberry pi? I’m looking for something that could run at 30fps in Pi 4.
Hi adrian,
Can i use this code on raspberry pi and interface stepper motor to continuously track the particular selected object in real time ??
Yes. Raspberry Pi for Computer Vision covers that exact project.
Hi adrian,
can i combine your pan tilt code with this code and run it on raspberry pi.
Take a look at Raspberry Pi for Computer Vision which will show you how to do exactly that.
Hi Adrian!
Many thanks for the great tutorial! As always!
Just one question: I am trying to save output file (.avi) in output folder with the help of args constructor. Till now, I managed to save the output file with green bounding boxes in the frames. Tracker (csrt) worked quite well!
Is there a chance to save in the output file also manually selected first blue bounding box, so that the output file looks like your gif file above?
Many thanks in advance!
Elena
You can use the “cv2.rectangle” function to draw the blue bounding box on the first frame if that what you are asking.
Hi adrian, wow you are truly awsome.
i have a problem, how do i run the code? can i do it via Picharm? or only via terminal?
You can use PyCharm but you’ll want to configure your command line arguments. I personally suggest just using the command line/terminal.
How can I increase my video stream to full screen?
Amazing tutorial! Thank you very much!
Thanks Medison!
Hi adrian, you are truly awsome.
I have a project related to object detection and tracking. Can you help me on how to combine both the object detection and object tracking phase into a single script?
I hope you can help me. Thank you so much !!
I would suggest you follow this tutorial which addresses your exact question.
can draw bounding box be implemented on flask? so the application run as web based
Hi Adrian! I hope you will be fine. You are doing a great job.
I want to ask that can we calculate the speed of an object (preferably a fast moving object) using this approach.
Like we have a high resolution and high speed camera installed somewhere on the side of road and we have to calculate the speed of a car in between two points ( say from point A to point B ) in real time and one more thing that is detection of car’s number plate or aircraft tail number using the same data set taken from the camera.
Can you please guide me about this thing that how can we calculate the speed of a fast moving object in real time and detection of number plate or aircraft tail number in real time.Even if you have some knowledge about this thing I’ll be honored to have it.
Waiting for your reply.
Regards
I would suggest you use this vehicle speed detection tutorial as a starting point.
can we combine it with deep learning based object detection? so after detection we use those trackers algorithm
Yes, refer to my people/footfall counter tutorial.
Hey Adrian! Need a suggestion in this OpenCV Object Tracking code. While tracking an object, If the object gets out of the view still the bounding box is there on the screen. So I want to ask how should I remove the detection bounding box if the object is no more in the view.
You can check to see if the bounding box coordinates are close to the border. If they are there for N frames, remove the tracker.
Hi Adrian,
Big fan of your blogs, always straightforward and detailed.
I am curious if it is possible to continuously repeat the video and tracking until I quit the program. I have time-lapse videos of human cells over a couple of days so the videos are very short and it is difficult to evaluate how the tracking algorithm is working, as it immediately closes after the first ‘loop’
Thanks in advance!
Brett
I’m not sure what you mean by “continuously repeat the video and tracking”. Do you mean the script automatically exits?
Yes, sorry about the phrasing. I tried tweaking the code to get the script to keep running but with no success (still relatively new to python). Any suggestions would be great! Thanks again.
Basically, you would:
1. Create an infinite loop
2. Inside that loop you would open a pointer to your video file
3. Loop over all frames to the video file and apply your tracker
4. When the video ends you go back to #1 and then the file is reopened and the process repeats until you exit the script
It’s okay if you are new to Python/OpenCV but I would recommend reading Practical Python and OpenCV to help you learn the fundamentals first.