<jupyter_start><jupyter_text>IntroThis is a quick and dirty notebook for showing how I would use OpenCV to detect and track motion in a room. I will load all the code from proper Python files and leave what I consider to be elucidating comments in the markup secetions of this notebook.  Imports and ConstantsFor now, we are going to load cv2 and numpy. I will also define some constants here (it is proper to capitalize all letters in a constant, even though Python does not have constants). My constants are the window name (which I need to create handles to my GUI) and the Display Denominator. (Some cameras produce a resolution (or image size) that is very simlar to the display resolution (size). I scale my frames down so the window is smaller)<jupyter_code># %load -r :6 src/main.py
import cv2
import numpy as np
import time
WINDOW_NAME = 'win'
DISP_DENOM = 2<jupyter_output><empty_output><jupyter_text>Creating GUI and a Probablistic Background SubtractorThe approach I propose for detecting motion in a frame is via probabalistic background subtraction. There are many background subtracting algorithms (OpenCV implements 4), but I think the K-Nearest Neighbor (KNN) implementation is the best.The `BackGroundSubtractorKNN`, when applied to a frame, produces a mask. I will apply this mask to the frame, and thus produce only the foreground that is interesting.Here, I setup the camera, background subtractor, and the trackbars. I am going to use the trackbars to play around with the attributes of the background subtractor. The background subtractors in OpenCV have many parameters, but I found these two to be the crux. Checkout the OpenCV reference to learn about the other parameters, or the papers on the background subtraction algorithms.**Important note**: Much of computer vision is tweaking parameters to different situations. I frequently use setups like the one we are creating here to tune parameters. Different cameras, room lighting, room coloring, and use cases need different parameters.**Important Note**: I initialize an empty matrix to store the product of the mask and the frame. Do stuff like this when you are consistently doing the same operation and you know what the output dimensions are going to be.<jupyter_code># %load -r 7:22 src/main.py
cap = cv2.VideoCapture(0)
fgbg = cv2.createBackgroundSubtractorKNN(
    history=50, dist2Threshold=100.0, detectShadows=False)

cv2.namedWindow(WINDOW_NAME, cv2.WINDOW_NORMAL)
# cv2.startWindowThread()
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)//DISP_DENOM)
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)//DISP_DENOM)
display_img = np.empty((height, width, 3), dtype=np.uint8)
cv2.createTrackbar('History', WINDOW_NAME,
                   fgbg.getHistory(), 500,
                   fgbg.setHistory)
cv2.createTrackbar('dist2Threshold', WINDOW_NAME,
                   int(fgbg.getDist2Threshold()), 400,
                   fgbg.setDist2Threshold)<jupyter_output><empty_output><jupyter_text>Applying mask, resizing the image using the trackbarsI have to add a dimension to the mask so it can be broadcasted to the color channels (BGR) in the frame. Checkout `fgbg.shape` and `frame.shape` if that does not make sense. I also cast it as a boolean (it defaults to 0, 127, 255). I log the position of the sliders via over-writting print statment so I know what they are. **No Window Appears**: If no window appears a immediatly, try looking around. Sometimes the Python GUI appears behind this browser window or somewhere else.**Python hangs**: Makes sure the Python OpenCV process is the focused window for hitting the key stroke that breaks the while loop.<jupyter_code># %load -r 23:40 src/main.py
frame_cnt = 0
while(True):
    ret, frame = cap.read()
    fgmask = fgbg.apply(frame)
    display_img = cv2.resize(
        frame*np.atleast_3d(fgmask.astype(np.bool)),
        (width, height), interpolation=cv2.INTER_AREA)
    cv2.getTrackbarPos('History', WINDOW_NAME)
    cv2.getTrackbarPos('dist2Threshold', WINDOW_NAME)
    cv2.imshow(WINDOW_NAME, display_img)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break
    frame_cnt += 1
    msg = 'History {0}, dist2Threshold {1}'.format(
              fgbg.getHistory(), fgbg.getDist2Threshold())
    print("\r", msg, end="")<jupyter_output>History 50, dist2Threshold 100.0History 50, dist2Threshold 100.0<jupyter_text>Close the windows and release the camera<jupyter_code># %load -r 41: src/main.py
cap.release()
cv2.waitKey(1)
cv2.destroyAllWindows()
cv2.waitKey(1);<jupyter_output><empty_output><jupyter_text>PerformanceWhile figuring out what algorithms/operations I use, I consider the affect they have on the FPS (Frames Per Second). *Note*: I do not use `imshow` while I do this because showing images is an operation in itself. **Note**: Some operation parameters heavily infulence not only the accuracy of the system but the runtime/memory performance. For instance, `history` set to 32 & 400 gives me an average (out of 3 trials each) of 7.372 & 7.645. If your FPS is too slow to detect the motion you are looking for, you can reduce the resolution size of the original frame right after you capture it. Just modify the `resize` code I used above.<jupyter_code>num_frames = 200
# Start time
start = time.time()

cap = cv2.VideoCapture(0)
fgbg = cv2.createBackgroundSubtractorKNN(
    history=500, dist2Threshold=400.0, detectShadows=False)
tmp =  np.empty((int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)), 
                 int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)), 3), 
                dtype=np.uint8)

# Grab a few frames
for frame_cnt in range(0, num_frames) :
    ret, frame = cap.read()
    fgmask = fgbg.apply(frame)
    tmp = frame*np.atleast_3d(fgmask.astype(np.bool))
    print("\r", frame_cnt, end="")
# # End time
end = time.time()

# # Time elapsed
seconds = end - start
# print("Time taken : {0} seconds".format(seconds))

# # Calculate frames per second
fps  = num_frames / seconds;
print("\r", "Estimated frames per second : {:3.4}".format(fps));

# # Release video
cap.release()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08245382585751979, 'fraction_numerical': 0.020448548812664908, 'mean_word_length': 4.400712377560107, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '562207', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1863, 'n_tokens_neox': 1751, 'n_words': 732, 'path': 'notebook.ipynb', 'repo_name': 'texastony/Motion_Detection_Obj_Recog', 'size': '6064'}