<jupyter_start><jupyter_text>In this notebook we show how to use the flytracker software. It's pretty simple, but there's a few things to keep in mind. You'll find a minimal running script in this same folder called 'example_run.py'.<jupyter_code>from flytracker import run
from flytracker.utils.param_helpers import load_frame, test_mask
from flytracker.analysis import annotate

from skimage.filters import threshold_local
import torch

import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>The first step is to create a mask which selects the arenas in the video. To do so we load a single frame from the video:<jupyter_code>#Location of a movie - this needs to be a mp4!
# If you get weird errors you probably won't have specified the path correctky.
movie_path = "video_segments/segment_000.mp4"
image = load_frame(movie_path, frame=100, color=False)
plt.figure(figsize=(15, 10))
plt.imshow(image, cmap='gray')<jupyter_output><empty_output><jupyter_text>The mask should be 1 inside the arenas and zero outside. Especially getting rid of the QR codes and dark edges here is important.<jupyter_code>mask = torch.ones((1080, 1280), dtype=bool)
mask[:130, :] = 0
mask[-160:, :] = 0
mask[:, :270] = 0
mask[:, -205:] = 0

mask[:190, :350] = 0
mask[:195, -270:] = 0
mask[-220:, :340] = 0
mask[870:, 1010:] = 0

masked_image = test_mask(image, mask)
plt.figure(figsize=(15, 10))
plt.imshow(masked_image, cmap='gray')<jupyter_output><empty_output><jupyter_text>Next we need to build a threshold, which, given the value of a pixel, decides if the pixel depicts a fly or not. While you can set a global threshold ("If the pixel value is lower than x, its a fly"), a local threshold will do better and will cope with different lighting. Such a local threshold should thus have the same shape as the image, and we make one using sk-image local threshold. You can probably do better filters with some tuning and thinking, but this does decently.The idea is that flies are fairly sparse (they make up a small part of all the pixels), and that we can get the local lighting / background by using a big enough median filter. Everything more than 30 away from the local lighting is a fly. Since lighting remains constant, we only need to do this once.<jupyter_code>def local_threshold(image, mask, threshold=40, block_size=31):
    background = threshold_local(image.numpy(), block_size=block_size, method='median', )
    # we apply the mask after the background to avoid annoying edge effects
    background = test_mask(torch.tensor(background, dtype=torch.uint8), mask)
    return background - threshold
local_threshold = local_threshold(image, mask) # feed in unmasked image!
global_threshold = 120<jupyter_output><empty_output><jupyter_text>Now let's compare the local and global threshold:<jupyter_code>plt.figure(figsize=(15, 10))

plt.subplot(121)
plt.title("Global")
plt.imshow(masked_image < global_threshold, cmap='gray')

plt.subplot(122)
plt.title("Local")
plt.imshow(masked_image < local_threshold, cmap='gray')<jupyter_output><empty_output><jupyter_text>Clearly the local threshold is better, and since we calculate it once before running everything it will be just as fast :-)Now that we have our requisites (mask, threshold), we can run the algo. We can either specify a single movie, or a folder with numbered movies (doesn't matter exactly how as long as it's obvious):<jupyter_code>path = "video_segments/segment_000.mp4" # only analyzes this file
# path = "video_segments/" # analyzes all files in folder<jupyter_output><empty_output><jupyter_text>Then just run:<jupyter_code>%%time
df = run(
    path,
    mask,
    n_arenas=4,
    gpu=False,
    n_ini=100,
    threshold=local_threshold,
    max_change=2.0
)<jupyter_output><empty_output><jupyter_text>The other inputs are:* n_arenas specifies the number of arenas present in the video, * gpu selects if you want to use a GPU (which will be very fast!), * n_ini sets the number of initial frames used to figure out how many flies we have.* max_change is a measure for the change between frames. Two adjacent frames should yield similar results, so if they're far apart something's wrong. If the change is higher than max_change, we simply skip the frame and try again the next frame. 2.0 seems a decent value but you might need to fiddle a bit.* if n_frames is set it only analyzes the first n_frames. it's good practice to first run with 1000 or so frames to check if your mask and threshold are correct, and then run the full / all movie(s). We return a dataframe with the ID of the fly, position, frame and arena index:<jupyter_code>df.head(5)<jupyter_output><empty_output><jupyter_text>You're responsible for saving and exporting the dataframe. We recommend saving to .hdf with compression so it doesn't become a massive file using the following line<jupyter_code>df.to_hdf("tracks.hdf", key="df", complevel=9, complib="blosc")<jupyter_output><empty_output><jupyter_text>An example of the inferred tracks:<jupyter_code>for ID in df.ID.unique():
    plt.plot(df.query(f"ID=={ID}")["x"], df.query(f"ID=={ID}")["y"])<jupyter_output><empty_output><jupyter_text>Finally - what if you want to see the tracks overlaid on video? We have you covered with the annotate function. Note that it's quite slow, so we recommmend doing this only when you want to check the results. It writes two videos (one normal which you can delete and one smaller compressed). track_length sets the temporal length of the tracks and when flies are closer than the given touching_distance we turn the number red so you can easily check for inversions.<jupyter_code># Note this only works for a single video.
annotate(df, path, "annotated_video.mp4", track_length=30, touching_distance=10,)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08401177897107223, 'fraction_numerical': 0.019573878399445695, 'mean_word_length': 5.458612975391499, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '109243', 'license': 'MIT', 'n_tokens_mistral': 1819, 'n_tokens_neox': 1695, 'n_words': 726, 'path': 'examples/example.ipynb', 'repo_name': 'GJBoth/flyTracker', 'size': '5773'}