<jupyter_start><jupyter_text>Custom Pipeline using MIPI as input source---Please use Jupyter labs http://&lt;board_ip_address&gt;/lab for this notebook.This notebook shows your how to create a custom pipeline that uses the [Digilent Pcam 5C Camera](https://reference.digilentinc.com/reference/add-ons/pcam-5c/start) as input video source Aims* Use Pcam 5C as input video source* Compose branched pipeline on the input video* Enhance the output by using a simple threshold Table of Contents* [Download Composable Overlay](download)* [Start Pcam 5C and Display Port video pipeline](start_video)* [Branched Pipeline](branched)* [Enhance the Output](enhance)* [Stop Video Pipeline](stop_dp)* [Conclusion](conclusion)---- Revision History* v1.0 | 15 June 2021 | First notebook revision.* v1.1 | 11 August 2021 | Update notebook to composable overlay API 1.0.0* v1.2 | 12 November 2021 | Update notebook to use `VideoStream`----  Download Composable Overlay Import the pynq video libraries as well as Composable class and the drivers for the IP.Download the Composable Overlay using `pynq.Overlay` and grab a handler to the `composable` hierarchy<jupyter_code>from pynq import Overlay
from pynq.lib.video import *
from pynq_composable import *
import numpy as np

ol = Overlay("cv_dfx_4_pr.bit")

cpipe = ol.composable<jupyter_output><empty_output><jupyter_text>Start Pcam 5C and Display Port video Get `VideoStream` object and start video    Warning:Failure to connect Display Port cables to a valid screen and to connect Pcam 5C may cause the notebook to hang<jupyter_code>video = VideoStream(ol, source=VSource.MIPI)
video.start()<jupyter_output><empty_output><jupyter_text>Branched Pipeline In this part of the notebook, we will bring new functionality into the four DFX regions to compose the [Difference of Gaussians](https://en.wikipedia.org/wiki/Difference_of_Gaussians) application that was also introduced in the previous session. Load dynamic IP<jupyter_code>cpipe.loadIP(['pr_fork/duplicate_accel', 'pr_join/subtract_accel', 'pr_0/filter2d_accel'])<jupyter_output><empty_output><jupyter_text>Grab handlers and set up default values<jupyter_code>filter2d = cpipe.filter2d_accel
duplicate = cpipe.pr_fork.duplicate_accel
subtract = cpipe.pr_join.subtract_accel
filter2d_d = cpipe.pr_0.filter2d_accel

filter2d.sigma = 0.3
filter2d.kernel_type = xvF2d.gaussian_blur

filter2d_d.sigma = 12
filter2d_d.kernel_type = xvF2d.gaussian_blur<jupyter_output><empty_output><jupyter_text>Grab handlers to the MIPI camera<jupyter_code>mipi_in = cpipe.mipi_in
mipi_out = cpipe.mipi_out<jupyter_output><empty_output><jupyter_text>The Difference of Gaussians is realized by subtracting one Gaussian blurred version of an original image from another less blurred version of the original. In the Composable Overlay this is achieved by branching the pipeline, which is expressed as a list of a list.<jupyter_code>video_pipeline = [mipi_in, filter2d, duplicate, [[filter2d_d], [1]], subtract, mipi_out]

cpipe.compose(video_pipeline)

cpipe.graph<jupyter_output><empty_output><jupyter_text>Enhance the Output Depending on light condition and ambient the result of the Difference of Gaussians can look weak. We are going to use the LUT IP to set a threshold to improve how the output results.Grab a handler to the LUT IP and configure the numpy array with the thresholding values<jupyter_code>lut = cpipe.lut_accel
rgb2gray = cpipe.rgb2gray_accel<jupyter_output><empty_output><jupyter_text>Set up a simple threshold. Any channel in the input pixels with intensity above 11 will be mapped to an intensity of 255 in the output.<jupyter_code>thr = np.array([np.ones((3,3), dtype=np.uint8) * 11 , np.ones((3,3), dtype=np.uint8) * 255],dtype=np.uint8)
lut.threshold = thr
cpipe.insert(([rgb2gray,lut], 5))

cpipe.graph<jupyter_output><empty_output><jupyter_text>You can also observe the output in color by removing the `rgb2gray` IP<jupyter_code>cpipe.remove([rgb2gray])
cpipe.graph<jupyter_output><empty_output><jupyter_text>Stop Video Pipeline Finally stop the Display Port video pipeline    Warning:Failure to stop the Display Port Video may hang the board when trying to download another bitstream onto the FPGA<jupyter_code>video.stop()
ol.free()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07959614933082883, 'fraction_numerical': 0.018079361352430146, 'mean_word_length': 6.383015597920277, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '384005', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1423, 'n_tokens_neox': 1345, 'n_words': 445, 'path': 'boards/KV260/notebooks/custom_pipeline/09_mipi_pipeline.ipynb', 'repo_name': 'Kgfu/PYNQ_Composable_Pipeline', 'size': '4259'}