In the construction process of smart cities, more and more video surveillance systems have been deployed for traffic, office buildings, shopping malls, and families. Thus, the security of video surveillance systems has attracted more attention. At present, many researchers focus on how to select the region of interest (RoI) accurately and then realize privacy protection in videos by selective encryption. However, relatively few researchers focus on building a security framework by analyzing the security of a video surveillance system from the system and data life cycle. By analyzing the surveillance video protection and the attack surface of a video surveillance system in a smart city, we constructed a secure surveillance framework in this manuscript. In the secure framework, a secure video surveillance model is proposed, and a secure authentication protocol that can resist man-in-the-middle attacks (MITM) and replay attacks is implemented. For the management of the video encryption key, we introduced the Chinese remainder theorem (CRT) on the basis of group key management to provide an efficient and secure key update. In addition, we built a decryption suite based on transparent encryption to ensure the security of the decryption environment. The security analysis proved that our system can guarantee the forward and backward security of the key update. In the experiment environment, the average decryption speed of our system can reach 91.47 Mb/s, which can meet the real-time requirement of practical applications. With the continuous development of digital video cameras and network infrastructure, video surveillance is widely deployed in smart cities ,2,3. MorSecurity and privacy issues exist widely in video surveillance systems deployed in smart cities. With the privacy leakage of Apple iCloud and the Verkada camera, the privacy of multimedia has attracted more and more attention. PatronuS  was propHowever, privacy is a complex concept to define. It is even more difficult to define privacy areas in video, an unstructured file rich in various information . It is nThe main challenge of data security faced by video surveillance systems is how to ensure the video integrity, confidentiality, and availability during the data life cycle. Sufficient encryption for multimedia is necessary. However, how to implement terminal authentication and key management securely is equally essential. In addition, many monitors exist in different locations of video surveillance systems, which need to obtain the suitable key and decrypt sensitive videos in specified areas. Hence, ensuring the monitor obtains the appropriate permission and builds a secure decryption environment is also meaningful. Monitors with excessive authorization may infringe upon other people’s privacy. Insufficient authorization of monitors may cause the video surveillance system to fail to work properly. For many small-scale scenes, such as homes and stores, a monitor is often replaced by a personal computer, which lacks the necessary security protection. It is obvious to ensure the security of the decryption environment. As a whole, in multi-level, multi-regional video surveillance systems for smart cities, the challenges mentioned above are unprecedented.In our manuscript, we build a secure, convenient, and flexible video surveillance framework for use in a smart city. First, a trusted authority (TA) is built as a trust center of the city. Then, any terminal, whether a camera or a monitor, must be registered with the TA at the factory. When the terminal wants to access one video surveillance system, it should be authenticated to obtain the corresponding key legally. Specifically, the contributions of this manuscript include the following:(1)A secure model is proposed to realize unified protection of video surveillance systems in smart cities. The model provides terminal authentication, key management, and secure video decryption for video surveillance systems with different scales, such as shops, families, and transportation.(2)A secure authentication protocol is implemented for a secure video surveillance framework. Based on the Diffie–Hellman protocol, we add secret and timestamp elements that can effectively resist man-in-the-middle attacks (MITM) and replay attacks.(3)Combined with the actual needs of smart city video surveillance, we build a hierarchical key management architecture named the Key Management Scheme Based on Normal Tree (KMSNT), which is based on the group key distribution and the Chinese remainder theorem (CRT). The TA manages the key of all video surveillance systems in the city. Group keys maintained by the TA are sent to video surveillance systems with different scales, such as families, shops, and buildings. When a group member changes, KMSNT updates the group key efficiently to ensure the data security.(4)Considering that the decryption environment in a small-scale video surveillance system monitor may not be safe, we construct a video security decryption suite based on transparent encryption. It provides a secure environment for identity authentication, key acquisition, and data decryption of monitors. At the same time, it can prevent the terminal from illegally copying and modifying the video.The remaining sections of this paper are organized as follows. The second part introduces related work. We prepare some relevant knowledge for our proposed scheme in We need to consider the security of video surveillance systems from two aspects. First, an adversary may access a video by illegal methods at a certain stage , leading to a privacy leakage. Second, an adversary may hijack the terminal to interfere with the normal work of the system. The related research can also be divided into two aspects: data protection and system security.From the perspective of video data protection, digital video surveillance systems are mainly faced with three threats: confidentiality, integrity, and availability of the video. A secure video surveillance framework was proposed in  which usFrom the perspective of the attack surface of video surveillance systems, unauthorized video monitoring and stealing archived video footage are two common attack vectors used to attack video surveillance systems . First oWhen a large number of terminals need to achieve encryption or decryption, the one-to-one key distribution protocol exposes the bottleneck of a low update efficiency and poor real-time performance. Group key management aims to achieve an efficient key update . A groupUser join. When user 3 joins the group, the group key GK and the internal keys  belonging to the path from the root to the user 3 node are updated in the following steps to prevent the new user from obtaining the old keys.(1)GK′, KM′, KI′, KB′  are encrypted by K3 and K4 and unicast to user 3 and user 4;The new keys (2)GK′, KM′, KI′ are encrypted by KA and multicast to users 1 and 2;(3)GK′, KM′ are encrypted by KJ and multicast to users 5 to 8;(4)GK′ is encrypted by KN and multicast to users 9 to 16.User leave. When user 11 leaves the group, GK, KN, KK, KF need to be updated to ensure the forward security. The update steps are described below:(1)GK′, KN′, KK′, KF′ are encrypted by K12 and multicast to user 12;The new keys (2)GK′, KN′, KK′ are encrypted by KE and multicast to users 9 and 10;(3)GK′, KN′ are encrypted by KL and multicast to users 13 to 16;(4)GK′ is encrypted by KM and multicast to users 1 to 8.The group key update scheme dramatically reduces the time consumption of the key update process. However, there is still room for optimization in the video surveillance system.N, the number of keys stored in the leaf node is k is the branch number of the tree. For example, when the user number is 16, the leaf node needs to maintain five keys.First, with the increase in users, the number of keys that users in the leaf node need to maintain increases heavily. When the number of users is k is the number of branches, and d is the depth of the tree.Second, there are still several events of encryption and multicasting during the key update process. For example, when one user is evicted from the group, the new keys need to be distributed m1, m2,…, nm are n pairwise relative prime numbers, then for any integer a1, a2,…, na, the group of equations (Equation (1)) always has a unique integer solution modulo Assuming that X and Y that satisfy Equation (1), then x can be calculated as follows:If there are two integers In Equation (2), there is Combined with the actual situation of video surveillance systems in smart cities, the CRT is used to improve the efficiency of the group key distribution.Transparent encryption aims to provide a secure encryption and decryption environment, which protects the decryption in personal computers used as a monitor. Three transparent encryption algorithms for H.264/AVC and H.264/SVC compressed videos have been proposed . All thrTherefore, we proposed a multimedia decryption middleware scheme based on transparent encryption. In the scheme, decryption is conducted by the transparent encryption driver provided by the operating system. The scheme can ensure that the plaintext only exists in memory and can only be used by authorized applications and prevent illegal disk writing and network transmission. However, only encryption is not enough for the authorized multimedia distribution. Authority control is also important. In our paper, key management and file decryption are integrated together as one middleware and provide a security service to the upper multimedia applications. This not only eliminates the coupling between the complex decryption process and the complex decoding process, reducing the difficulty and security requirements of the upper software development, but also improves the decryption efficiency and reduces the delay. The middleware completes a series of complex operations such as authentication, key acquisition, and decryption without the user’s awareness, which enhances the user experience.In this section, we propose a secure video surveillance system model including terminal registration, authentication, key management, and video encryption. Considering private protection in real life, a video surveillance based on cloud computing faces two security threats. (1) Traditional video streams are mainly transmitted and stored in a plaintext model, which leads to the risk of data leakage ,32. (2) In order to cover the above security risks, we built a typical smart city security video surveillance model . It is wtrusted authority (TA) is responsible for the registration, authentication, and key management  of terminal devices, including monitors and cameras. The TA is trustworthy, and its structure can be central or hierarchical.The media cloud is used to store and process monitoring videos. We believe the media cloud is curious, that is, the media cloud honestly follows the protocol execution, but curiosity propels it to speculate over and analyze the data and searchable index tree available at the server.The camera is the basic video capture unit in the system. Multiple cameras can form a group to monitor a specific area, such as traffic, a shop, a family, and a building. In our system, any camera needs to be registered with the TA in its factory settings. Before the camera joins the video surveillance system, it should pass the security authentication. The legal camera captures and encrypts the video and then sends the encrypted video to the media cloud and local temporary storage server.The monitor can view the videos captured by the cameras under its jurisdiction. In the actual scene, monitors and cameras have an obvious hierarchical structure. Simply, a monitor in a small shop can only view the video of the cameras belonging to the shop. A video surveillance system for an office building that has many independent companies has a more obvious hierarchy. One company’s monitor can only view the videos covering its office area but cannot view the videos beyond that area. However, the property company of the building has the right to view the surveillance video of the whole public area.The Our model’s hierarchical key management architecture has three advantages for a secure video surveillance model in smart cities. (1) Cameras often work as a cluster, and hierarchical key management is more suitable for group key management for each cluster. (2) The hierarchical key management architecture has better visualization to represent the monitors and cameras in different regions and geographical locations. (3) The architecture can be easily extended, and the TA can be transformed from the central model to the distributed model, which is necessary for large-scale video surveillance in smart cities. Taking the central model architecture as an example , we defii through the secure channel established between the TA and the factory. During this process, the TA records terminal information, including the manufacturer, registration time, device name, and type. Then, the TA assigns a unique public ID named Monitors and cameras need to register with the TA before they leave the factory. The registration process is initiated by the terminal i joins the system, the TA and the device should be mutually authenticated. Firstly, the device i sends the authentication request message  to the TA. Then, the TA starts the authentication phase and sends a message T. The message T. In a is a positive integer randomly chosen by the TA:Before the device i receives the message T are separated from i calculates its own ik is stored in the terminal i in the register processing. The MAC is compared with the received T can resist replay attacks.When the terminal i generates a random integer b and calculates i sends the response message After passing the server authentication, the terminal a, iK, iID, TAID, and T), i succeeds. After mutual authentication, the TA builds a secure channel with the terminal i and shares the secret When the TA receives the response message For privacy protection in video surveillance, we propose the Key Management Scheme Based on Normal Tree (KMSNT). KMSNT realizes the group key distribution and provides secure methods for member join and leave to ensure backward and forward security. Aiming to solve slow and complex group key updates, we introduce the CRT to achieve simple and fast group key renewal.There are three main functions in KMSNT: (1) group formation and group key distribution; (2) group key retrieval; (3) group member change. Additionally, the participants in KMSNT are the TA, monitor, and camera.i obtains the following data after authentication:According to the devices’ authentication information, the TA maintains a large tree that denotes the grouping status of each monitor and its cameras. No matter the monitor or camera, the device j, the TA generates a group key n is the number of cameras in the group. Then, the TA prepares a message fragment iM for each camera. The message for the ith camera consists of the group key T, a message authentication code For each monitor jjC is encrypted by iK and sent to the group member.n is the number of cameras belonging to the monitor x is encrypted using the key k.The final group key distribution message for the monitor m receives the ciphertext T. When the validation succeeds, the camera obtains the group key When the camera j′ accesses the encrypted data produced by its cameras, the monitor j′ should be verified following the steps in j′. The group key is released following the same steps as those with the camera.Before the monitor In order to ensure the security of the group key, the group key should be updated when the group members are changed. Member join and leave are two basic operations. Complex group changes, such as group merging and deletion, are made up of these two operations. In general, a group member change for video surveillance systems mainly occurs in the cameras because only the cameras are located on the leaf node.Member Join. When a new camera wants to join the group, the TA should update the group key to ensure the new member cannot access the old key. This guarantees the backward security. Using the CRT, we realize a convenient key update.i in the group. Additionally, n is the member number of the original group, and Firstly, the TA calculates the value j. According to the CRT, we can obtain the following equations:Secondly, the TA generates a new random key i. The value iP is the secret prime number produced by the authentication process. Since P1, P2, …, nP, Pn+1 are a set of prime numbers selected randomly, they are pairwise relative prime integers and N.i, there is x is sent to the whole new group members. When the ith camera receives x, the new group key For each Member Leave. When the old member leaves the current group, the TA should update the group key to maintain the forward security. The leaving member can no longer access the group key. For the traditional group key distribution scheme, the group key update finishes after k is the number of branches of the tree, and d is the depth of the tree. However, with the CRT, we only need one round of message exchange.The steps of the group key update in member leave are similar to those in the member join process. To transmit the new group key x is sent to the reserved group members. When the ith camera receives x, the new group key Then, Video encryption has been fully discussed in many papers. We do not provide any new video encryption algorithm in this manuscript. However, we attempt to solve a new problem which is exposed in video surveillance systems. In some realistic video surveillance scenes, such as shops and families, the monitor is often a personal computer without any professional protection. This leads to the decryption environment of the monitor being complex and vulnerable. Based on transparent encryption, we designed a secure decryption scheme to resist vulnerable environments. The proposed scheme for monitors consists of a user module and a kernel module. The former is used for monitor authority control to communicate with the TA to obtain the decryption key. The kernel module finishes the video integrity validation and executes secure decryption to protect the plaintext from modification.Authority control is realized for monitors to obtain the decryption key legally. We assume that the monitor has registered in the TA earlier. Firstly, the monitor and TA verify each other according to the steps described in m sends mP, a timestamp T, a message authentication code The monitor m is encrypted by m.The final group key distribution message m and TA, For the monitor When the monitor obtains the decryption key fData is input into the encryptor, and the group key fKey has been prepared well, encryption follows the steps below:(1)pData with fKey and obtain the ciphertext cData = fKey(pData);Encrypt (2)cData represented by digest and put digest in the header of cData;Generate the digest of (3)encFlag to ENC in the prefix, which means the following data are encrypted;Set (4)Pad the file prefix up to a multiple of 32 bytes;(5)PSize. Then, the output isCount the number of padding data We briefly describe the video encryption process firstly. For each encrypted multimedia segment, the encryption information shown in For symmetric encryption, decryption is the inverse process of encryption. However, in practical scenarios, decryption has to face more security risks than encryption. Specifically, (1) the environment for decryption may not be secure enough; and (2) the decrypted information may be abused or forged. In our scheme, we use two filters in the kernel module, named the process filter and the file filter, to ensure the security of the decryption environment and prevent the plaintext from being abused.Flag = True. At the same time, the process filter uses writing blocking to reject the writing operation from any application or process.The process filter aims to prevent the plaintext from being leaked out by malicious applications or processes. During the filtering process, the validity of the application in the receiver side will be checked. The MD5 value of the application will be compared with the trusted application list. When the application comparison result is true, we set digest and encFlag) preset in the file prefix will be read out, and Flag is checked in pre-operation. In post-operation, a new generated digest1 will be compared with digest to check whether the file data are identical to the original version before the decryption key request is sent out. Then, the ciphertext cData will be decrypted.After the process filter, file filtering is used to verify the integrity of the encrypted files and decrypt the ciphertext. If the encryption flag in the file header is true, the file integrity will be verified. File filtering has two steps, pre-operation and post-operation, as shown in As the core function, the decryption process is realized based on a Minifilter. Integrity is inspected using the message digest algorithm simultaneously while accessing a file. In addition, sensitive applications and processes are kept from writing operations to eliminate data leakage and modification.In this section, we analyze the security and the real-time performance of the proposed video surveillance scheme. In the security analysis part, we mainly focus on the forward/backward security of the group key management scheme and the monitor environment security during the video decryption process. For the real-time performance, we believe that the requirement of real time in video surveillance systems is in monitor decryption. There are two reasons. First, when the video surveillance system works smoothly, it is not common for a camera to join or leave frequently. Second, the camera that captures and encrypts the video can be specified and customized to meet the real-time performance. However, the monitor may be a personal computer with poor properties or a dedicated server with a strong calculation capacity. Therefore, it is more meaningful to discuss the real-time performance of decryption in monitors. In monitor decryption real-time performance analysis, the experiments were implemented by PHP and C++ and run on a computer . All tests were carried out under the Gigabit LAN.In this section, we discuss the forward and backward security of the group key management scheme and the security of the monitor decryption environment.i belongs to the remaining members in the group. The new group key is randomly generated by the TA and has no relation with the old key. For the evicted member, it is necessary to guess the secret parameters iP and ith camera, and iP is also the secret prime number only known by the TA and the ith camera. Therefore, a brute force attack is the only way for the evicted member to obtain the correct group key. Hence, forward security is maintained.When any leaf node of the tree built in our key management scheme is evicted from the group, the group key update algorithm must guarantee forward security to prevent the evicted node from reading the new message. As shown in ith camera. The secrecy of ik is only shared between the TA and the ith terminal, meaning the MITM attack cannot work. The timestamp Similar to forward security, the secure anchor is on the security of the secret From the perspective of confidentiality, sensitive information includes multimedia data and secret keys. The main risk of them is that they may be stolen through devious ways. In a monitor, a sensitive video can be accessed only when the decryption is executed with a correct key. Additionally, after decryption, a Minifilter ensures that decrypted data are only obtainable for verified trusted applications until they are destroyed and will not be leaked out by disk writing or network transmission. The secret key is stored in the kernel memory and insulated from unauthorized access until it is destroyed, meaning the security of the secret key in the monitor can be maintained.From the perspective of integrity, multimedia may be tampered or damaged. However, tampering definitely changes the digest of the file. When accessing a forged file, file filtering usually fails because the file cannot pass the integrity verification. Even if the digest on the file prefix can be replaced by that of the forged file, it is impossible to obtain a correct key that was indexed by the original file digest. Therefore, tampering will be detected immediately. Damage makes encrypted multimedia unusable. However, authorized users can retrieve an original copy from the cloud at any time. Unauthorized users cannot obtain any useful information from the damaged file.As mentioned before, we focus on the decryption speed in the monitor. When the monitor obtains the decryption key, the next step is to use the key to decrypt the encrypted multimedia. In the decryption process, the validity of the application and the integrity of multimedia will be verified firstly. In the decryption test, we consider the efficiency of the decryption, that is, the decryption speed. The validity of the application happens before decryption, and it needs to communicate with the server. In the actual implementation, we submit the MD5 value of the application together with the key request to the server. That ensures all operations in filters are local.Therefore, there are two factors that affect the efficiency of the transparent decryption, the AES decryption speed and the digest generation speed. In this experiment, we used 128-bit ECB mode AES for encryption, and 256-bit MD5 for digest generation. The result is shown in The efficiency of multimedia encryption and decryption is closely related to the encryption method and encryption algorithm. In our demo, we adopt full encryption for multimedia, including videos. The decryption speeds for different types of multimedia using different encryption algorithms are shown in More and more video surveillance systems have been deployed in the smart cities now. The security of video surveillance faces serious challenges, especially video data. The main challenge of data security faced by video surveillance systems is how to ensure the video integrity, confidentiality, and availability. The key to solving the problem is to build an appropriate scheme for device authentication, key management, and data encryption/decryption to prevent criminals from attacking the system on different attack surfaces. In this manuscript, we built a secure video surveillance model which includes four participants: trusted authority (TA), media cloud, monitor, and camera. Then, terminal registration, mutual authentication, and a group key management method named KMSNT were implemented. By adding secrets and timestamps into the Diffie–Hellman protocol, the proposed mutual authentication protocol can resist MITM attacks and replay attacks. Security analysis proved KMSNT can guarantee the forward and backward security of the key update. Considering the decryption environment in monitors of small-scale video surveillance systems may not be safe, we constructed a video security decryption suite based on transparent encryption. The suite provides a secure environment for identity authentication, key acquisition, and data decryption in the monitors. At the same time, it can also prevent the terminal from illegally copying and modifying the video. The video security decryption scheme has a fast decryption speed which can reach 91.47 Mb/s on average.Future work. We are considering building a more complex tree to achieve more practical key management requirements in video surveillance systems, such as regional division and cross-level access. We will also try to optimize the video encryption scheme to improve the encryption efficiency and reduce the encryption consumption in the cameras.


Video object and human action detection are applied in many fields, such as video surveillance, face recognition, etc. Video object detection includes object classification and object location within the frame. Human action recognition is the detection of human actions. Usually, video detection is more challenging than image detection, since video frames are often more blurry than images. Moreover, video detection often has other difficulties, such as video defocus, motion blur, part occlusion, etc. Nowadays, the video detection technology is able to implement real-time detection, or high-accurate detection of blurry video frames. In this paper, various video object and human action detection approaches are reviewed and discussed, many of them have performed state-of-the-art results. We mainly review and discuss the classic video detection methods with supervised learning. In addition, the frequently-used video object detection and human action recognition datasets are reviewed. Finally, a summarization of the video detection is represented, e.g., the video object and human action detection methods could be classified into frame-by-frame (frame-based) detection, extracting-key-frame detection and using-temporal-information detection; the methods of utilizing temporal information of adjacent video frames are mainly the optical flow method, Long Short-Term Memory and convolution among adjacent frames. Video object detection and human action recognition are applied to various scenarios, such as the recognition of vehicle plate numbers in traffic monitoring systems, the detection of dangerous vehicle behaviors, the detection of running red lights, the detection of abnormal production behaviors in industrial production, the identification of abnormal passenger behaviors at stations and airports, etc.The difficulties of video detection include video defocus, motion blur, part occlusion, etc. Video defocus would be generated during the focusing process. The defocus of the video and the motion of the object may cause the video defocus and motion blur. Occlusion between objects may cause the part occlusion. In addition, the shape of the objects in the video may be changing with the distance of the camera. Therefore, compared with image detection, video detection should be more challenging.The existing video detection methods are operated on frames. Most of the existing video detection methods are to decompose the video into frames, and then use the image detection method to detect the frames. Therefore, the speed of video detection depends on the speed of image detection. In addition, some methods directly operate on the video, however, these methods are also frame-based. They operate adjacent frames by using specific algorithms. Therefore, for video detection, image detection methods are still important.Before, the methods applied to image detection include Histogram of Oriented Gradients (HOG) , Scale-IMachine learning classifiers include Support Vector Machine (SVM) , Random Random Forest is based on Decision Tree . DecisioIn the deep learning tasks, some loss functions can also be regarded as the classifiers, since only the class which conforms to the loss function can be detected and recognized. These loss functions include cross-entropy function  and someBefore deep learning, local feature extraction methods such as SIFT, HOG, etc., did not have the ability of feature translation invariance. The possible reason may be that the features extracted by these methods may be simpler than the deep learning methods.The detection speed of one-stage detector is usually faster than two-stage detector. The two-stage detector has higher detection accuracy, but the detection speed would be reduced. The two-stage detector usually follows the two steps: extracting features from the input (feature extractor), recognizing the features by the trained classifiers (classifier). The difference between one-stage detector and two-stage detector is mainly that the two-stage detector often has a separate feature extractor, which is called “backbone” in some literatures. Meanwhile, the one-stage detector combines the feature extractor and classifier into one, which can reduce the complexity of the network structure and improve the detection speed, but the detection accuracy may be reduced.Generally, video object and human action detection can be classified into three categories: detecting frames by the image detector, extracting the key frames for the detection, or using temporal information between adjacent frames. The former is implemented frame by frame, and the speed of video detection depends on the speed of the frame detection. Moreover, these methods usually do not extract the key frames from videos. Thus, the base of these methods is still image detection. Some algorithms, such as You Only Look Once (YOLO) , use theThe latter of the above paragraph includes optical flow  and LongThe connection between object tracking and object detection is the method of object tracking could be used for object detection, since object tracking and object detection both use the temporal information of the video frames.The contribution of this paper is summarized as follows:(A) Review the commonly-used video-based detection datasets, and their application scope.(B) Review the machine learning-based models, which are used for video object and human action detection.(C) Summarize and analyze the performance of the classical video detection algorithms, and summarize the methods of improving the speed of video detection.(D) Summarize and analyze the image and video evaluation metrics used in the literatures, and illustrate that most video metrics use image detection metrics by frames.(E) Summarize the algorithms of video surveillance system, face detection, face recognition, face tracking, image and video quality enhancement, respectively.(F) Summarize the three main ideas for video detection: the first is to detect each frame; the second is to extract the key frames; the third is to adopt the LSTM structure, the optical flow method or convolution among adjacent frames for using the temporal information among adjacent frames.The main structure of the paper is: first, the video datasets are introduced; second, the video detection algorithms are introduced and analyzed; third, the video detection algorithms are discussed and evaluated. Among them, the video detection algorithms are classified as the frame-by-frame algorithms  and the The paper reviews the video object detection and human action recognition using deep learning methods, summarizes the current video detection approaches. Our paper is organized as follows: accuracy, precision, recall, F1-score, AP, mAP, ROC, AUC, etc. These metrics are from the classification results of the positive and the negative data. The positive data indicates the correct data, and the negative data indicates the wrong data. The calculation factors of these metrics are: True Positive (TP), True Negative (TN), False Positive (FP), False Negative (FN). TP represents the number of the positive classes predicted as the positive classes. TN represents the number of the negative classes predicted as the negative classes. FP represents the number of the negative classes predicted as the positive classes. FN represents the number of the positive classes predicted as the negative classes.The machine learning-based evaluation metrics used for image detection mainly include: Receiver Operating Characteristic (ROC) curve uses the False Positive rate and the True Positive rate as the coordinate axes. The area under the ROC curve is Area Under Curve (AUC). From the existing video detection literatures, it could be found that most of the video detection evaluation use the image metrics. The method is to use video frames for calculation. We think that there are many differences between video and image. Video frames are always not independent of each other, but images are independent of each other. Thus, using image metrics to the video frames may cause the duplicated statistics, such as TP, FP, etc., since the same objects in the frames may be counted repeatedly.The commonly used video classification datasets are as follows: ImageNet VID dataset , which hSome video datasets are used for video segmentation, which is based on video object detection. These video datasets include: Cambridge-driving Labeled Video Database (CamVid) , which iSome video datasets are used for pedestrian and human face detection. These datasets include: UCSD Pedestrian dataset  is a vidVideo tracking is based on video object detection either. Amsterdam Library of Ordinary Videos for tracking (ALOV++)  is an obHuman action recognition uses the video object detectors. The datasets include: HMDB51 , which iMost video detection methods decompose the video into frames, use the image detection model to detect. Therefore, almost all image detectors can be applied for video detection. The other video detection methods utilize the correlation between frames, operate on adjacent frames. Some of the methods which operate on adjacent frames use LSTM-like models. The following discusses in detail.The current object detection methods are divided into two categories, one-stage object detection and two-stage object detection. In the two-stage object detection, feature extraction is the first stage, the classification is the second stage. One-stage object detection methods include YOLO , SSD 4343 and ReYOLO  makes thThe network structure of YOLO uses the structure of GoogLeNet  for clasObject) is the probability of the object, IoU represents Intersection-over-Union, which is the quotient of the intersection and union of the candidate bounding box and the ground truth bounding box. The Pr(Object) is 1 when an object locates at a grid cell, or else the value is 0. The second item of the right side of the above equation is the IoU value, which is the overlapping area of the bounding box and the ground truth.YOLO divides the input image into s × s grid cells, each grid cell only predicts one object, but an object may occupy multiple cells and may be predicted by multiple cells. Every grid cell generates B (number) bounding boxes, and the bounding box like the anchor box used in Faster R-CNN, has several different scales and aspect ratio. If an object occupies multiple grids, the bounding boxes of the grids merge into one. The bounding box is not only used to positioning, but also used to generate the object confidence. Each bounding box has 5 parameters: x, y, w, h, and the confidence.  denotes the center coordinate of the bounding box, w and h denote the relative location to the whole image. The detection confidence value is defined as follow:In the test, the bounding boxes are filtered by Non-Maximum Suppression (NMS). Compared with the previous video frame-based object detection methods, such as the frame detection methods using R-CNN, Faster R-CNN, Deformable Parts Model (DPM)  etc., YO(A) The detection speed of YOLO is very fast. As YOLO does not have the separate stage of generating the region proposals, the detection speed of YOLO is 45 FPS using TITAN X GPU, and the speed of Fast YOLO can reach 155 FPS with the same type of GPU. Since the playback speed of the video is about 30 frames per second, the real-time video detection can be implemented when the video detection speed reaches 30 FPS. Compared with the previous real-time detection system, such as Deformable Parts Model (DPM) , YOLO’s (B) YOLO uses the context information to enhance the detection accuracy. Since YOLO does not limit the number of grids occupied by the object, then, the relationship between the grids is relatively close. In the two-stage object detection, the classifier only detects the pixels in the region proposals, while YOLO uses the context information more. Thus, the detection accuracy is higher.(C) YOLO can learn high semantic features. As shown in YOLOv2  uses a sYOLOv2 uses the following methods to improve the detection speed: (A) YOLOv2 adopts Darknet19 as the detection neural network, which has 19 convolutional layers with 3 × 3 filter and 5 max pooling layers with doubling the number of channels compared with the previous layer. (B) YOLOv2 follows almost every 3 × 3 convolution layer with a 1 × 1 convolution layer, which may reduce the complexity of network computing and improve the detection speed. (C) YOLOv2 does not use the dropout layer, which may reduce the network computational complexity and help increase the network speed.In addition, the following methods are used to improve the detection accuracy: (A) YOLOv2 uses Batch Normalization (BN)  after evThe structure of DarkNet19 used in YOLOv2 is shown in YOLOv3  still usYOLOv3 uses multi-scale prediction to enhance the detection accuracy. In DarkNet53, the final detection result is synthesized by Scale1, Scale2 and Scale3, which is illustrated in YOLOv3 uses convolution instead of pooling for down-sampling, which reduces the information loss in the neural network iteration. Usually, the information loss of pooling may be large, because the pooling operation merges multiple pixels into one pixel.The detection speed and detection accuracy of YOLOv4  are imprYOLOv4 uses Cross Stage Partial Networks (CSP Darknet)  as the bYOLOv4 uses PANet  as the nYOLOv4 uses the YOLOv3 detector as the head. The characteristic of the head is fast detection speed and high detection accuracy. In the head, each object class generates three kinds of anchor boxes, corresponding to the three different object scales and sizes. The structure of YOLOv4 is shown in Yavariabdi et al.  propose Yavariabdi et al.  also proSingle Shot MultiBox Detector (SSD)  uses theIn the multi-scale feature maps of SSD, the large feature map is used to detect small objects, and the small feature map is used to detect large objects. Since the information loss of the large feature map is little.N is the number of boxes, which is mapped to Ground Truth, and the parameter x evaluates how much the i-th detection bounding box matches the j-th ground truth bounding box, and i-th detection box matches the j-th ground truth, and the class of ground truth is p. The parameter c is the value of class confidence. The parameter l is the location value of the detection box, and g is the position value of the ground truth.The loss function of SSD has two parts: the loss that calculates the detection confidence of the object, and the loss that calculates the location of the object. As shown in Equation (8).x, l, g, c, N are the same with Equation (8), m is the pixel within a bounding box. Pos denotes the positive samples of the bounding boxes, Neg denotes the negative samples of the bounding boxes, smoothL1 is the Smooth L1 loss function.The location regression loss uses Smooth L1 loss function, which is one of the classic regression functions. The confidence loss is a classic softmax loss function. The location regression loss and the confidence loss are shown in Equations (9) and (10).The detection speed of SSD is fast, and the detection accuracy is similar or higher than Faster R-CNN . SSD alsCompared with SSD, a difference of DSSD  is the aOther improved versions of SSD include Rainbow SSD (RSSD) , FeatureRSSD improves SSD algorithm from the following two aspects: (A) RSSD uses the classification networks to strengthen the connection of feature maps between different layers, which reduces the duplicate frames; (B) RSSD increases the number of feature maps in the multi-scale feature map prediction, increasing the robustness of the detection of small objects.In the network structure of RSSD, pooling and deconvolution are implemented simultaneously. Before the concatenation, Batch Normalization (BN) operations are performed on the feature maps to unify the feature distribution of the data, thereby improving the detection accuracy.FSSD emphasizes on the fusion of shallow and deep features. Shallow features have a low semantic level, while deep features have a high semantic level, if they are directly fused, these features will not be able to make full use, and the information loss may occur.The basic model of FSSD is basically the same as that of SSD, using VGG-16 as the basic model. The structure of FSSD is shown in Since video is composed of frames, theoretically, all two-stage image detection methods could be used for video detection by detecting the frames. In general, since the detection speed of the two-stage detector would be not very fast, this form of video detection cannot implement real-time detection.Two-stage object detection has a separate module for extracting features and region proposals, which is called backbone. Therefore, the detection speed is slower than the one-stage detector, although the detection accuracy is always higher than the one-stage detector.Now the classic two-stage object detection models include Regions with CNN (R-CNN) , SpatialThe mixed-stage object detection is a mixture of one-stage detection and two-stage detection, or other video detections which could not be classified as one-stage or two-stage detection.Minimum Delay video object detection  uses oneLyu et al.  uses ConZhang et al.  find thaQin et al. apply convolutional neural networks to human behavioral recognition, to constitute an intelligent city system . The proWang et al. present three-level hierarchical context modeling, which can recognize the events in videos by using the previous events . Hu et aCao et al. propose a Teacher Network and Student Network architecture, which realize the real-time video detection from vehicle cameras . The TeaZheng et al. propose a video dynamics detection method that can detect the events in videos . The metWang et al. integrate the method of object detection into video saliency detection, propose a feature hybrid framework to detect the spatiotemporal saliency in videos . Their mLi et al. consider the background and foreground of the video frames both, and implement background extraction and foreground detection . The appA novel convolutional neural network (CNN) architecture is proposed to recognize human action in videos . The arcDiba et al. present a kind of 3D convolution to extract the features of videos . It perfZhou et al. present a novel unsupervised learning architecture, to automatically learn the features of Depth and Ego-Motion from videos . The arc2DL), which uses intra-video distance metric and inter-video distance metric to learn the features. Intra-video distance metric is used for learning the features within a video, and inter-video distance metric is used for learning the features between videos. The method is also implemented on the video frames. An unsupervised learning representation called DRNET is proposed by Denton et al., used for many tasks, such as predictions of future frames in videos [Liu et al. propose an unsupervised learning architecture named Deep Voxel Flow (DVF) . DVF usen videos . The modn videos .Fast YOLO  adopts pYuan et al. proposed a framework which can efficiently detect the traffic sign in videos . The met2+  challenge concludes that the methods with better video detection effect use spatiotemporal context method [Chen et al. present a set of video detection metrics, to assess the dynamic detection effect of videos, such as center jitter error (CJE) and size jitter error (SJE) of them . Bengar t method .Luo et al. use spatial-temporal context aggregation (STCA) to fuse the feature maps . STCA leHan et al. invented Seq-NMS to find the most accurate region proposals from a video clip . The metScale-Time Lattice (ST-Lattice)  also doePouyanfar et al. propose a kind of deep learning architecture to detect the semantic event of videos . They exThe following methods use the temporal information by Long Short-Term Memory (LSTM) , by optiLong short-term memory (LSTM)  is suitaThe architecture of Association LSTM is shown in TD-Graph LSTM  is a weaThe workflow of TD-Graph LSTM is shown in Bottleneck-LSTM  is a ligBottleneck-LSTM is a combination of LSTM and SSD, i.e., SSD is integrated into the front and back of LSTM, and the gates of LSTM, such as the forget gate, the input gate and the output gate, are improved to make it more efficient, and more suitable for mobile terminal.Patchwork  introducSince each frame only detects a part of the frame (which is the sub-window), in order not to omit some important information, patchwork combines the feature map of the previous frame with the feature map of the sub-window by the patchwork cell. In this way, detection of sub-window improves the detection speed, while patchwork cell makes the network not omit the entire frame information. Patchwork also uses a Q-learning method  to enhanGuo et al. propose the Progressive Sparse Local Attention (PSLA)  to transThe detection structure of PSLA is similar to LSTM. Different from LSTM, the recursive module of this structure uses the proposed Recursive Feature Updating (RFU) and Dense Feature Transforming (DenseFT). The core of RFU and DenseFT is the proposed PSLA. Both RFU and DenseFT are used to transfer temporal information from the previous frame feature map to the next, RFU is used on key frames, while DenseFT is used on non-key frames.Two kinds of mapping are used in the implementation of PSLA. One is to use a sparse matrix to scatter each feature map cell to the periphery. The advantage of this is to enhance the neighbor information of each cell. The other is to integrate the results of the first step with the corresponding weights. The approach gets 81.4% mAP on ImageNet VID.Zhu et al. present a light weight video object detection architecture with high performance . The arcThe detection network of the proposed architecture adopts RPN  and R-CNThe optical flow method is often used to transfer feature maps. Jiang et al. proposed Learnable Spatio-Temporal Sampling (LSTS) . DiffereThe framework of the proposed system is also like LSTM. The key part of the system is Sparsely Recursive Feature Updating (SRFU) and Dense Feature Aggregation (DFA), SRFU is used for key frames, and DFA is used for non-key frames. The LSTS module is integrated into SRFU and DFA. The proposed system has good detection results on ImageNet VID dataset, with 82.1% mAP score and less computation time.The proposed approach  is appliThe AST-GRU module is also used for extract the spatiotemporal relationship between the adjacent frames. The STA in AST-GRU is used to detect the foreground objects, and the TTA in AST-GRU is used to detect the dynamic objects.The proposed system has a good performance on nuScenes dataset , which iLiu et al. develop a temporal memory fusion module from LSTM, to fuse the feature maps of adjacent frames . The feaThe approach achieves good performance on the dataset of ImageNet VID 2015, compared with other mobile video detection methods. The running speed of the proposed system can reach more than 70 FPS on PIXEL 3.Zhang et al.  proposedSpatial temporal memory network (STMN)  may be aA Tubelet Proposal Network (TPN)  is preseT-CNN  is propoT-CNN uses optical flow to adjacent video frames. T-CNN calculates the average optical flow vectors in the region proposal boxes. This may improve the robustness of video detection. The flow chart of constructing tubelet is shown in T-CNN achieved good detection results on ImageNet VID dataset and YouTube Objects (YTO) dataset, and won the VID challenge in the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) 2015.task). Since only the key frames are convoluted, DFF reduces the computation of video detection.This paper  combinesResNet-50 or ResNet-101  is used Another study by Zhu et al.  uses keyKey frame adjustment generates the feature consistency metric to measure whether the two frames have obvious optical flow motion. If the offset is too large, this frame will be taken as a key frame, and the feature will be extracted by image object detection method; if the offset is less than the threshold (0.2 in the literature ), it meaFGFA  finds thFGFA includes the modules of feature map extraction, optical flow transmission and feature map aggregation. The feature map extraction module is called backbone in some literatures, which is ResNet-50, ResNet-101  and InceThe structure of FGFA is shown in MANet  is propoLong Short-Term Feature Aggregation (LSFA)  is propoIn long-term feature aggregation, key frames and non-key frames are fused by optical flow method. After the key frame goes through the large feature extraction network, a feature map is generated. The generated feature map is then fused with the previous fused feature map, and finally sent to R-FCN for detection.In short-term feature aggregation, motion vectors and residual errors are calculated from the non-key frames and key frames. After that, motion vectors, residual errors, and the feature maps generated in long-term feature aggregation are fused with the non-key frames, and then R-FCN is used for detection. This method has achieved good results on the large-scale ImageNet VID benchmark, and meanwhile improved the detection speed. The structure is shown in The literature  is to coConvolution among adjacent frames is generally used for video object tracking. The idea of video tracking is generally convolution among adjacent frames first, and aggregation second. Some methods are to convolve adjacent frames sequentially for extracting the temporal features, such as 3D convolution .Three-dimensional convolution  was propThe ways of extracting temporal information include: using LSTM to operate multiple adjacent frames ,131, comTemporal Convolutional Network (TCN)  is propoTCN merges and convolves adjacent frames, and then convolutions after sampling. In addition, the shortcut in ResNet is added to the convolution. The structure of TCN is shown in The idea of this paper  is that RRM  uses theThe illustration of RRM is shown in STSN  extractsSince STSN combines the reference frame and the supporting frame for detection, this may have a significant detection effect on the video part occlusion and motion blur, for the reason that there are always clear objects in the frames.Integrated Video Detection and Tracking  uses theThe framework of this method adopts the structure which is similar to Faster R-CNN . The strRDN  uses a dThe relation module is derived from the literature . ObjectsFor video detection, motion blur, occlusion, and object deformation are often the difficulties. The temporal information is often one of the effective methods to solve these difficulties. The architecture is proposed by Shvets et al. It uses the relation block between frames to learn the correlation information among frames, which improves the accuracy of video object detection, and the detection consumption does not increase much .The structure of the relation block is described as follows. The target feature map and the support feature map are embedded in the linear layer. After the normalization, the matrix multiplication, the softmax layer and linear conversion layer, the both feature maps are concatenated to obtain the synthesized relation block feature map. The framework reaches good detection accuracy on ImageNet VID dataset, reaching 84.1% frame mAP value.The difficulties of the video detection are motion blur, part occlusion, and object deformation, etc. One of the ideas which could solve the problems is to transfer the feature maps from clearer frames to the blurry frames. Traditional methods often use optical flow or Recurrent Neural Network (RNN) to transfer the feature maps. Wu et al. proposed the novel Sequence Level Semantics Aggregation (SELSA)  for featSELSA uses a novel algorithm to calculate the similarity of the region proposal box between frames, calculates the weight from the similarity by another proposed algorithm, and integrates the feature map of the support frame into the current frame by the calculated weight.The framework uses Faster R-CNN  to extraLiu et al. propose a video detection system  to execuMEGA  combinesThe architecture of MEGA is illustrated in Lin et al.  develop For the videos with a long-time span, such as the hidden video camera for filming wild animals, and the video surveillance of the traffic conditions, their backgrounds are always the same or similar. However, the previous video detection methods probably analyze every frame or most of the frames. The same backgrounds of these frames may cause a waste of computing.The architecture of Context R-CNN  is basedThe method performs well on the Snapshot Serengeti (SS) , CaltechRN-VID  is propoThe method is based on RetinaNet , fuses tThe proposed network  is compoThe Convolutional Regression Tracking module uses a crossed cascade structure. A Region of Interest (RoI) is converted into two branches, the size of the RoI at the bottom branch is 3 times larger than the top branch one, as the object in the top branch RoI may move to the adjacent area. Next, the two branches are crossed concatenated to four channels, and the next is the fully connected layers which are used for the regression.The Convolutional Regression Tracking module can be inserted into other image detectors, as the module is light weighted. The approach has a good performance on the dataset of ImageNet VID.The approach  proposedGAST-Net contains two inputs, one is the video frames, which pass through the backbones and fuse into a feature map, the other is the geometry input which is generated from the input video frames. The geometry input passes through Geometry-Aware Attention Maps, and the feature maps are fused with the above feature map, and the next is to pass through the prediction module to classify and locate the objects.Tang et al.  proposedZhang et al.  propose Jiao et al.  proposedThis method uses human pose information to predict the occluded pedestrian in the next frame. PEN has two steps: the first step is to use the Region Proposal Network (RPN) to extract the features of pedestrians, and to generate the boxes. The second step is to use the Pedestrian Recognition Network to generate human pose information, and perform detection. The pipeline of PEN is shown in Cores et al.  presentsKang et al. propose Temporal Convolutional Network (TCN) . TCN opeVideo detection can be classified into video object detection, video saliency detection and video object behavior detection, etc. Video saliency detection is closely related to video object segmentation. Many valuable deep learning algorithms are developed in the recent years on video detection. YOLO detects the real-time stream videos at a very high frame rate. FGFA  can deteThe current ideas for improving the speed of video detection mainly are: (A) Increase the single frame detection speed, while maintaining the detection accuracy, and the frame-by-frame method can even reach the real-time detection. (B) Implement the accurate image detection (such as using the two-stage detectors) on the key frames of the video, and skip the non-key frames. Some methods shown in  use optiThe video evaluation metrics used in the literatures, such as mAP, accuracy, FLOPs, AP, AR, etc., are almost all calculated by image metrics, i.e., calculated by the frame. Some paper proposes the special video metrics. Mao et al. propose a novel metric called Average Delay to comprehensively evaluate video object detection . Delay rZhu et al. established a dataset for unmanned aerial vehicles (UAVs) video detection named VisDrone, which was taken by UAVs in many cities in China . The datThe methods of video surveillance system using deep learning are as follows: (A) Convolutional Neural Networks (CNN) based methods, the CNN includes AlexNet , VGG NetFor face detection, the following methods are proposed: (A) Template matching methods, which is based on the pre-learned face templates ,198; (B)For face recognition, the following methods are proposed: (A) Total matching methods, which compare the whole face region to the pre-learned face database, such as Eigenfaces , PrincipFor face tracking, the following categories of methods are proposed: (A) Point-based tracking, which includes Kalman and Particle filter methods ; (B) MulFor the image and video quality enhancement, the following algorithms are proposed: (A) Methods based on traditional image enhancement, such as contrast enhancement , waveletThe relationships between detection and tracking is that tracking often relies on detection. The main difference between them is that detection makes a classification, tracking only marks the object. Most algorithms show that the faster the detection speed, the lower the detection accuracy, and the lower the detection speed, the higher the detection accuracy.The followings are only our own opinions. The limitation of current video object and human action detection algorithms might be that the detection speed and detection accuracy often could not be acquired at the same time. Some algorithms can reach high detection speed while improving the detection accuracy, such as YOLO. However, for those particularly blurry videos, the detection speed could not be increased for now. Most of the current video detection algorithms could not reach the real-time detection speed. On the contrary, the detection and recognition speed might be slow. The phenomenon is especially obvious for those algorithms that use the temporal information. Some literatures of those algorithms did not state the detection speed, but only state the detection metrics. Those particularly blurry videos may require the temporal information detection, thus, we think that real-time detection of the blurry videos could not be implemented for now.However, the detection accuracy of those algorithms has already been highly improved. Some algorithms can reach a high level on the blurry videos, such as FGFA shown in . NeverthThe conceivable future research directions might be the following three aspects: (A) Enhancing the detection accuracy, especially for the blurry videos. The main idea is to use the temporal information among adjacent frames. (B) Enhancing the detection speed. The existing algorithms have advantage on those distinct videos, however, for those blurry videos, the detection speed could be improved. (C) Enhancing the detection accuracy and speed simultaneously. The main idea might be extracting the key frames. Since detecting the key frames only, the detection speed could be enhanced. The key frames are often clearer, and have more standard shape objects or human action shapes, which could enhance the detection accuracy and speed both.The discussed algorithms in the paper include object detection, action analysis, etc. The video detection algorithms are all frame-based for now. Currently there are three ideas for the video detection: (A) the first is to detect each frame. Some algorithms, such as YOLO, can realize very fast detection speed; (B) the second is to extract the key frames, and the detection depends on the algorithm of extracting key frames; (C) the third is to use LSTM structure or the optical flow method for extracting the temporal information among adjacent frames. The image detector can be applied to video detection in the three ideas above. Some video detection algorithms extract the temporal information by tracking on adjacent frames, such as the bounding box tracklet of the same object in adjacent frames . Many viFor blurry videos, or the videos with part occlusion, making full use of the temporal information among adjacent frames can significantly improve the detection accuracy, since the blurry or occluded objects could be detected in adjacent frames. From the existing literatures, the approaches of using the temporal information of adjacent frames include the LSTM structure, the optical flow method and convolution among adjacent frames. Some approaches transfer feature maps between frames via convolutions, such as Motion History Image. We think extracting the key frames could enhance both the video detection accuracy and speed.


Objective: To describe and explore pediatric ambulatory video visit use by patient characteristics during the coronavirus disease 2019 (COVID-19) pandemic.Methods: We conducted an explanatory sequential mixed methods study with integration at the design and methods level. Phase 1 was a cross-sectional analysis of general and specialty pediatric ambulatory encounters to profile the use of video visits by patient characteristics. We performed descriptive analyses for each variable of interest and estimated a multivariable logistic regression model to analyze factors associated with the odds of having a video visit. Phase 2 was a qualitative exploration using semi-structured interviews with healthcare team members to understand the contextual factors influencing video visit usage. We used an interview guide to solicit information related to general perceptions about ambulatory video visits, reactions to the quantitative phase data, and strategies for optimizing equitable reach of video visits. Data were analyzed using a combination of deductive and inductive analysis.Results: Among the 5,464 pediatric ambulatory encounters completed between March 11 and June 30, 2020, 2,127 were video visits. Patient factors associated with lower odds of having a video visit rather than an in-person visit included being Spanish-speaking  and other non-English-speaking  in comparison to English-speaking. Patients with public insurance also had a lower odds of having a video visit in comparison to privately insured patients . Qualitative interviews identified five solution-based themes: (1) Promoting video visits in a way that reaches all patient families; (2) Offering video visits to all patient families; (3) Mitigating digital literacy barriers; (4) Expanding health system resources to support families' specific needs; and (5) Engaging and empowering health system personnel to expand video visit access.Conclusion: We identified differences in pediatric ambulatory video visit use by patient characteristics, with lower odds of video visit use among non-English-speaking and publicly insured patients. The mixed-methods approach allowed for the perspectives of our interview participants to contextualize the finding and lead to suggestions for improvement. Both our findings and the approach can be used by other health systems to ensure that all patients and families receive equal video visit access. Telehealth is defined as the use of medical information exchanged via electronic communications to support and provide health care . Live enThe coronavirus disease 2019 (COVID-19) pandemic has accelerated the adoption of video visit use for ambulatory patient care encounters –10, in gExperts have raised concerns that the expanded use of healthcare-related technology during COVID-19 can exacerbate healthcare disparities for vulnerable populations despite the promise for improving healthcare outcomes overall –16. TherWe conducted an explanatory sequential mixed methods study. This two-phase design began with a cross-sectional analysis of pediatric ambulatory encounters using electronic health records to profile the use of video visits . The second phase was a qualitative exploration using semi-structured interviews and a combination of inductive and deductive methods to generate contextualized understanding of video visit usage .In addition to implementing integration at the design level, we integrated at the methods level through connecting , wherebyThis study took place at a quaternary care academic health system with an integrated primary and specialty ambulatory clinic network in Northern California. This medical center is the referral center for children across a 33-county region covering 65,000 square miles and serving over 1 million children. More than 120 physicians provide ambulatory care to our pediatric patients and families.Our health system began offering pediatric ambulatory video visits in March 2019. Prior to COVID-19, the use of pediatric ambulatory video visits represented 1% of all pediatric ambulatory visits. We used the telehealth platform Epic MyChart  to conduct video visits. Parent or guardian (referred to as “parents” hereafter) requirements to conduct a video visit included establishing a MyChart account for the child and having a video-enabled smart device with WiFi or cellular access. The clinician requirements to conduct a video visit included having an iOS smart device with WiFi or cellular access. Clinicians could not use a desktop computer nor Android device, as these equipment types were not supported by our telehealth platform.The quantitative phase of this study began during the initiation of a statewide COVID-19 shelter-in-place order. Initially, the general pediatric clinic tried to restrict in-person visits to children 2 years and younger. This policy was lifted after 2 months and did not occur in specialty clinics. General and specialty clinic physicians reviewed their scheduled patients to determine which appointments could convert to video; this decision was at the discretion of the physician. Two months into the study, safety protocols were implemented , and the pediatric clinics began scheduling new appointments as in-person visits again. Visitation restrictions permitted one parent to attend the in-person visits with the child.For this cross-sectional analysis, we included all patients age 0–25 years who completed an ambulatory encounter with a Department of Pediatrics physician between March 11 and June 30, 2020. We included both in-person and video visits. All general and specialty ambulatory pediatric clinics located on the medical center campus were included; we excluded satellite and outreach clinics. Ambulatory encounters for procedures were excluded. We included patients up to age 25 years in order to include the young adults seen by pediatric physicians.We obtained patient-level data from the electronic health record. Patient characteristics included age, gender, race/ethnicity, language, insurance status, clinic type , and driving distance from residence to clinic (determined using the Google Cloud Platform's Distance Matrix API).We performed descriptive analyses for each variable of interest. For each categorical characteristic, we compared data between in-person visits vs. video visits using Pearson's chi-square tests. For each category, we calculated the proportion of video visits as a ratio of the number of video visit encounters (numerator) to the number of video visit plus in-person visit encounters (denominator). We estimated a multivariable logistic regression model to analyze factors associated with the odds of having a video visit, including covariates from the univariate analyses.We initially used purposive sampling  to identOne-on-one interviews were conducted from August through November 2020. Participants were recruited via e-mail. Eligible participants were aged 18 years and older and English-speaking. We collected demographic information during the interviews, including age, gender, race, ethnicity, clinical role, and years' experience. All interviews were conducted via videoconference and audio recorded and transcribed. Interviewers used an interview guide to structure the interview. The guide included questions to solicit information related to the following three topics: (1) general experiences with and perceptions about ambulatory video visits, (2) reactions to the video visits usage data from the quantitative phase, and (3) strategies for optimizing equitable reach of video visits. Interviewers maintained field notes with contextual observations. Each participant provided verbal informed consent and received a $50 gift card. Interviews were conducted until thematic saturation was reached.We used a combination of deductive and inductive analysis . Data weData validation occurred through investigator triangulation . The quaDuring the 16-week study period, there were 5,464 pediatric ambulatory encounters, of which 3,337 were in person and 2,127 were video visits. As shown in In multivariable analysis, patient factors associated with higher odds of having a video visit rather than an in-person visit included age > 18 years, English-speaking, private insurance, and non-surgical specialty clinic . Latinx/n = 3), IT (n = 2), professional medical interpreting (n = 1), physicians (n = 2), marketing (n = 1), triage nurses (n = 1), clinic nurses or managers (n = 2), and medical office service coordinators (n = 4). Characteristics of participants are provided in We conducted sixteen ~30–45-min interviews with individuals representing the telehealth program  Promoting video visits in a way that reaches all patient families; (2) Offering video visits to all patient families; (3) Mitigating digital literacy barriers; (4) Expanding health system resources to support families' specific needs; (5) Engaging and empowering health system personnel to expand video visit access. These themes are explored in more detail below with representative quotes in Multiple participants across different roles reported that the awareness of video visits was not necessarily known by all of our diverse groups of patient families. Specifically, participants shared that lower-income, publicly-insured, and non-English-speaking patients were frequently less aware of the video visits option. This lack of awareness was thought to be a contributing factor for the quantitative results regarding lower odds of video visit usage among non-English-speaking and publicly-insured patients. Participants stated that limited awareness existed in part due to marketing mostly being in English and via the medical center's website or social media platforms, which thus may not reach those populations noted in the quantitative data of having lower use.Another aspect highlighted by participants was the need to address parents' skepticism about video visits. They perceived that parents of publicly-insured children sometimes misunderstood their insurance coverage and feared that a video visit was more costly than the in-person charges (or lack thereof) to which they were accustomed. Participants also mentioned that non-English-speaking parents did not always know that a professional medical interpreter was available for video visits. These factors were thought to drive certain parents to preferentially use in-person visits.Many of the participants shared that clinical providers and staff offered video visits less frequently to certain groups of patient families. Participants explained that inconsistent workflows and training on to whom to offer video visits resulted in this practice. In addition, almost every participant commented that navigating the video visit process was challenging. Therefore, any additional element that could make the process even more difficult—whether it be a real or perceived element—exacerbated the selective offering of video visits. Participants said that biases about a family influenced the perceptions on who was more or less likely to navigate the process. In particular, English-speaking parents and those profiled as “tech savvy” were often categorized as people who would be successful in navigating video visits. Additionally, video visits for patients in foster care required the foster parent to complete additional paperwork and this may lead to video visit failure. Thus, video visits were almost never offered to children in foster care. Some participants emphasized the need to standardize the video visit scheduling process in order to overcome the practice of selectively offering video visits to the patient families deemed most likely to successfully navigate the video visit process.The most recurring topic discussed by all participants was that the video visit platform, MyChart, was very “difficult” and “cumbersome” to navigate. The most challenging aspect of using MyChart in the context of setting up video visits was parents' creating the account. As a result, patients whose parents could not set-up MyChart were unable to conduct their ambulatory encounter as a video visit and defaulted to in-person visits. Furthermore, the MyChart platform was only available to our patient families in English, and assistance in navigating the process for non-English-speaking parents was variable. This language limitation was thought to be a primary driver of the language gap identified in the quantitative data. Numerous participants stated that a multi-language platform is a necessary investment that should be prioritized.The need to expand video visit resources to better support our patients and families was consistently expressed among all participants. First, the video visit written instructions were perceived to be overly complicated and not useful for those with low health and digital literacy. Additionally, the instructions were almost exclusively in English. Participants reported that some fliers were translated into Spanish but not into other languages. However, again, they emphasized that language appropriate instructions have limited effect when the MyChart platform was only in English.Second, participants across various roles shared that they needed more clinical staff and IT personnel than was provided to assist patients and their families with the video visit process. In the setting of COVID, the exponential growth of video visits has surpassed their ability to address every patient family's needs. Participants stated that, for certain patients such as those who did not speak English, video visits required more time and effort from the health system in comparison to in-person visits. As a result, participants stated that “it's too much work” to do a video visit for certain patients. Additionally, video visits required extra steps on the family's end. Some parents who wanted to have a video visit for their child were left feeling frustrated and ultimately gave up despite assistance from the health system. Others could not have a video visit due to lack of equipment or insufficient internet access.Some participants articulated the need to raise awareness about the differential uptake of video visits by certain groups of patient families. They explained that frontline providers and administrators were likely not aware of the differences in video visit access in certain populations. Very few of our participants responded that there were differences in uptake of video visits by groups of patients defined by insurance or language status when asked the open-ended question about their observations of any differences in video visit use depending on patient or family characteristics. However, upon being presented with the quantitative analyses of this current study, almost every participant said, “I'm not surprised.” They requested that these analyses be disseminated so that data transparency could influence individual- and systems-level changes. In addition to increasing awareness, participants recommended empowering all providers to advocate for their patients' needs and to provide providers with the necessary resources to best support their patient families.This mixed methods study profiled pediatric ambulatory encounters during COVID-19 and identified that being non-English-speaking and having public insurance were independent risk factors for lower odds of having a video visit rather than an in-person visit. The subsequent qualitative exploration provided a contextualized understanding of video visit usage and identified various factors perceived to contribute to the video visit access inequities. Our solution-based themes identified that strategies to improve equitable access include expanding the reach of video visit promotions, standardizing the process of offering video visits, enhancing resources to support all patient families, and engaging all health system personnel to address inequities.The data from our present study adds to the growing body of evidence suggesting that the expanded adoption of telehealth during COVID-19 may be taken up by some groups of patients more than others. Our findings, that insurance and language are predictors of video visit use, are consistent with previous non-pediatric research highlighting this differential use. Data from one study examining internal medicine primary care visits suggested decreased video visit access among patients who were publicly-insured, non-English-speaking, and non-Hispanic White . FurtherThe limited pediatric-specific research on video visits during COVID-19 has examined video visit utilization among single specialties and has mixed results. One study of pediatric neurology encounters found video visits to be less frequent among patients in racial or ethnic minority groups . AnotherFindings from our current mixed-methods study were presented internally and activated immediate actions to begin addressing the identified differential uptake of video visits by certain groups . First, As our process of disseminating our findings internally reflects best practices, the strategies identified by our study participants to improve video visit use by all of the patient families reflected in our study are consistent with strategies being promoted by national public agencies. The Centers for Disease Control and Prevention COVID-19 Response Health Equity Strategy emphasizes the importance of disseminating materials that are tailored to be culturally relevant and linguistically relevant for diverse groups . SimilarThe findings in our study were specific to our medical center and may reflect circumstances and contextual factors that were unique to our setting. Other medical centers might not identify the same patterns that we found in our study. Nevertheless, many of the findings are likely transferable to other pediatric ambulatory practices, and our study highlights the importance of conducting such investigations that explore potential disparities in video visit access and areas for improvement. Another limitation in our study was that the appropriateness of the type of visit was not assessed. We thus could not determine if a video visit vs. in-person visit was warranted for a particular encounter. The qualitative findings represent only the perceptions from our group of participants. However, we interviewed a diverse group with respect to their clinical roles and experiences. Interview participants shared their perceptions of patient and parent experiences; however, we did not include patients or parents in this study. Gathering data from the perspectives of patients and parents was beyond the scope of this present study but should be explored in future research. Addressing patient families' unique needs requires their input and perspectives. Importantly, future research that gathers data from patients and parents should ensure diversity in participants in order to understand how to best support diverse populations with telehealth services. Furthermore, participants who agreed to participate could also have extreme perceptions. Despite these limitations, this mixed methods study provided useful information to inform interventions to improve the pediatric video visit program and mitigate access inequities.In conclusion, our profile of pediatric ambulatory video visit use by patient characteristics during COVID-19 identified differences, with lower odds of video visit use among non-English-speaking and publicly insured patients. Our mixed-methods approach allowed for the perspectives of our interview participants to contextualize the finding and lead to suggestions for improvement. We found that expanded reach of video visit promotions, standardized offering of video visits, enhanced resources to overcome digital literacy barriers, expanded resources to support all patient families, and engagement of providers to address inequities are potential strategies that may be incorporated to improve equal access to video visits among our diverse patient population. Therefore, both our findings and the approach to obtaining them present models for other health systems to ensure that all patients and families receive equal opportunity to reap the benefits of video visits, during the COVID-19 pandemic and thereafter.The datasets presented in this article are not readily available because of patient anonymity. Requests to access the datasets should be directed to the corresponding author.The studies involving human participants were reviewed and approved by University of California Davis IRB. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements. Written informed consent was not obtained from the minor(s)' legal guardian/next of kin for the publication of any potentially identifiable images or data included in this article.JR conceptualized and designed the study, conducted interviews, analyzed the data and interpreted the results, drafted the initial manuscript, and revised and approved the final manuscript as submitted. CO conceptualized and designed the study, conducted interviews, interpreted the results, and revised and approved the final manuscript as submitted. AS and EFG analyzed the data, interpreted the results, and revised and approved the final manuscript as submitted. All authors contributed to the article and approved the submitted version.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


Arteriovenous malformation (AVM) is a vascular lesion with a direct communication between an artery and a vein without a capillary system. AVM primarily occurs in the intracranial area, but can also occur in the extracranial area. If there is a tender mass with pulsation or recurrent bleeding in the auricle, AVM should be considered even though it rarely occurs in the auricle. AVM in the ear should be managed carefully because the skin is thin in the ear, the cartilage could be involved, and progressive growth or inappropriate management could cause bleeding, infection, and cosmetic problems such as deformity. I present a case of a 59-year-old woman with AVM in the auricle. Arteriovenous malformation (AVM) is a vascular lesion with a direct communication between an artery and a vein without a capillary system. Mulliken and Glowacki classified vascular lesions into hemangiomas and vascular malformations –3. A hemAVM primarily occurs in the intracranial area, but can also occur in the extracranial area. Extracranial AVM are most frequently identified in the head and neck region; the most common site is the cheek followed by the ear , 6. AVM A 59-year-old woman visited my otorhinolaryngology clinic for a cystic mass which had caused intermittent pain in the right auricle for several years. She had no history of trauma or infection in that area. On physical examination, a soft and cystic mass with purple skin discoloration was observed in the helix of the right auricle . Facial Under local anesthesia, the skin was incised, and the whitish soft cystic mass was dissected and removed . The masThe pathogenesis of AVM has not been elucidated , 2. It iSchobinger classified AVM into four stages. In stage I (quiescence), the symptom is warm and discolored skin; in stage II (expansion), the symptoms are bruit, pulsation, and swelling. In stage III (destruction), there is pain, ulceration, and bleeding; and stage IV (decompensation) is characterized by cardiac failure . Based oPlain X-ray and computed tomography have limitations for evaluating AVM. Magnetic resonance imaging and magnetic resonance angiography are the best approaches for evaluating the extension and confirming the diagnosis. In angiography, AVM shows tortuous and dilated arteries with arteriovenous shunts and dilated draining veins .The most common symptoms of AVM in the auricle are pulsation, bleeding, and pain. Treatment is not necessary if the AVM is small and does not induce symptoms . TreatmeIn the auricle, blood is supplied by superficial temporal, posterior auricular, and occipital arteries. The nidus of the AVM is usually supplied by one or more of these main vessels . With emThe incidence of AVM in the auricle is low, and there are limited reports on the guidelines for managing it. Vilela Chagas Ferreira et al. reported a protocol for resection and reconstruction of the AVM in the auricle. They recommended total resection with cartilage preservation for the lesion without cartilaginous involvement; total resection with cartilage resection was recommended for partial involvement of the auricle with cartilaginous involvement; auricular amputation was recommended for total involvement of the auricle or extra-auricular involvement with cartilaginous involvement. Preoperative embolization was not recommended for partial involvement of the auricle without cartilaginous involvement. Consequently, immediate or delayed reconstruction might be necessary .If there is a tender mass with pulsation or recurrent bleeding in the auricle, AVM should be considered even though it rarely occurs in the auricle. For a symptomatic large-sized and cartilage-involved AVM, superselective embolization followed by surgical excision is necessary.


Wiener, MD (Video).Please watch a video commentary on “A Case Report of Capsular Contracture Immediately Following COVID-19 Vaccination,” ojab026_Supplementary_DataClick here for additional data file.