Their Vδ2+ VDJ-rearrangements are short and simple in the fetal thymus and gradually increase in diversity and CDR3 length along with development. So-called “public” versions of Vδ2+ TCRs are shared among individuals of all ages. However, it is unclear whether such frequently occurring “public” Vγ9Vδ2+ T cell clones are derived from the fetal thymus and whether they are fitter to proliferate and persist than infrequent “private” clones.Vγ9Vδ2+ TCRs were identified from Vδ2+ TCR-repertoires collected from 89 individuals, including newborns (cord blood), infants, and adults . Distance matrices of Vδ2+ CDR3 were generated by TCRdist3 and then embedded into a UMAP for visualizing the heterogeneity of Vδ2+ TCRs.Shared “public” Vδ2+ CDR3 distance matrix embedded by UMAP revealed that the heterogeneity of Vδ2+ TCRs is primarily determined by the J-usage and CDR3aa length, while age or publicity-specific motifs were not found. The most prevalent public Vδ2+ TCRs showed germline-like rearrangement with low N-insertions. Age-related features were also identified. Public Vδ2+TRDJ1 TCRs from cord blood showed higher N-insertions and longer CDR3 lengths. Synonymous codons resulting from VDJ rearrangement also contribute to the generation of public Vδ2+ TCRs. Each public TCR was always produced by multiple different transcripts, even with different D gene usage, and the publicity of Vδ2+ TCRs was positively associated with expansion status.Vδ2+ TCRs is mainly determined by TRDJ-usage and the length of CDR3aa sequences. Public Vδ2+ TCRs result from germline-like rearrangement and synonymous codons, associated with a higher expansion status.To conclude, the heterogeneity of Vδ2 TRG and TRD. The diversity of these CDR3 regions is further amplified by the insertion of palindromic sequences (P nucleotides) and additional non-templated nucleotides (N-insertions) introduced by terminal deoxynucleotidyl transferase (TdT)  consisting of both rearranged γ (TRG gene) and δ (TRD gene) chains. Like αβ T cells, γδ T cells use the recombination of variable, diversity, and joining gene segments (V(D)J recombination) to generate the complementarity-determining region 3 (CDR3) of the se (TdT) , 2.+ chains mostly pairing with Vγ9-JP chains -4-hydroxy-3-methyl-but-2-enyl pyrophosphate (HMB-PP) and host-derived Isopentenyl pyrophosphate (IPP) in a pMHC-unrestricted manner J recombination, frequency, and MHC restriction need to be considered in the analysis of this. Recently, different computational tools were developed to discover TCR clusters based on the sequence patterns –17. For + TCRs, we determined the publicity of TCRs from Vδ2+ TCR repertoires of 89 individuals from cord blood (CB), infant peripheral blood, and adult peripheral blood. Vδ2+ CDR3 amino acid (CDR3aa) sequences were embedded into the distance matrix by TCRdist3 and visualized by Uniform Manifold Approximation and Projection (UMAP). We found that both the J-usage and length together defined the heterogeneity of Vδ2+ CDR3aa sequences. Both germline-encoded and age-dependent features were preserved among public Vδ2+ TCRs, indicating that they are produced in the fetal and adult thymus. Interestingly, we additionally revealed a higher expansion status of public Vδ2+ TCRs than private Vδ2+ TCRs.To investigate the heterogeneity and ontogeny of public Vδ2+ clones, we collected TCR repertoires containing 213,391 Vδ2+ CDR3aa sequences from 11 cord blood (CB), 55 infant peripheral blood, and 23 adult peripheral blood samples. Eighty-one samples were collected from our published studies  and TRDJ3 (length 19aa) regions for a more in-depth re-analysis. This showed that publicity  . In contrast, both CB and adult groups largely shared their public Vδ2+ repertoire with the infant group    TCRs and then labelled the remaining as low frequency (low-freq) TCR’s . For a group of TCRs in one individual, the expansion status is calculated by dividing the number of high-freq TCRs in the group by the number of low-freq TCRs followed by a log-transformation. Hence, the higher the expansion status score, the more high-freq TCR’s in that group. An expansion status score of > 0 means the group has more high-freq TCRs than low-freq ones. Interestingly, the median expansion status score of “high public” TCRs was 0.37, and that of the “low public” TCR’s remained significantly higher than the private TCR values   and adult age groups were characterized to show only a few or no N-insertions and shorter CDR3length (TRDJ1 of public (but not of private) γδ TCRs in CB displayed a relatively high number of N-insertions and longer CDR3 lengths. Moreover, it was additionally revealed that, compared to private Vδ2+ CDR3aa sequences, the public Vδ2+ CDR3aa sequences were prone to be generated from multiple CDR3nt transcripts even within one individual. Thus, it could be concluded that germline-like rearrangement and synonymous codons used by CDR3nt sequences contribute to the generation of public CDR3aa. Finally, public Vδ2+ TCRs displayed a higher expansion status than private Vδ2+ TCRs.In this study, we applied TCRdist3 to systematically investigate the Vδ2R3length , 13. Une+ TCRs are instead generated from relatively fixed options and are thought to uniformly recognize phosphoantigens . There is a rearrangement bias. where the publicity of Vδ2+ TCR CDR3aa positively associates with the number of corresponding CDR3nt sequences. Therefore, the γδ T cells with a public Vδ2+ TCR may have multiple sources from different TCR rearrangements, resulting in a higher copy number. 2). Most public Vδ2+ TCRs, especially those shared between many individuals, are rearranged early in life and persist into adulthood J rearrangement. Thus, detecting the different patterns between αβ TCRs is considerably easier. In our case, the TCRdist3 detected the heterogeneity of Vδ2+ TCRs generated by length and J-usage, but not by publicity or age. Our sequence pattern analysis also failed to find heterogeneity between public and private TCRs. Furthermore, the existing possibility that more subtle and essential substitutions hiding in public Vδ2+ TCRs cannot be excluded. Currently, methods for TCR clustering are all based on the CDR3aa sequences, which is sufficient to study antigen-specificity. However, the ontogeny of TCRs can be better determined if CDR3nt sequences are included to provide crucial information about VDJ rearrangement and N-insertions.The TCRdist3 method has proven to be a very useful tool for analyzing human αβ T cells, and the software is able to support γδ TCR analysis . However+ TCRs are a heterogeneous population with both germline and age-related features that confer expansion advantages over private TCRs. Given that expressing γδ TCRs on αβ T cells is a promising immunotherapy strategy against tumors  and cord blood (CB) donors (n = 5) were collected at Hannover Medical School  after written informed consent. This study was performed in accordance with the Declaration of Helsinki and approved by the institutional ethics review board at Hannover Medical School under study numbers 1303-2012  and 7901-2018 . PBMCs and CBMCs were purified from the blood samples by Ficoll-Paque density gradient media separation. These cells were then stored at -80°C in 90% fetal bovine serum and 10% DMSO freezing medium before use.Fluorescence-activated cell sorting (FACS) was performed using the FACS Aria Fusion flow cytometer . PBMC and CBMC were incubated with 5% Fc-receptor block before staining. The following antibodies were used: anti-CD3 , anti-CD3 , anti-γδ TCR , anti-Vγ9 , anti-Vδ2 .─CD3+γδ+Vγ9+Vδ2+ cells from PBMC or CMBC by an RNAeasy Micro Kit (Qiagen). Reverse transcription was carried out with Superscript III reverse transcriptase (Invitrogen) and oligo(dT) primers. As previously described (via TRDV2 specific primers hTRDV2: ATTGCAAAGAACCTGGCTGT and hTRDC: GACAAAAACGGATGGTTTGG. The PCR program was set as follows: 1). 95°C for 3 min; 2). 95°C, 63°C, and 72°C for 30s each, for 5 cycles; 3). 95°C for 30s, 72°C for 35s, for 20–25 cycles; 4). 72°C for 4 min.All the newly generated data was sequenced and pre-processed in the same way as other published data used in this study , 20, 21.The amplified cDNA library with Illumina P5 and P7 adaptor was sequenced by Illumina Miseq using 500 cycles of paired-end sequencing.Raw reads alignment annotation was performed with MiXCR software v.2.1.12 to international immunogenetics information system (IMGT) reference . Unproduvia VDJtools output as following: For TCRs rearranged with a TRDD segment: N-insertion = (Jstart – Dend – 1) + (Dstart – Vend - 1); For TCRs without D-usage: N-insertion = Jstart – Vend – 1.VDJtools output files from all the 89 individuals from the published and newly generated datasets were merged together. Since some datasets comprised entire TCRδ repertoires , 21, nonVend, Dstart, Dend, Jstart are the start/end position of V, D, J segments on CDR3nt sequence.Publicity of TCRs were defined based on the CDR3aa sequence by whether a sequence is shared among a certain percentage of the population. “private” CDR3aa is defined as CDR3aa that only appears in only one individual, “high public” TCRs are shared among at least 10% of the population, i.e. shared among 9 or more individuals in our study, the remaining TCRs are defined as “low public”, i.e. shared by at least 2 individuals to 10% of the population.+ CDR3aa sequences with a length from 14 to 22 aa were preselected and downsampled for TCR distance calculation. CDR3s rearranged with TRDJ4 segment were excluded. For each age group, the numbers of CDR3s were randomly down-sampled to 17,398 - 17,401 sequences. TCR distances were computed according to the protocol of TCRdist3 (Vδ2TCRdist3 . Brieflyi.e. the high public TRDJ1 Vδ2+ TCR in the donor CB2) as follows:In each individual, CDR3aa sequences were ranked by the frequencies from high to low. The top 25% of CDR3s were assigned as “high frequency” TCRs, and the rest were labelled as “low frequency” TCRs.  Research Unit FOR 2799 to SR and IP (Project ID 395236335) and under Germany’s Excellence Strategy, EXC 2155 “RESIST,” (Project ID 390874280) to IP and SR.We thank Ms. Alexandra Morse for proofreading this manuscript. We would like to acknowledge the assistance of the Cell Sorting Core Facility at the Hannover Medical School supported in part by Deutsche Forschungsgemeinschaft and the IT department at Center for Molecular Neurobiology Hamburg, University Medical Center Hamburg-Eppendorf.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The current study investigated the effects of Chinese young adult users’ perceived information overload  on their intention to stop using short-form video applications. Specifically, this study accomplished this by measuring the direct and indirect effects of social media fatigue, maladaptive coping, and life dissatisfaction in relation to users’ intention to discontinue their use of short-form video applications. The data were collected using a web-based survey and validated questionnaire, with a sample of 340 young adult (18–26 years old) respondents. The results indicated that perceived information overload had a direct effect on the intention to discontinue the use of short-form video applications. Moreover, short-form video fatigue, maladaptive coping, and life dissatisfaction all played mediating roles in the relationship between perceived information overload and the intention to discontinue the use of short-form video applications among young adults in China. In recent years, the influence of social media has risen continuously . DespiteIn recent years, short-form video applications have been recognized as the fastest-growing forms of social media . They arThe discontinued use of media has been discussed thoroughly in the field of digital and social media ,19. Fu eTo fill this gap, this study drew on stress-coping theory as a guide, further exploring the effect of the perceived information overload from short-form video applications on Chinese young adult (18–26 years old) users’ intent to stop using said applications. This study also explored the mediating roles of short-form video fatigue, maladaptive coping, and life dissatisfaction in this process.Lazarus and Folkman defined coping as an individual’s response to stressful situations involving important and potentially negative consequences . SeveralIn recent years, cutting-edge new media platforms have greatly attracted Chinese young adults’ interest. Short-form video applications are among the most frequently used communication platforms in China . The conFarhoomand and Drury  claimed One of the critical factors that predicts individuals’ discontinued use of social media is social media fatigue . Social Two types of coping strategies are involved in efforts to release stress from various stressful events. Among these coping strategies, maladaptive coping strategies were largely applied by Chinese social media users when they experienced information overload . In our The factors that determine an individuals’ life dissatisfaction include health behavior, social factors, personality, and mood factors . The curH1. Perceived information overload has a significant and positive effect on (a) the intention to discontinue the use of short-form video applications, (b) short-form video fatigue, (c) maladaptive coping, and (d) life dissatisfaction.A handful of researchers have found that there is a positive relationship between individuals’ perceived information overload and their levels of social media fatigue ,27,45 FuHowever, previous studies related to media and behavior have demonstrated that social media users’ perceived information overload increases their level of maladaptive coping ,53,54. FH2a. Short-form video fatigue mediates the relationship between perceived information overload and the intention to discontinue the use of short-form video applications.H2b. Maladaptive coping mediates the relationship between perceived information overload and the intention to discontinue the use of short-form video applications.Scholars have discussed users’ excessive engagement with social media and how being exposed to information overload leads to increases in their life dissatisfaction  . EventuaAs mentioned above, previous literature has explored life dissatisfaction as one of the key psychological factors that indirectly affects this causal relationship. Therefore, this study argues that life dissatisfaction could mediate the effect of the relationship between perceived information overload and the intention to discontinue the use of short-form video applications. Hence, the following hypothesis was thus proposed :H3. Life dissatisfaction mediates the relationship between perceived information overload and the intention to discontinue the use of short-form video applications.The current study adopted 5 measures  from existing studies. In addition, all questions related to each measurement were adapted from verified studies ,24,55,56To observe perceived information overload, we derived the information overload scale from Misra and Stokols’s  study. TThe short-form video fatigue scale was adapted from Bright et al. . The shoThe 3-item maladaptive coping scale was adapted from Lin et al. . This scTo measure life dissatisfaction, we adapted the life dissatisfaction scale created by Diener et al. . The oriThe intention to discontinue the use of short-form video applications scale was adapted from Maier et al. . This scIn total, 340 valid responses were collected. The demographic characteristics of the survey participants are outlined in p < 0.001), short-form video fatigue , maladaptive coping , and life dissatisfaction  were all found to be significant. Therefore, H1(a–d) are fully supported  was used to test the mediation analysis of the cognitive and psychological factors’ effects on the relationship between perceived information overload and the intention to discontinue the use of short-form video applications among Chinese young adults. The current study applied bootstrapping to obtain bias-corrected 95% confidence intervals in order to make statistical inferences about specific indirect effects. p < 0.001). Meanwhile, perceived information overload did not predict the intention to discontinue the use of short-form video applications . The indirect effect was significant . Therefore, the full mediation effect of short-form video fatigue was confirmed. The second mediation model’s result indicated that maladaptive coping positively predicted the intention to discontinue the use of short-form video applications . Perceived information overload did not predict the intention to discontinue the use of short-form video applications . The indirect effect was significant . Thus, the full mediation effect of maladaptive coping was confirmed. The third mediation model showed that life dissatisfaction positively predicted the intention to discontinue the use of short-form video applications . Perceived information overload significantly predicted the intention to discontinue the use of short-form video applications . The indirect effect was significant  The partial mediation effect of life dissatisfaction was confirmed. Therefore, H2 and H3 were fully supported.The results show that, in the first mediation model, short-form video fatigue bias positively predicted the intention to discontinue the use of short-form video applications  and psychological (life dissatisfaction) factors on Chinese users’ intention to stop using short-form video applications.In terms of direct effects, perceived information overload was positively associated with Chinese young adults’ discontinued use of short-form video applications. This finding is consistent with those of previous studies ,12. For In regard to the mediated effect, firstly, short-form video fatigue mediates the relationship between perceived information overload and Chinese young adults’ discontinued use of short-form video applications. This shows consistency with previous studies, which have demonstrated that social media users’ perceived excessive exposure to COVID-19-related information has increased their level of social media fatigue. Ultimately, this has further reinforced users’ intentions to stop using social media ,52. SecoThe practical implications of this study relate to three aspects. Firstly, even though short-form video applications have brought a lot of benefits to Chinese people amid the COVID-19 pandemic, they have led to negative effects that are amplified by information overload. Therefore, the providers of short-form video applications should be aware of the potential negative outcomes. Additionally, reducing the number of daily notifications and instead setting more user-friendly alerts, such as push notifications, should ensure that users receive a limited amount of daily information. Secondly, the current study provides empirical evidence that short-form video fatigue is one of the key mediators in the relationship between perceived information overload and the discontinued use of short-form video applications. Moreover, spending time on these applications may induce individuals to experience short-form video fatigue, resulting in users quitting short-form video applications. Therefore, these applications’ providers should also pay more attention to the users’ time limitation when using these applications. In addition, the applications’ service providers can add features to applications that remind users of the recommended healthy amounts of time to spend on these applications. Social support has been shown to be one of the key factors that can offset the level of depression (life dissatisfaction) . TherefoThere are several limitations of this study. Firstly, the analysis was conducted using a cross-sectional study. In this type of investigation, it is difficult to identify the direct and indirect effects of the tested factors. Secondly, the respondents’ education was mainly at the undergraduate level. Therefore, these findings cannot be generalized to young adult users in China from all education groups. Lastly, the current study adapted the original scales for measuring individuals discontinuing the use of short-form video applications. Undeniably, it is difficult to measure their actual behavior change. Instead, this study used the perspective of “expected” or “perceived” as a behavior change. A more accurate measurement would be beneficial for future studies.Future research needs to further investigate the impacts of cognitive and psychological factors. For instance, whether self-efficacy is directly or indirectly associated with the discontinued use of short-form video applications should be analyzed. Similarly, whether an individual’s social comparison is positively or negatively associated with users avoiding the use of short-form video should also be investigated. These potential future studies will contribute to providing a holistic perspective of short-form video users’ behavior.The current study discovered how information overload affects Chinese young adult users’ (18–26 years old) intention to discontinue using short-form video applications during the COVID-19 pandemic in China. This is the first study to explore both the direct and indirect effects of critical factors  on such a relationship. The findings of this study make contributions to behavioral science research in the following aspects: First, the current study fills a gap in the prior literature on the discontinued use of short-form video applications in the context of the COVID-19 pandemic in China. Second, this study confirms that both cognitive  and psychological (life dissatisfaction) factors act as mediating mechanisms in the relationship between information overload and Chinese young adult users’ intention to discontinue using short-form video applications. Future studies are recommended to investigate whether self-efficacy and social comparison are directly or indirectly associated with discontinuing the use of short-form video applications. The exploration of these factors will provide a better understanding of the negative outcomes of Chinese short-form video application users’ problematic media use.


Scientific Reportshttps://doi.org/10.1038/s41598-022-17679-x, published online 13 September 2022Correction to: The Supplementary Video S1 published with this Article contained an error, where the text displayed between 0:58 second and 1:10 minute of the video incorrectly stated that the aerial tool "The Mamba" has led to a new species description. The original Supplementary Video S1 is provided below.“This remote controlled tool has enabled rare plant collections from more than 1 km away and has led to a new species description.”now reads:“This remote controlled tool has enabled rare plant collections from more than 1 km away and has helped find potential new species.”This error has now been corrected in the Supplementary Video S1 file that accompanies the original Article.Supplementary Video S1.


Since their emergence a few years ago, artificial intelligence (AI)-synthesized media—so-called deep fakes—have dramatically increased in quality, sophistication, and ease of generation. Deep fakes have been weaponized for use in nonconsensual pornography, large-scale fraud, and disinformation campaigns. Of particular concern is how deep fakes will be weaponized against world leaders during election cycles or times of armed conflict. We describe an identity-based approach for protecting world leaders from deep-fake imposters. Trained on several hours of authentic video, this approach captures distinct facial, gestural, and vocal mannerisms that we show can distinguish a world leader from an impersonator or deep-fake imposter. In the early days of the Russian invasion of Ukraine, President Zelenskyy warned the world that Russia’s digital disinformation machinery would create a deep fake of him admitting defeat and surrendering. By mid-March of 2022, a deep fake of Zelenskyy appeared with just this message . This viIn addition to adding jet fuel to disinformation campaigns, this new breed of synthetic media also makes it easier to deny reality—the so-called liar’s dividend —as seen These recent events are likely just the beginning of a new assault on reality in both recorded and live videos. While we may have the ability to perceptually detect some deep-fake videos , our abiThe computational detection of deep-fake videos can be divided into three categories: 1) learning-based, in which features to distinguish the real from fake are explicitly learned ; 2) artiThe advantage of identity-based techniques is that they are resilient to adversarial and laundering attacks and are applicable to the many different forms of deep fakes. The disadvantage is that these techniques require an identity-specific model, generated from several hours of authentic video footage. Because we are focused on protecting world leaders for whom hours of video can easily be acquired, we contend that an identity-based approach is the most sensible and robust approach.We describe an integrated facial, gestural, and vocal model that captures an individual’s distinctive speaking mannerisms. After training a model on several hours of authentic video, the model can be used to distinguish one person from impersonators and deep-fake imposters. Given the recent attacks on President Zelenskyy—and the associated risks in the fog of war—we describe an in-depth analysis of the efficacy of this model in protecting President Zelenskyy against deep-fake attacks. Our approach, however, can be deployed to protect any world leader or other high-profile public figures.Shown in With only a single feature set, the classifier struggles to consistently distinguish Zelenskyy from decoys and deep fakes. The pair-wise combined features (rows 4–6) provide a boost in accuracy with the triple features (row 7) providing a significant boost yielding perfect accuracy across all datasets. Note that the vocal features provide no benefit for the FaceForensics++ dataset because these videos contain no audio track.To determine how many of the 780 behavioral features are needed to achieve the classification accuracy reported in n corresponds to vocal characteristics).We next trained 4,000 classifiers on random feature subsets of size 10. The discriminatory power of each feature is computed as the weighted average of each classifier accuracy to which a feature contributed. Classifier accuracy ranges between 18.1% and 1.2%, with a median accuracy of 2.4%. The following are five most and least discriminative features’ correlation and respective accuracy  unlikely. We also benefit from a relatively long analysis time window (10 s). By comparison, synthesis engines (currently) generate only one video frame, or a few, at a time and are therefore unaware of these longer temporal patterns. Despite this seemingly upper hand, we do not expect to publicly release our classifier so as to slow counterattacks. We will, however, make our classifier available to reputable news and government agencies working to counter deep-fake-fueled disinformation campaigns.Although we have focused our analysis on one world leader, our approach is applicable to any world leader or high-profile individual for whom sufficient authentic video is available. Our experience has been that at least 8 h of training video is required. Deep fakes continue their trajectory of creating increasingly higher quality fake videos while continually lowering barriers to creation in terms of data, computing power, and technical skill required. This democratized access to powerful video synthesis technology is sure to yield exciting and entertaining applications while simultaneously posing new threats to our ability to trust what we see and hear online.We downloaded 506 min of video of President Zelenskyy from YouTube and the official website of the office of the Ukrainian presidentA total of 57 min of interview-style videos of seven world leaders  were used as decoys . Our deep-fake detection is designed to distinguish Zelenskyy’s behavioral and gestural mannerisms from imposters driving the creation of a deep fake, so these decoy videos—regardless of the identities—serve as proxies for deep fakes. An additional 63 min of video from 250 distinct individuals in the FaceForensics++ FaceShifter dataset  was usedFor each video segment, the OpenFace2 toolkit  extractsx-, y-coordinates corresponding to the shoulder, elbow, and wrist of both arms, yielding a total of 12 individual measurements per video segment. These upper-body coordinates, initially specified relative to the video-frame size, are normalized into a speaker-centric action plane , the mel frequency cepstral coefficients (MFCCs)  are extr40C2 = (40 × 39)/2 = 780 correlations are extracted from overlapping 10-s video segments extracted from an input video.Correlations between all pairs of the 20 facial, 12 gestural, and 8 vocal features are used to capture individualized mannerisms. A total of γ) and outlier percentage (ν), are optimized by performing a grid search over these parameters across the training set. The validation set is used to determine the SVM classification threshold that yields a specified true positive rate (correctly classifying a video segment as Zelenskyy). The classifier is then evaluated against the testing set. This entire process is repeated 10 times with randomized data splits, from which we report average classification accuracy in Trained on authentic video of a person of interest, a novelty detection model in the form of a one-class, nonlinear support vector machine (SVM)  is used


This is a difficult task, but it is important to automate, improve, and lower the cost of the detection of crimes and other accidents. The UCF–Crime dataset is currently the most realistic crime dataset, and it contains hundreds of videos distributed in several categories; it includes a robbery category, which contains videos of people stealing material goods using violence, but this category only includes a few videos. (2) Methods: This work focuses only on the robbery category, presenting a new weakly labelled dataset that contains 486 new real–world robbery surveillance videos acquired from public sources. (3) Results: We have modified and applied three state–of–the–art video surveillance anomaly detection methods to create a benchmark for future studies. We showed that in the best scenario, taking into account only the anomaly videos in our dataset, the best method achieved an AUC of 66.35%. When all anomaly and normal videos were taken into account, the best method achieved an AUC of 88.75%. (4) Conclusion: This result shows that there is a huge research opportunity to create new methods and approaches that can improve robbery detection in video surveillance. The research area of video surveillance anomaly detection is a promising visual understanding area with several real applications. The goal is to learn the normal pattern from surveillance videos and detect situations that deviate from normality ,5,6,7,8.A system that detects one or more situations that are considered abnormal could be designed using supervised deep learning, but the main difficulty in this case is the need for a huge dataset, since the occurrence of such events in this context is rare in the video data. Furthermore, even if a big dataset was available, the labelling process is expensive, since the annotations have to be made at the frame level to indicate exactly where the abnormality starts and ends in each video.To deal with these problems, a classical approach is used to consider the anomaly detection process as a reconstruction problem ,12,13. CHowever, these methods provide poor performance for real–world videos. Thus, aiming to improve real–world anomaly detection, Sultani et al.  proposedA dataset containing 486 new real–world video surveillance clips showing robberies that were acquired from public sources such as social media is presented.A benchmark is created through the application of three state–of–the–art video surveillance methods that were originally applied to the UCF–Crime dataset. These methods were standardised for a fair comparison, which was not performed in the original UCF–Crime comparison.Thereafter, several publications made progress in real–world anomaly detection, but because UCF–Crime has 13 anomaly categories, each type of anomaly has a few video clips. Taking into account that human–related crimes  represent a problem that must be alleviated, this paper focuses on the robbery anomaly and proposes a new dataset for this type of crime. Thus, the main contributions of this work are as follows:The remainder of this text is organised as follows: In Classical Methods: In this category are the non–deep–learning–based methods, such as that proposed in , which fHybrid Methods: These methods aim to model some part of the anomaly detection using classical models and deep learning. Rezaei and Yazdi  used a cDeep Learning Methods: These approaches are based only on deep learning. Singh and Pankajakshan  used a COptimisation based on the reconstruction error is widely used in video surveillance anomaly detection because only normal videos are needed in the training phase and there is no need to label these normal videos. Several approaches belong in this paradigm:Since the reconstruction error approach cannot provide good results on real–world datasets ,2,3, Sulk segments from each video. Furthermore, this method utilised an attention mechanism called non–local neural networks [In , the robnetworks  to aggrenetworks . Other wnetworks ; they usOther datasets were proposed in the literature to allow the development of video surveillance anomaly detection methods, but these datasets are limited in terms of the camera variation, resolution, point of view, weather conditions, etc. They offer alternative benchmarks for current academic works on video surveillance anomaly detection. The UCSD dataset  containsIn Each video is called a bag and is divided into several segments. In the training phase, a deep neural network (DNN)–based model receives the following two input videos: (i) a video containing at least 1 anomaly segment, and (ii) a video containing no anomaly segments. The output of the DNN is an abnormality score for each segment of the two bags (videos). The objective function aims to increase the distance from the higher–scoring segment in the positive bag to the higher–scoring segment in the negative bag, decreasing the number of false negatives and increasing the number of true positives. The videos need to be weakly labelled, which requires much less effort than normal labelling. Thus, this approach uses information from abnormal videos in the training process, unlike the reconstruction error approach, providing an increase in the performance [The CamNuvem dataset consists of 972 videos in total, including 486 abnormal videos collected from public sources such as YouTube, Facebook, and other social media websites. The videos were edited to remove useless segments that were not recorded by the surveillance camera, such as introductions and endings. All the abnormal videos consist of people committing robberies. This category of videos was split into 437 videos for the training set and 49 videos 10%) for the test set. As with UCF–Crime [0% for thThere are also 486 normal videos extracted randomly from UCF–Crime; 437 of these videos were used for the training set and 49 were used in the test set. Thus, the main novelty of this dataset is the aggregation of a new robust set of abnormal videos to improve the development of robbery detectors, which can be used with datasets that are already available.All the videos were adjusted to 320 × 240 px and 30 fps. As shown in Furthermore, as shown in The values of the area under the curve (AUC) from the receiver operating characteristic (ROC) curve for the CamNuvem dataset were obtained using the video anomaly detection methods RTFM , WSAL 33, and RATraditional convolutional blocks, when applied to a single image, can aggregate features due to their hierarchical nature. Thus, a CNN can predict some value in the final layer by taking into account features and objects that are in the same image. However, a video may be seen as a set of images (frames) that are linked together in a well-defined order; i.e., each frame is linked to the posterior and previous frames, and information flows through these frames along the temporal axis. When a CNN is applied to a video dataset to extract information, the traditional convolution block can see only a single frame; it ignores the features in the neighbouring frames, and, therefore, on the temporal axis, which can represent an important information loss.A natural extension that allows the traditional convolution block to deal with temporal datasets (such as video datasets) involves adding a dimension to the convolutional kernels. This technique is known as convolution 3D (C3D) . In thisHowever, convolution 3D aims to acquire local and temporal features from datasets, and long–range features are aggregated together in the hierarchical neural network structure. This hierarchical structure has a negative impact on computational optimisation because the signals captured by single kernels represent information from a small patch of the image, and the only way to capture long–range dependencies between features is to feed this signal to the deepest layers repeatedly, causing computational inefficiency due to parallelisation constraints. This scenario also occurs in recurrent neural networks (RNNs), which are often utilised for textual datasets; long–range features have to be repeatedly passed through the forward layers to be aggregated together, causing computational difficulties and limiting the power of an artificial neural network.To confront this problem, non–local neural networks , which wT is the number of segments into which a video is split  and D is the dimension of the feature vector. Each F, the RTFM method predicts whether each segment is anomalous or not using the model k largest segment feature magnitudes from the normal videos and maximise the k largest segment feature magnitudes from abnormal videos:m is a predefined margin and i , where olutions  over theolutions  is used given bygθ,k(X)=mgiven bygθ,k(X)=mF has a feature vector extracted from some CNN. These feature vectors can be denoted by f and its g is a function that measures the anomaly score margin between the segments j and k). For the normal videos, it is intended that all segments should present a g value near 0. In the abnormal videos, it is intended that the g value should be higher because the anomalous segment will result in a high F is anomalous  module uses two components to mine variations from video features: (i) the semantic component and (ii) the variation component. For the semantic component, it is assumed that the videos can be interpreted as a time series, and the extraction of high–level features can be performed as a regression task. In this way, given feature vectors f1,f,…,fT extd segments. To do this, the variation component uses the metric The variation component pays attention to the variation from two adjacent segments, opposing the semantic component, which pays attention to the variation from a window of Then, the following second–order discrepancy is calculated to obtain the anomaly score:The two semantic components and variation components give context cues that can be used to detect abnormalities in video segments. The L1–distance is used to measure the variation from consecutive segments:Putting Equation  results quations  and 12)Ssem and iven by1S0=Ssem proposals to extract features based on human interaction in videos and (b) the investigation of methods that can be used to achieve better anomaly localisation. Thus, one can conclude that video surveillance anomaly detection has many research opportunities; new approaches are needed to make real–world applications possible.


Zeus to characterize this relationship in various realistic usage scenarios. Our measurement of task-level time consumption demonstrates the importance of the camera CPU-GPU transfer and the server initialization, as well as the negligible effect of 360° video stitching on the delay metrics. Finally, we compare Zeus with a commercial system to validate that our results are representative and can be used to improve today’s 360° video camera sensing systems.360° video camera sensing is an increasingly popular technology. Compared with traditional 2D video systems, it is challenging to ensure the viewing experience in 360° video camera sensing because the massive omnidirectional data introduce adverse effects on start-up delay, event-to-eye delay, and frame rate. Therefore, understanding the time consumption of computing tasks in 360° video camera sensing becomes the prerequisite to improving the system’s delay performance and viewing experience. Despite the prior measurement studies on 360° video systems, none of them delves into the system pipeline and dissects the latency at the task level. In this paper, we perform the first in-depth measurement study of task-level time consumption for 360° video camera sensing. We start with identifying the subtle relationship between the three delay metrics and the time consumption breakdown across the system computing task. Next, we develop an open research prototype  Camera sensing through visual sensors has been an important technology in wireless sensor networks ,2,3. Recstart-up delay, decreasing users’ willingness to continue the viewing. The start-up delay may in turn incur a long event-to-eye delay which is the time interval between when an event occurs on the remote scene and when it is displayed on the user’s screen. Long event-to-eye delay causes significant lags in streaming live events such as sports and business meetings and would be detrimental in applications that require real-time interaction. Moreover, the frame rate of the 360° video is determined by how fast frames can be pushed through the system pipeline. A low frame rate would lead to an unappealing viewing experience for the users.A critical requirement of 360° video camera sensing systems is the delay because it plays a pivotal role in ensuring users’ viewing experience. Complex system initialization may lead to an excessive It is challenging to guarantee acceptable delay and user experience in 360° video camera sensing systems. This is because 360° video camera sensing systems generate far more data than traditional 2D video camera sensing systems. A fundamental prerequisite to preventing undesirable user experience caused by delays is understanding how different components of a 360° video camera sensing system contribute to the aforementioned three delay metrics.Although many measurement studies have been conducted on 2D video camera sensing and streaming ,6,7, theThe goal of this paper is to fill this gap and conduct an in-depth measurement study of the time consumption across the end-to-end system pipeline in 360° video camera sensing. We aim to identify the bottleneck of a 360° video camera sensing system in terms of different delay metrics and call for design guidelines for optimized 360° video systems.Zeus. Zeus includes a 360° camera, camera-server transmission, a video server, server-client transmission, and a video client. Implemented by off-the-shelf hardware, publicly available SDKs, and open-source software packages, Zeus can be easily reproduced for future camera sensing research areas, e.g., simulation, modeling, and algorithm design.One key challenge of the proposed measurement is that commercial 360° video systems usually behave as black boxes. The closed-source nature of these systems makes it almost impossible to study the per-task delay of the 360° video camera sensing pipeline. In this paper, we overcome this challenge by building a 360° video camera sensing research prototype—Zeus and profile the delay characteristics of the system under various realistic scenarios. Our micro- and macro-benchmarks demonstrate four crucial findings. First, transferring video frames between the CPU and GPU in the camera is a critical task for reaching a satisfactory frame rate because this internal camera transfer consumes non-negligible time. Second, stitching a 360° video frame surprisingly has a negligible effect on the frame rate. Third, server initialization before the live video streaming is greatly time-consuming. This long start-up delay adds to a significant event-to-eye delay, which could lead to an annoying time lag between what happens in the remote scene and what is displayed on the screen. Fourth, we found across different real-world scenarios that camera motion is a deciding factor in the system delay and viewing experience. Overall, the 360° camera is the bottleneck for frame rate, whereas the server is the primary cause for low start-up and event-to-eye delay.We measure the time consumption of each computing task in all five system components on Zeus and commercial 360° video platforms are not exactly the same, the absolute values of the results obtained with Zeus may be different from those measured on commercial platforms. Hence, we additionally measure the delay on a commercial system that is based on Ricoh Theta V 360° camera and YouTube live streaming servers. We treat it as a black box and compare its component-wise time consumption with Zeus. The comparison shows a strong correlation between the two systems, indicating that our observations can be generalized to exiting 360° video camera sensing systems in the current practice.As the implementations of We identify the relationship between the three delay metrics and the time consumption breakdown across the system computing tasks .Zeus (https://github.com/junyiwo/Zeus (accessed on 27 June 2022)). The methodology for building Zeus can be reused in future 360° video research  streaming services over Internet such as YouTube and Vimeo were widely examined ,20,21,22Measurement studies on streaming pre-recorded 360° videos have received a growing interest ,25,26,27A wealth of studies were conducted to understand 2D video camera sensing systems that live capture and stream 2D videos. The delay analysis of Periscope, a popular live video App, showed that client buffering, chunking, and polling are the major contributors to the high latency of the HTTP-based live streaming systems . The QoEA limited number of studies were focused on 360° video camera sensing. 360° video camera sensing using commercial cameras and YouTube live streaming platforms was investigated recently. The results for streaming videos with up to 4 K resolution showed that users suffer from a high event-to-eye delay in live 360° video systems . SimilarIn contrast, the proposed study delves into each component of a canonical 360° video camera sensing system and each task of the components in order to perform the in-depth delay analysis. Our preliminary results were shown in . This paZeus prototype. We also explain the rationale of our measurement design in dissecting the latency of 360° video camera sensing.In this section, we introduce the design of our measurement study including system architecture, delay metrics, and components—a camera, camera-server transmission (CST), a server, server-client transmission (SCT), and a client. Each component must complete several computing tasks in sequence.We first discuss the canonical system architecture of 360° video camera sensing as illustrated in In particular, the 360° camera is typically composed of multiple 2D camera lenses configured in specific directions to cover the ominidirectional scene. Multiple 2D video frames of a remote scene are first captured. These frames are moved to the camera GPU and stitched into a 360° equirectangular video frame. The equirectangular frame is transferred to the camera CPU and then encoded at the CPU. The camera then uploads the frame to the server via its wireless connection to the Internet. The server parses the video data received from the camera and stores it in a video buffer in the server memory. Before the buffered video data reaches a certain amount, no client request will be accepted by the server. Once the buffered video data is sufficient, a URL to access the live streaming session will become available. Clients  can then initiate live streaming via the URL. After a connection between the client and the server is established, the server creates a metadata file about the streaming session and sends it to the client. Then the server starts streaming data from its buffer. Upon receiving data packets from the server, the client decodes, renders, and displays 360° video frames on the screen.one-time tasks for a given session between the server and the client, whereas all other tasks are pipeline tasks that must be passed through for every video frame. We aim to measure the delay of all these tasks.We now identify the delay metrics that affect viewing experience and discuss how they are determined by the time consumption of system components. The amount of time consumption is denoted by the length of a pipe in Start-up delay defines the time it takes for the first video frame to display on the client screen since a client initiates a viewing session. One main reason discouraging users from continuing and complete the video viewing is the long start-up delay . We formNote that the time consumption in the camera and camera-server transmission does not affect the start-up delay. The reason is that the live streaming of camera-recorded videos will not start until the server buffers sufficient video frames from the camera. Hence, video frames should have already been stored in the server before the server creates the streaming URL and accepts the client’s request.The time difference between the moment when an event occurs on the remote scene and the moment when the event is displayed on the user screen is the event-to-eye delay. An excessive event-to-eye delay makes users perceive the annoying mismatch and lag between what happens and what is displayed in camera sensing of live events and activities. This greatly degrades the responsiveness of real-time communications in interactive applications such as disaster command and response . The eveFrame rate represents the number of frames that can be processed and pushed through system components in the pipeline per unit time. The end-to-end frame rate of the system, We have defined the three delay metrics in terms of the time consumption of system components. It is straightforward to dissect the delay metrics using task-level time consumption because the tasks within each component are mostly serialized. Specifically, By replacing the component time in –3) by t by t3) bZeus, shown in Zeus by only using off-the-shelf hardware, publicly available SDKs, and open-source software packages such that the research community can readily reproduce the reference implementation for future research.Measuring the delay in 360° video camera sensing systems is challenging because camera sensing systems using commercial cameras and live streaming platforms are closed-source. Furthermore, no measurement tool is available to examine the black-box commercial cameras , live streaming servers , and players . Although commercial APIs are provided to allow App development, none of them can be used to obtain the delay breakdown at the task level. To enable the measurement of how the time consumption at the task level affects 360° video camera sensing and user experience, we build an end-to-end system prototype, Zeus consists of multiple 2D camera lenses—six GoPro Hero cameras [Similar to a commercial 360° camera, the 360° camera in  cameras . These 2SuperView mode. The VRWorks 360 Video SDK [cv.fisheye.ca- librate and the second-order distortion model [cudaMemcpy2D and stitched to a 360° frame via nvssVideoStitch. Finally, FFmpeg is used for encoding and streaming the 360° video. Real-Time Message Protocol (RTMP) is utilized to upload the 360° video to the server for low-delay transmission. Our upload protocol is similar to most commercial cameras, e.g., Ricoh Theta V and Samsung Gear 360.To capture wide-angle video frames from the 2D camera lenses, we configure the six GoPro in the ideo SDK  is used on model  to reducListing 1<?xml version="1.0" encoding="utf-8"?><rig coord_axes="z-up" rig_diameter_cm="9.5"><camera width="1024" height="768" layout="equatorial"><input_calib_file name="camera1.jpg"/><pose><rotation yaw_deg ="0" pitch_deg ="0.0" roll_deg="-90.0" />......</rig>nginx-http-flv-module [As for the video server, we run an open-source Nginx-1.16.1 web server. The video is streamed from the server to the client through the HTTP-FLV protocol since HTTP-FLV can penetrate firewalls and is more acceptable to web servers. While other popular live streaming protocols, e.g., HLS, may have been used, HLS protocol consumes significant time for segmenting a video stream into chunks with different video quality. As a result, the start-up delay might be higher. To allow the server to receive RTMP live video streams from the 360° camera and deliver HTTP-FLV streams to the client, we configure Nginx as the v-module .render. The spherical frame is stored at the HTML5 element <canvas> to display on webpages. The client software is embedded in a Microsoft Edge browser. We enable hardware acceleration to support the rendering and decoding tasks.To support the general applicability of client software, the client is implemented as an HTML5-based web video player using FLV.js, a flash-based module written in JavaScript. We use Three.js to fetch video streams from Flv.js and project the equirectangular frame onto the sphere format using Zeus and calculating the time interval before and after certain code functions are executed. The only exceptions are the camera-server transmission (CST) and server-client transmission (SCT), where the video stream is fragmented into packets for transmission because both the RTMP and HTTP protocols are built on top of TCP. Since the frame ID is not visible at the packet level, we cannot measure the actual transmission time of each frame directly and individually. Instead, we propose to approximate this per-frame transmission delay as the average time consumption for transmitting a video frame during the CST and SCT. For example, to derive the per-frame time consumption of CST, we first obtain the time interval between the moment when the camera starts sending the first frame via stream_frame and the moment when the server stops receiving video data in ngx_rtmp_live_av. We then divide this time interval by the number of frames transmitted to calculate the average time for transmitting a frame.We are able to measure the time consumption of most computing tasks by inserting timestamps in We present the measurement results in this section. Specifically, we show the time consumption of the tasks in 360° video camera sensing under different impact factors. We also repeat the delay measurement using multiple real-world scenarios. For each system component, we discuss how the start-up delay, event-to-eye delay, and frame rate would be affected by system parameters. We aim to identify the potential mitigation of delay issues to improve user experience.Our default measurement is conducted in a typical lab environment located in a university building. The lab hosts the camera and the client. To mimic the real-world conditions experienced by commercial 360° video systems, we deploy the server at another university campus over 800 miles away. Note that putting the camera and the client in the same building does not affect the results because the video data always flows from the camera to the server and then to the client. It is the server location that determines the transmission distance.We fix the camera on a table so that the video content generally contains desks, office supplies, and researchers. By default, the capture resolution of the 2D cameras is 720p and the stitched 360° video is encoded at 2 Mbps with the resolution ranging from 720p to 1440p (2 K). We do not choose 4 K or larger resolutions for encoding 360° videos because our previous study shows that 4 K live 360° video streaming often leads to poor system performance and user experiences . Since nWe vary the resolutions of the captured 2D videos and illustrate the video capture time in We next measure the CPU-GPU transfer time using different types of memory. Results in We measure the stitching time under different stitching algorithms. To this end, we configure different levels of stitching quality available in the VRWorks 360 Video SDK to test different algorithms. For instance, “high stitching quality” utilizes a depth-based mono stitching to improve the stitching quality and stability. We discover in After the stitched 360° frame is moved back to the CPU, the video frame in RGB format must be converted to YUV format before encoding. We show time consumption for this format conversion in We show the encoding time under different encoding bitrates and resolutions in Moreover, it is interesting to find in Our camera is able to achieve 360° camera sensing of 720p videos at 24–30 fps, which is consistent with the performance of state-of-the-art mid-end 360° cameras such as Ricoh Theta S . We findNote that the tens of milliseconds of camera time consumption will not realistically affect the event-to-eye delay defined in Equation . The typWe now evaluate the time consumption of the camera-server transmission (CST). We measure the CST time under different encoding bitrates and resolutions of the 360° video sent by the camera. Results in NetLimiter. We can see from In order to evaluate the impact of network conditions on the CST time, we fix the encoding bitrate at 2 Mbps and throttle the upload bandwidth of the CST to 2, 4, and 8 Mbps using packet by packet. As long as consecutive packets are pushed to the CST component back to back, the output frame rate of the CST component will not change regardless of the processing time of a packet. An exception might occur when the variance of the packet transmission time in the CST component (jitter) is significantly high. Fortunately, we found in It is important to observe that the time consumption in the CST component generally does not affect the frame rate. This is because the CST component continuously handles video data As for the start-up delay, the CST component is similar to the camera and does not affect it. Nevertheless, the CST time plays a pivotal role in the event-to-eye delay, especially when streaming high-quality videos in live interactive applications.Since modern WiFi (802.11ac) has sufficient bandwidth to support a reasonable CST time and stable delay jitter, today’s CST component is ready to support typical camera sensing applications. Future efforts should focus on enhancing the transmission protocols in challenging network conditions for diverse sensing environments.ngx_http_request_handler, for accepting the client request. The server then sends the initial HTTP response (excluding video data) to the client. As the HTTP response is not large, we see that a higher download bandwidth does not decrease the connection time significantly.Once the 360° video frames received from the camera are sufficient, the server becomes ready to accept a client request. To proceed, the connection task is first conducted. As shown in After the HTTP connection is established, the server generates a metadata file and transmits it to the client to convey application-related information. The metadata generation and transmission time under the download bandwidth of 2, 4, and 8 Mbps are shown in After the connection and metadata tasks, the server is ready to stream video data to the client continuously through the pipelined packetization and transmission. According to our measurement, packetizing video data at the server for video streaming to the client consumes negligible time. The received camera captured frames are packetized as soon as possible without any additional delay. The packetization process is fast itself because the Nginx server leverages pointers to record the locations of the received video data in the server buffer and then directly fetches the video data using the pointers when adding FLV headers to generate HTTP-FlV packets. No memory copy or transfer is needed for the received video data to become a packet.As the connection and metadata tasks executed in the server happen before any video frames are pushed into the system pipeline for camera sensing and streaming, they do not affect the frame rate at all. Due to the negligible packetization time, the server also does not affect the end-to-end frame rate in 360° video camera sensing.However, the large time consumption of the connection and metadata tasks causes a long start-up delay that may discourage users from staying in the video viewing after they click the video URL and initialize the session. The lengthy start-up delay in turn yields an excessive event-to-eye delay. Although the connection and metadata tasks occur only once, video data are accumulated in the server buffer during the session initialization. The video frames arriving at the server later cannot be sent until previous frames are sent out. These frames have to wait for all previous frames, leading to a long event-to-eye delay. This long event-to-eye delay would reduce the responsiveness of the camera sensing applications and degrade the viewing experience. To mitigate the negative effects of long start-up and event-to-eye delay, future efforts can be made to optimize and parallelize the computing workflow and data management in the server in order to minimize the preparation time during the server-client initialization.Similar to the CST component, we measure the SCT time for streaming a 360° frame from the server to the client. Results in Different from the CST component, the SCT component plays a critical role in the start-up delay since the first video frame has to be first streamed to the client before it can be displayed on the client screen. As for the event-to-eye delay and the frame rate, however, the impacts of the SCT are similar to the CST. In particular, the SCT time directly influences the event-to-eye delay. If the streaming time from the server to the client is high, users would have a laggy viewing experience. The impacts of the SCT on the frame rate depend on whether or not packets can be continuously streamed to the client. If the network conditions are unstable, the continuous packet reception in The 360° equirectangular frames received at the client must be first decoded into raw video frames. We show the decoding time of a 360° frame at the client in Once the viewport of the 360° frame is rendered, the viewport data are transferred to the display buffer at the client. Then, the client screen refreshes at a pre-defined frequency to display the most recently buffered data. These two steps consist of the display task. We discover that sending data to the display buffer consumes negligible time. Therefore, the display time is actually determined by the refresh frequency. In our system, the screen refreshes at 60 Hz, equivalent to a 16.67 ms display time.The frame rate of the client output is determined by the inverse of its time consumption because the frames are processed one by one in a non-parallel manner at the client. This is similar to the case of the 360° camera component. Our measurement demonstrates that even with the mandatory 360° video projection and rendering, all client tasks can be completed within a fairly short time to meet the 24–30 fps frame rate. Given the small time consumption at the client, the client’s contribution to the start-up delay and the event-to-eye delay is significantly less than the contribution of the server or SCT component. Hence, we reach a conclusion that the client has minor impacts on the user experience because of its negligible impacts on the three delay metrics. We believe today’s 360° video clients are already well-designed for high-requirement 360° video applications.Simple: The experiment configuration of this scenario is based on the Default scenario. The only difference is that all the lights in the lab are turned off. The captured content is almost dark except for a laptop screen showing a timer that is used to remind us of the experiment time.Lab Moving: Building on top of the Default scenario, the Lab Moving scenario involves a moving camera. A person holds the camera rig and walks around the lab while the camera captures content.Street Still: Instead of fixating the camera in the lab in the Default scenario, Street Still places the camera in a fixed position on the sidewalk of a street in a city center. Other settings remain the same as in the Default scenario.Street Moving: Based on the Street Still, we configure the Street Moving scenario by having a person holding the camera rig while walking on the street sidewalk.We have so far focused on the measurement result obtained from the Default scenario defined in The time consumption of CST, SCT and server components is demonstrated in As for the client, the task time almost remains the same across different scenarios and thus its impacts on the delay metrics do not change. This is because the camera motion would not significantly affect the workflow of the client.Zeus can be generalized to commercial live 360° video streaming systems and provide insight for system optimization, we perform cross validation by comparing Zeus to a system using commercial products. As it is infeasible to break down the task-level time consumption of black-boxed commercial products, we compare the component-level time consumption.To confirm that the results collected by IFrame APIs and install it on the same laptop used in Zeus.The commercial system uses a Ricoh Theta V as the camera with an Adreno 506 GPU for 360° video processing. An open-source plug-in  is instascheduleStreaming.setSchedule) and when the frame starts being transmitted (via rtmpExtend.startSteam). As for the YouTube server, we measure the time by monitoring its status change via the webpage interface used to configure the camera URL. We utilize the timestamps when a packet is received at the YouTube server and when the streaming starts at the server. Similarly, the time consumption on the YouTube client can be measured by monitoring the buffering status YT.PlayerState. In terms of the frame transmission time for the CST and SCT, we record the timestamps when the sender sends the first packet and when the receiver stops receiving data and then divide this time interval by the number of frames delivered during this period.While obtaining the time breakdown of each task is infeasible on commercial products, we can utilize high-level APIs provided by these products to measure the time consumption on each component. We measure the time consumption of a frame in the Ricoh camera by recording the timestamps when it begins to create a 360° video frame  [The comparison of the time consumption across the five system components of the two systems is shown in nt (PCC) , the Disnt (PCC) , and thent (PCC)  of the tThe results in Zeus component-wise measurement results with the commercial 360° video system imply that our results with Zeus are representative of commercial live 360° video streaming systems. Our insight can thus be generalized to minimize the adverse effects on user experience caused by different delay metrics in today’s commercial systems.The strong correlation and similarity of the Zeus server. We confirm this effect by running a sensitivity study, where we vary the number of clients sending requests to the Zeus server. The results in Zeus server time does significantly increase as the number of clients increases. Second, we see a longer time at the YouTube client. By scrutinizing the workflow, we find that this is attributed to its larger player buffer (∼1500 ms) compared to Zeus (∼40 ms).In addition to correlation, we observe two minor differences between the two systems. First, it can be seen that the YouTube server consumes more time. This may be because it handles a larger number of clients than the Zeus has been developed to measure this relationship and examine the task-level time consumption in various usage scenarios. We finally compare Zeus with commercial systems and validate that our measurement results are representative.We perform the first in-depth study of delay across the computing tasks in 360° video camera sensing in this paper. We have characterized the three important delay metrics by the time consumption breakdown across the tasks. A prototype Our findings provide critical insight for improving 360° video camera sensing. First, the bottleneck of a higher frame rate is the 360° camera. Although the space for optimizing the stitching is limited, enhancing the encoding and CPU-GPU transfer may elevate the frame rate to the next level. Second, the server plays a decisive role in meeting the requirement of start-up delay and event-to-eye delay. The existing sever workflow can be optimized to reduce the server time.Zeus in terms of camera frame rate and server delay, it is important to compare Zeus with emerging commercial systems that support 360° video live streaming through advanced cameras, such as Ricoh Theta Z1. A full scale research including extensive experiments are needed to confirm the applicability of the findings in this paper in the context of challenging 4 K 360° video systems. Moreover, since user experience is also determined by the quality of the received 360° videos, it is critical to understand how the delayed and missed packet delivery would affect the distortion and quality of the video frames viewed by the users. A widely-accepted system should provide users with both acceptable latency discussed in this paper as well as satisfactory video quality dictated by 360° image assessment metrics [In light of these observations, future work should focus on algorithm design in the camera to improve frame rate as well as in the server to shorten the delays and to support multiple clients. After optimizing  metrics ,51,52.


Clinical methods examining oxygenation parameters in humans have been used in many different care settings, but concerns have been raised regarding their clinical utility when assessing people with darker skin pigmentation. While saturation values can be crucial in emergency medicine, they are equally valuable in assessing disease mechanisms and monitoring change in disease progression. Retinal pigmentation varies across individuals and hence, can impact on retinal oxygen parameters. The objective of this study was to quantify and eliminate the impact of retinal pigmentation on retinal vessel oxygen saturation parameters measured in the superficial retinal arterioles and venules.105 healthy individuals of varying skin colour, iris colour and heritage were included. Following a full eye exam to exclude any ocular abnormality, all participants underwent intraocular pressure, systemic blood pressure measurements and dilated dual wavelength retinal photography. Rotation matrices were employed to minimise the dependency of retinal pigmentation on arterial and venous oxygen saturation measurements determined in a concentric measurement annulus.y = 0.34 × x + 38.598), whereas arterial saturation followed a polynomial pattern (y = 0.0089 × x2 + 0.7499 × x + 85.073). Both arterial and venous saturation values were corrected using local fundus pigmentation. Pre-correction retinal arterial and venous oxygen saturation were 89.0% (±13.1) and 43.7% (±11.5), respectively, and post- correction values were 94.8% (±8.7) for arteries and 56.3% (±7.0) veins.Retinal oxygen saturation in venules showed a linear correlation with retinal pigmentation (When assessing multi-ethnic cohorts, it is important to consider the impact of pigmentation on imaging parameters and to account for it prior to clinical interpretation. Dual-wavelength retinal vessel oximetry has become a promising ocular imaging modality to assess and study the mechanisms involved in ocular vascular conditions such as glaucoma –3, diabeThe measurement principle is based on a modification of the Beer–Lambert Law (see Eq. ), wherebEquation : The Beein) and relate this to the optical density obtained from neighbouring tissue (Iout) [As it is not possible to directly measure the light being absorbed and transmitted through the blood vessel, a modified version of the Beer–Lambert law allows an approximation to determine the oxygen content in retinal vessels by measuring the optical density of the retinal blood vessels (Ie (Iout) . EquatioEquation : AlgoritEquation : AlgoritOS is 0.0023/%SO2, ODRa,100 is 0.01357 and VD is the vessel diameter.Additionally, a, b, c and d are different for arteries and veins, and refer to 109 μm, 0.0667% SO2/μm, 0.265 and 15.149% SO2 for arteries and 125 μm, 0.2626% SO2/μm, 0.272 and 51.055% SO2 for veins [The experimentally derived constants or veins .This method is not without limitations since it does not measure light leaving the blood vessel directly. The use of reflectivity rather than absorption is influenced by media opacites –20, reflMore importantly, saturation values may also vary depending on retinal pigmentation , 21. BeaFollowing favourable opinion from the Aston University Ethics Committee  we recruited 105 otherwise healthy individuals (41 men and 64 women) of different ethnic backgrounds and varying iris colour (see Table Exclusion criteria for the study were intraocular pressure (IOP) > 24 mmHg, hypertension (systolic blood pressure (SBP) >140 mmHg and diastolic blood pressure (DBP) > 90 mmHg), any systemic or ocular vascular condition, previous refractive surgery, and previous ocular trauma. All participants were advised to abstain from caffeinated beverages for a minimum of 4 h prior to their scheduled appointment. After written informed consent was obtained, we measured axial length  and IOP of all participants using a rebound tonometer . Systemic blood pressure (BP) was assessed by calculating the mean of three measurements using a digital sphygmomanometer  in a sitting position after resting for 15–20 min in a temperature-controlled room at 22 degree C. Iris colour was classified according to the iris colour scheme published by Franssen et al. . Figure + fundus camera with a dual bandpass filter inserted in the illumination pathway  [One unselected eye of each participant was dilated with one drop of tropicamide . Once pupils were fully dilated, retinal images were captured using the IMEDOS Oxygen module consisting of a Zeiss F450Germany) . The sysGermany) . To ensuFor image analysis the multi-measurement facility of the VesselMap software was used . The software allows for identical vessel segments being measured across multiple images for each participant, three images were analysed by including all vessels cursing through the measurement annulus around the ONH  using the equationFollowing normalisation of global retinal pigment  was plotted against arterial vessel saturation and against venous oxygen saturations to identify the nature of the relationships. To calculate RP independent saturation values, a rotation matrix (see Eq. ) was appEquation : (a) refi% and SO2 values was of linear nature (y = 0.34 × x + 38.598). The data was rotated clockwise around the venous SO2 value y = 54% and x = 43.1 for RPi%, which corresponds to the apex of the BFC for arterial SO2 data. The rotation angle (Rt) for venous data was calculated as tan(Rt) which equates to the slope of the trendline and was 0.34.For veins, the relationship of RP2 analysis (Microsoft Excel) was used as measure of pigmentation dependency of SO2 values. A R2 of less than 0.14 was considered a ‘small’ dependency and a R2 greater or equal than 0.59 was considered ‘large’ dependency. Sample size was estimated for the assumption of a linear dependency and one independent variable according to the equation N = 104 + p, with p being the number of independent variables [All SO2 data were tested for normal distribution prior to further analyses by using the Kolmogorov–Smirnov test. IBM SPSS version 28.0.1.0 (142) was used for normal distribution testing. Stepwise forward regression analysis was conducted to explore if ocular oxygen parameters were associated with ocular biometry and refraction. Data transformation to achieve pigmentation independent SO2 values for arteries and veins was carried out using Microsoft Excel. Rariables .Demographic data of the sample can be found in Table 2 for the correlation between arterial and venous SO2 with RP were 0.613 and 0.588, respectively. After data rotation R2 were 0.145 and 0.00 for arterial and venous SO2 vs RP relationships, respectively.Retinal saturation values plotted against retinal pigmentation values of the surrounding tissue followed a polynomial pattern in retinal arteries (red data points), whereas the venular saturation (blue data points) decreased linearly with increasing levels of pigmentation see Fig. . Before The mean SO2 results of the sample before data rotation were 89.0% (±13.1) and 43.7% (±11.5) for arteries and veins, respectively. Following correction for retinal pigmentation, retinal arterial and venous SO2 were 94.8% (±8.7) and 56.3% (±7.0).To date, only a handful of studies explored retinal oxygen saturation parameters, and whether there is an impact due to different retinal pigmentation , 21, 33.Uncorrected oxygen saturation values are affected by retinal pigmentation as dual wavelength retinal photography relies on reflectance values as described earlier. This paper shows that both, retinal arteriolar and venular SO2 values vary with iris and fundus pigmentation see Fig. . HoweverOpposing to the study by Hammer and colleagues , which fn = 61) and did not cover the full range of the Franssen scores of iris pigmentation [A study by Jani et al. used the same iris classification system as this study, but its sample size was smaller  in studies with multi-ethnic populations.Dual wavelength retinal oximetry results may depend upon retinal pigmentation.The study highlights the importance of considering retinal pigmentation in oximetry as results are influenced by retinal pigmentation values.Supplemental Table