Humans observe a wide range of actions in their surroundings. How is the visual cortex organized to process this diverse input? Using functional neuroimaging, we measured brain responses while participants viewed short videos of everyday actions, then probed the structure in these responses using voxel-wise encoding modeling. Responses are well fit by feature spaces that capture the body parts involved in an action and the action’s targets . Clustering analyses reveal five large-scale networks that summarize the voxel tuning: one related to social aspects of an action, and four related to the scale of the interaction envelope, ranging from fine-scale manipulations directed at objects, to large-scale whole-body movements directed at distant locations. We propose that these networks reveal the major representational joints in how actions are processed by visual regions of the brain. How is action perception organized in the brain? Here, the authors report evidence for five networks tuned to actions’ social content and the scale of their effect on the world and propose that sociality and interaction envelope are organizing dimensions of visual action representation. Much previous work on the nature of action representation in the brain has focused on the end-stages of this process; for example, attempting to localize the more abstract and conceptual aspects of action representation8. However, recent research has begun to examine action-related processing from a more perceptual angle, asking how regions involved in high-level vision are organized to support action observation9. The present work follows this latter approach, leveraging an encoding analytical framework to characterize action responses in the visual system12.We witness a multitude of actions in daily life: running, jumping, cooking, cleaning, writing, and painting, to name just a few. How does the brain make sense of this diverse input? The process begins with basic perception, forming lines and shapes into bodies in motion with identities and kinematic properties, and ultimately derives rich representations about emotional states, social interactions, and predictions about what will happen next13? Many prior studies have used the tight link between actions and verbs as a guide, and have measured responses to written verbs or used verbs to guide the selection of experimental stimuli15. However, common verbs do not always have a clear corresponding visual action , and this approach may build in theoretical assumptions that actions that look very different but are described by the same verb share a neural representation . To approach action representation from a visual-perceptual level, we instead sampled our action stimuli based on common human experiences, using the American Time Use Survey as a guide17 (see “Methods”). These actions span a wide range of activities, such as cooking, traveling, exercising, and recreating, that a large set of Americans reported engaging in on a daily basis.To employ this framework, we considered three theoretical questions and their methodological implications. The first relates to the nature of the domain in question: what do we mean by “actions” and how should we sample this domain representatively20, which help to isolate actions from their backgrounds, but constrain researchers’ ability to discover joints in the broader domain of action perception. Others have taken a highly unconstrained approach, using rich, complex stimuli such as feature-length films22. This approach comes much closer to reflecting our daily perceptual experience with actions; however, too much complexity can make the resulting data challenging to wrangle into interpretable results. Thus, in the present work, we took an approach that sits in between these extremes: we selected a targeted and diverse subset of everyday actions, depicted using complex, heterogenous videos of a short duration.A related consideration is how best to depict these actions in order to probe their underlying neural representations at a meaningful level of abstraction. To date, some researchers have used a small number of highly controlled action video stimuli25. However, the perceptual processing mechanisms that operate over objects, bodies, and motion likely also play a role in representing ecological actions. Therefore, it is critical to consider more widespread action responses across broad swathes of occipitotemporal and parietal cortices16. Given this, we developed a novel method to identify voxels that reliably differentiate among different actions26, rather than constraining our analyses to specific regions of interest emphasized in the literature.The second major consideration concerns which brain regions to target. Prior work has revealed that watching other people’s actions engages a broad network of regions with nodes in all major lobes of the brain, known as the “action observation network”29, while inferior parietal sulcus contains information about actors’ goals30. Others have argued that an action’s sociality (whether or not an action is directed at a person) and transitivity (whether or not it is directed at an object) organize action processing in the lateral temporal cortex9. Drawing from this prior work, here we examined patterns of tuning to the different body parts that are involved in performing an action and what the action is directed at .The final major consideration concerns the nature of the tuning in these regions. That is, what is it about an action that makes a given voxel respond more to that action than to others? For example, actions differ in the “means” by which they are performed and “ends” or goals that they are accomplishing. Paralleling this division, regions of the lateral temporal cortex are tuned to body parts and posturesWith these considerations in mind, the goal of this study was to understand and characterize what properties of actions best predict corresponding neural responses, allowing for the possibility that different regions of cortex are sensitive to different properties. To preview, we find that the effectors used to perform an action and what an action is targeted at successfully predict responses to action videos in much of the occipito-temporal and parietal cortices. Further, an analysis of the voxel-wise tunings to these features reveals a large-scale organization of five networks that span the ventral and dorsal visual streams. The tuning of these networks is related to actions’ spatial scale of interaction (“interaction envelope”) and relevance to agents . We argue that these networks reflect meaningful divisions in how actions—and, more broadly, visual inputs across domains—are processed by the brain.31. These actions span broad categories such as personal care, eating and drinking, socializing, and athletics (see https://osf.io/5qk8j/ for a figure summarizing these videos). Two short (2.5 s) videos were selected to depict each action, then divided into two sets. Each video depicts a sequence of movements  rather than an isolated movement , and thus may alternatively be thought of as depicting “activities” rather than “actions.” However, “activity” can also connote something done over a much longer timescale and with greater variation in perceptual properties, such as “cooking dinner.” Thus, for clarity we refer to the stimuli as “action” videos. Participants (N = 13) watched these action videos while undergoing functional magnetic resonance imaging (fMRI) in a condition-rich design that enabled us to extract whole-brain neural responses to each video . Through this procedure, a voxel-reliability cut-off of r ≥ 0.30 was selected, yielding an average item-pattern reliability of r = 0.88 in the group data .The primary question of this study is whether these hypothesized feature spaces characterize neural responses to actions well, and if so where. To answer this question, we employed a voxel-wise encoding-model approachrCV-set1 = 0.41 and rCV-set2 = 0.42; max rCV-set1 = 0.76. rCV-set2 = 0.71; see Supplementary Fig. Figure 32 and Vul et al.33 for related analysis approaches). Through this method, we found evidence for five networks . Importantly, this method does not require that voxels are grouped into contiguous clusters, making it possible to discover both contiguous regions and networks of non-contiguous regions that have similar tuning functions. Further, this method does not presuppose any particular combinations of feature tunings ahead of time, allowing natural patterns to be revealed directly from the data  or the actor themselves (laughing), and those directed at objects and space. In fact, this division emerges early on: Network 1 separates out from the rest when voxels are grouped into just two networks  is primarily right-lateralized, covering regions along the fusiform gyrus and extending between the occipital face area (OFA) and superior temporal sulcus (STS). This network is tuned to face features and not hands and is directed at people (others and the actor) but not objects. It is near STS regions typically engaged by social processing40. Network 3 (dark green) contains voxels that stretch extensively along inferior IPS, into the transverse occipital sulcus (TOS), with a satellite node in the parahippocampal cortex (PHC). Inspecting the feature weights associated with this network also reveals strong tuning to both objects and the near space, but with more arm (and not finger) involvement.The remaining four networks are tuned to non-social aspects of action. The dorsal stream responses divided into two prominent networks. Network 2 (blue) contains voxels that stretch extensively along superior IPS, as well as a “satellite” node in lateral OTC. Inspecting the feature weights associated with this network indicates that these voxels respond most strongly to videos that involve finger and hand movements, and that are directed at objects in the near space . This finding resonates with a previously established tool networkNetwork 4 (purple) is restricted to the ventral stream, including bilateral regions in the vicinity of extrastriate body area (EBA). Regions in this network are tuned to hands, arms, the torso, and near space.43. However, compared with Network 3, Network 5 is tuned more strongly to actions directed at far space, the legs, and the whole body and less strongly to actions directed at objects. These differences in tuning echo prior work showing that at least one scene-preferring region—PPA—contains sub-regions for object and space processing40.Finally, Network 5 (light green) has nodes along PHC, TOS, and the medial surface of the cortex in both the parietal and retrosplenial regions. This final network effectively runs parallel to Network 3 (dark green): both networks anatomically resemble scene-preferring regions, such as the parahippocampal place area, occipital place area, and retrosplenial cortexInspecting these last four non-social networks reveals a possible overarching organization, where each network has a preference for a different “interaction envelope”, or the scale of space at which that agent-object interactions take place in the world. The four interaction envelopes move from small-scale, precise movements involving hands and objects (knitting), to less precise movements also involving hands and objects (loading a washing machine), to intermediate-scale movements involving the upper body and near spaces (golfing), to large-scale movements that engage the whole body and require more space (playing soccer).To examine the robustness of these findings in video set 1, we also conducted the same analysis on voxel tuning weights fit using video set 2. This five-network solution was quite consistent across the two video sets . We found that the spatial extent of motion in a video does track the size of the interaction envelope for small and intermediate interaction envelopes, but not the largest . Figure Third, are these neural responses simply reflecting what’s visible in each video? The body parts and action targets that are involved in an action are undeniably related to what is visible in its video. However, we found that ratings of the features’ visibility and involvement were not perfectly correlated  encompasses regions which are traditionally thought to be specialized for body parts, whole bodies, and faces49, and is right-lateralized, consistent with prior results51. Our results also converge with previous findings that responses in OTC are organized by animacy and object size54. In general, big objects are processed in regions tuned to larger-scale actions, small objects are processing in regions tuned to smaller-scale actions, and animals are processed in regions tuned to agents. Additionally, the network related to large-scale interactions such as locomotion (Network 5) encompasses well-known scene-selective regions near occipital place area, retrosplenial cortex, and along the parahippocampal place area. Thus, our results dovetail well with existing characterizations of this cortical territory. However, note these regions are typically identified by contrasting responses to isolated bodies, objects, and scenes, all of which are present in our videos. Therefore, our results likely converge with prior findings not simply because of the mere presence of these visual categories but because scenes and bodies are more relevant to some actions than others.The five sub-networks found in our data converge with several known networks of the ventral and dorsal stream, including those that display preferences for bodies, faces, objects, and scenes43. Relatedly, the sub-networks tuned to object targets within a more focal interaction envelope (Networks 2 and 3) are bilateral in this data set, which is consistent with some tool-network studies55, but not others56.In some cases, our results diverge from prior work. For example, the sub-network related to large-scale actions and far spaces (Network 5), is primarily right-lateralized in both video sets. However, most research on scene networks shows relatively clear bilateral networksPerhaps the most surprising result was that ventral stream responses were predicted best by whether body parts and action targets were involved in the action, while dorsal stream responses were predicted best by their visibility in the video. While it is clear that viewing actions drives both the ventral and dorsal visual streams, it is also clear that more work is needed to understand how the streams relate to one another during action observation. One potential avenue for clarifying this relationship is to characterize the relative contributions of what is visible and what participants fixate on. It is possible that participants fixate on the features that are more functionally relevant to an action. If this is true, fixated features may be prioritized in ventral stream, while the dorsal stream may represent the full action scene to guide subsequent fixations. Future eye-tracking studies are needed to test these hypotheses.22. However, our approaches differ in the granularity at which we predict voxel responses and subsequently infer voxel tuning properties. In Huth et al.11, a voxel’s response could be fit by putting weights on over 1000 predictors, including verbs like “cooking”, “talking”, and “crawl”, as well as nouns like “tortoise” and “vascular plant.” It is possible to map some of these specific features onto our more-general ones—for example, a voxel in their data set with high weights on “knitting” and “writing” might be fit in our data set by high weights on the hand, fingers, and object-target features. However, one potential advantage of characterizing the brain’s responses at the level of body parts and targets is that the features are more generative: it is simple to map any new action into this low-dimensional feature space, and this level of representation may therefore be more appropriate for characterizing the response tuning of mid-to-high-level visual cortex.In addition to converging with prior work on other categories of visual input, our methods build on the use of encoding models to predict responses to rich videos9 proposal that sociality and transitivity organize action representations. However, we did not find evidence for their anatomical proposal that these features are reflected in a ventral-dorsal organization across the lateral OTC  with normal or corrected-to-normal vision were recruited through the Department of Psychology at Harvard University and participated in a 2-h neuroimaging experiment. In addition, 802 participants completed behavioral rating experiments conducted online. All subjects gave informed consent according to procedures approved by the Harvard University Internal Review Board.61, and the University of Central Florida’s Action Recognition Data Set62. These were divided into two sets of 60 videos each, so that each set contained one exemplar depicting each of the 60 actions. All videos were cropped to a 512-by-512 px frame centered on the action, with no visible logos or borders, and stripped of sound. The 60 actions were selected based on the American Time Use Survey corpus31, which records the activities that Americans perform on a regular basis, across a range of general categories . These categories were used to ensure that we sampled a wide range of everyday visual experience but were not used in the analyses. Key frames from both sets of videos are available for download from the Open Science Framework.One hundred twenty 2.5-s videos of 60 everyday actions were collected from YouTube, Vine, the Human Movement DatabaseParticipants viewed 120 videos of everyday actions in a condition-rich design while undergoing functional neuroimaging . To ensuN = 182), raters indicated the body parts that were engaged by the action in the video by selecting them from a clickable map of the human body: when a body part was selected, it was highlighted in red , raters indicated the body parts that were visible at any point in the video using the same method. In Experiment 3 (N = 180), raters indicated what each action was directed at by answering five yes-or-no questions: “is this action directed at an object or set of objects?”; “is this action directed at another person (not the actor)?”; is this action directed at the actor themselves?”; “are the surfaces and space within the actor’s reach important for the action being performed?”; “is a location beyond the actor’s reach important for the action being performed?” , raters responded to similar questions about the targets that were visible in the videos .To collect action feature ratings, four behavioral experiments were conducted on Amazon Mechanical Turk. In all experiments, 9–12 raters viewed each video, named the action depicted, and answered experiment-specific follow-up questions. In Experiment 1 , yielding a 1,920-dimensional feature vector for each frame. These gist features were extracted for each video frame and then submitted to a PCA to obtain features that describe the global shape of the scene. Following Oliva and Torralba44, the first 20 PCs were then averaged across frames in a given video, resulting in a 120-by-20 gist feature matrix. Together, these 20 PCs accounted for nearly 100% of the variance in the videos.For comparison with the body parts and action target features, the “gist” modelr-maps. Within-set reliable voxels had better coverage of early visual cortex and were only used to compare the gist model to the body-part-action-target model  across a range of candidate cutoffs. Then, we selected the cutoff based on where the multi-voxel pattern reliability begins to plateau for all videos. This method takes advantage of the fact that after a certain point, using a stricter cutoff restricts coverage without significantly increasing the data’s reliability. Using this approach, we determined that any voxel with an average reliability of 0.3 or higher was a reasonable cutoff for inclusion in the feature modeling analysis because it maximized reliability without sacrificing too much coverage  using L2 (“ridge”) regularized regression. Models were fit separately for the two video sets. The regularization coefficient (λ) in each voxel was selected from 100 possible values . The final λ was the value that minimized the mean-squared error of the fit in a 10-fold cross-validation procedure, using the data from the other video set. To ensure that our models were not over-fit, we estimated their ability to predict out of sample using a cross-sets cross-validation procedure. This was done by training the model in every voxel, using the data from all 60 videos in one video set. We then calculated the predicted response magnitude for the second video set (beta weights from the training model * feature vector for the held-out video set). Finally, the predicted and actual data for the held-out actions were correlated to produce a single cross-validated r-value (rCV) for each voxel. All models were fit using responses from the group data  by both the models. Next, we used MATLAB’s implementation of the k-means algorithm with the correlation distance metric to cluster voxels by feature weight profile similarity . The correlation metric was chosen in order to group voxels together that have similar relative weightings across the features . For any clustering solution, the cluster centroid weight profiles reflect the average normalized profile for all voxels included in the cluster.We used k-means clustering to group voxels based on their feature weight profiles. This analysis groups voxels based on the similarity of the weights that the voxel-wise encoding model assigned to the 12 body-part and action-target features. These feature weights were measured by fitting the model on the complete dataset for each video set, and the clustering analysis was conducted separately in both Set 1 and Set 2, enabling a internal replication of the main result. In order to facilitate a comparison between the sets, we ensured these clustering analyses were conducted over the same set of voxels—thus, we only included voxels that were reliable across both video sets and were predicted well (k), we iteratively performed k-means on the data from video set 1, varying the possible k value from 2 to 20  and cluster center similarity (how similar the cluster centers are to one another on average). The logic for the latter measure is that if two clusters have very similar centroids, there is very little we can do to interpret their differences; thus, by considering solutions with lower cluster center similarity we are better equipped to interpret divisions within this feature space. In addition, we visualized the solutions at k = 2, 3, and 4 . Finally, analogous cluster centroids were correlated between sets  between the solutions for video sets 1 and 2. To do so, we created a voxels x voxels matrix for both video sets, with values equal to 1 if the voxels were assigned to the same cluster and 0 if they were assigned to different clusters. Hit rate was calculated as the percent of voxel-voxel pairs that were assigned to the same cluster in set 1 that were also assigned to the same cluster in set 2. False alarm rate was calculated as the percent of voxel-voxel pairs that were not assigned to the same cluster in set 1 but were assigned to the same cluster in set 2. Sensitivity (d′) was subsequently calculated as z(Hit)-z(FA). D′ was also calculated between the shuffled data for sets 1 and 2, producing a baseline value close to zero (mean across To compare how the voxels cluster independently of their feature tunings, we followed a similar procedure to group voxels by their response profiles. Here, instead of using feature weights, we performed the clustering over voxels’ response magnitudes to each action video. All other procedures were the same as those used for feature-based clustering.rCV and colored it so that the hue corresponds to the winning model and the saturation corresponds to the size of the difference between that value and the alternative model’s performance (e.g., a voxel where the gist model did markedly better than the body-part-action-target model is colored intensely yellow; Fig. r = 0.01 for video set 1 r = 0.05 for video set 2.To compare performance based on the low-level gist model to the higher-level body part and action target models across visual cortex, we calculated a two-way preference map. First, both the gist model and the body-part-action-target model were used to predict responses in within-sets reliable voxels. Within-sets reliable voxels were used to increase our coverage in the early visual cortex, where we anticipated the gist model would do best. Then, in each voxel, we found the model with the maximum To compare models based on the functional involvement and visibility of body parts and action targets, we calculated a two-way preference map. As described above, each voxel was colored according to the model that predicted its responses the best (purple: feature involvement, blue: feature visibility; Fig. Further information on research design is available in the Supplementary InformationPeer Review FileReporting Summary


Here, a three year-old boy with a novel variant in this gene and had difficulties with enteral hypomagnesemia treatment is presented. He had recurrent seizures since two years of age and was diagnosed with epilepsy and treated with multiple antiepileptic drugs. Subsequently, he was diagnosed with rickets due to severe hypocalcemia at another center. The patient was hypotonic and neurodevelopmentally poor. The most prominent laboratory finding was of hypomagnesemia with secondary hypocalcemia. The genetic analysis revealed a novel variant in the TRPM6 gene. After parental treatment of intravenous magnesium (Mg2+) sulfate and calcium, the treatment was switched to enteral Mg2+ medications, due to persistent hypomagnesemia and the gastrointestinal side-effects, different oral preparations were used. The patient was stable on an oral maintenance dose of Mg2+ oxide with borderline blood Mg2+ levels and resolution of hypocalcemia. Hypomagnesemia is one of the causes of hypocalcemia. Enteral replacement is the key treatment but the treatment should be individualized for each patient. Normalization of hypomagnesemia is not always easy and should not be the aim of the treatment.Hypomagnesemia is a rare cause of seizures in childhood but should be kept in mind in recurrent and intractable seizures and hypocalcemia in communities where consanguineous marriages are common. Familial hypomagnesemia with secondary hypocalcemia is a rare genetic cause of hypomagnesemia, due to variants in the  Hypomagnesemia is one of the causes of hypocalcemia. Enteral replacement is the key treatment but the treatment should be individualized for each patient. Normalization of hypomagnesemia is not always easy and should not be the aim of the treatment.Genetic analysis revealed a novel frame shift variant in the transient receptor potential melastatin 6 gene. Magnesium levels varied during treatment with different preparations so in these patients treatment should be individualized for optimal replacement.2+) dependent hypocalcemia  gene, cause the most profound genetic hypomagnesemia  is a rare autosomal recessive disorder that presents in infancy with neurological symptoms of magnesium , calcium 4.7 mg/dL, phosphorus 6.2 mg/dL, alkaline phosphatase 222 U/L, parathyroid hormone 6 pg/mL , 25-OH vitamin D3 60.1 ng/mL, sodium 139 mEq/L, potassium 4.24 mEq/L, albumin 3.2 g/dL, and uric acid 5.3 mg/dL. The urine fractional excretion of Mg2+ was 12%  with normal urine calcium/creatinine ratio. Laboratory examinations including acute phase reactants, serum glucose, and albumin levels, and liver and renal function tests were normal. The renal ultrasound did not show any medullary nephrocalcinosis. Electroencephalogram showed slow background activity without any epileptiform discharges and magnetic resonance imaging of brain showed mild diffuse cerebral and cerebellar atrophy . His prenatal and natal history was uneventful. His family history was unremarkable. His family history was negative for epilepsy and neurological abnormalities, as well as any known renal, thyroid, or parathyroid disease. Physical examination revealed his growth parameters were within normal limits . He did not show any dysmorphic or neurocutaneous features. He was conscious and had a speech delay. Bilateral horizontal nystagmus was prominent. He was hypotonic with normal reflexes. The rest of the physical examination was normal. Laboratory data included a serum Mg atrophy .2+ homeostasis. The genetic analysis of the patient revealed a novel, homozygous variant in the TRPM6 gene (NM017662.4: c.5473_5474insGCTTC (p.H18225Rfs*18) (p.His1825Argfs*18). This is a frameshift variant and based on American College of Medical Genetics and Genomics criteria this variant was classified as pathogenic. This is a null variant and is predicted to cause severe loss of gene function. TRPM6 gene sequence analysis was performed using MiSeq next generation sequencing platform, an Food and Drug Administration approved diagnostic system . Sequences were aligned to the hg 19 genome within MiSeq Reporter software (Illumina Inc.). Visualization of the data was performed with IGV 2.3 (Broad Institute-www.software.broadinstitute.org) software  but abdominal pain and diarrhea was significant. In addition, on this oral therapy, serum Mg2+ decreased to 0.2 mmol/L and convulsions re-occurred without hypocalcemia. The treatment was replaced with oral Mg2+ citrate and soon after switched again to Mg2+ carbonate because of persistent hypomagnesemia and gastrointestinal side-effects. Finally, Mg2+ oxide sachets were started and blood Mg2+ reached 0.78 mmol/L with high dose Mg2+ oxide (3x2 sachets ~ 150 mg/kg/d)  . AntiepiTRPM6 gene. The patient’s treatment was individually tailored according to blood Mg2+ levels and to minimize the side effects of a range of Mg2+-containing medications. The patient benefitted from Mg2+ replacement for neurodevelopmental improvement and showed satisfactory progress.We presented a case of HSH due to a novel variant in the 2+ permeable ion channel encoded for by TRPM6 gene on chromosome 9q22 . Thus, the type of oral Mg2+ preparation is important, since some preparations have been reported to have a better bioavailability than others . In a recent report, Mg2+ chloride or Mg2+ glycerophosphate was suggested rather than Mg2+ oxide or Mg2+ sulfate for oral Mg2+ supplementation  were shown to be effective in increasing Mg2+ levels in different tissues like brain and muscle  even with significant increases in supplemented dose (2+ replacement and preparation selection should be guided by follow-up in each individual patient and may also be affected by differences in the specific variant inherited.Milder clinical phenotypes may be due to different variants. However, due to the rarity of the condition, no definitive genotype–phenotype correlation has been established at present. The most commonly reported symptoms on admission were recurrent and intractable myoclonic or generalized tonic-clonic seizures . Mg2+ trtrations ,2,3,4. Tted dose . There iTRPM6 variant is a cause of profound hypomagnesemia with secondary hypocalcemia. With appropriate treatment, the seizures can be controlled and neurocognitive development of the patients can be improved. Rapid diagnosis and treatment of this rare disorder can significantly improve the quality of life of affected individuals.2+ should be included in the work-up, especially during a seizure episode. This is especially true in those communities where consanguineous marriages are common. Enteral or parenteral Mg2+ replacement is key in managing this condition and the aim should be to normalize serum calcium and control the symptoms. The treatment of hypomagnesemia is not always easy and may depend on the dose and the content of the medication. Individualized therapy and management should be tailored to each patient.Hypomagnesemia is one of the causes of hypocalcemia. A diagnosis of primary HSH should be considered in all pediatric patients presenting with generalized seizures or tetany. Measurement of serum Mg


Human adaptation to extreme environments has been explored for over a century to understand human psychology, integrated physiology, comparative pathologies, and exploratory potential. It has been demonstrated that these environments can provide multiple external stimuli and stressors, which are sufficient to disrupt internal homeostasis and induce adaptation processes. Multiday hyperbaric and/or saturated (HBS) environments represent the most understudied of environmental extremes due to inherent experimental, analytical, technical, temporal, and safety limitations. National Aeronautic Space Agency (NASA) Extreme Environment Mission Operation (NEEMO) is a space-flight analog mission conducted within Florida International University’s Aquarius Undersea Research Laboratory (AURL), the only existing operational and habitable undersea saturated environment. To investigate human objective and subjective adaptations to multiday HBS, we evaluated aquanauts living at saturation for 9–10 days via NASA NEEMO 22 and 23, across psychologic, cardiac, respiratory, autonomic, thermic, hemodynamic, sleep, and body composition parameters. We found that aquanauts exposed to saturation over 9–10 days experienced intrapersonal physical and mental burden, sustained good mood and work satisfaction, decreased heart and respiratory rates, increased parasympathetic and reduced sympathetic modulation, lower cerebral blood flow velocity, intact cerebral autoregulation and maintenance of baroreflex functionality, as well as losses in systemic bodyweight and adipose tissue. Together, these findings illustrate novel insights into human adaptation across multiple body systems in response to multiday hyperbaric saturation. Extreme environments have been explored for over a century to understand human adaptation, extrapolate insights into comparative pathology, and extend human exploration . Salient2) therapy or hyperbaric research. I)  with 21% O2 (0.21 PO2) and 78% nitrogen (0.78 PN2). Increased sea level depth (+1 ATA per 10 m depth) or forced air compression elevates PI, causing O2 and/or N2 arterial pressure to rise, a physiologic state termed “saturation.” Increased arterial PO2 and/or PN2 have been demonstrated to increase reactive N2 and O2 species (RNOS) production, modify redox balance, and impact neurosensory, pulmonary, autonomic, and cardiovascular systems (Hyperbaric and/or saturation (HBS) environments impact divers , personnel operating below sea level , and/or patients within pressurized chambers used for hyperbaric oxygen (Oarch. I) . PI is d systems . Beyond  systems . While m systems , with ev systems  or press systems  at varyiFlorida International University’s (FIU) Aquarius Undersea Research Laboratory (AURL) is a submerged 400 square feet 74,389 kg habitat 9 km offshore and is considered the world’s only existing underwater research laboratory allowing individuals to live and operate at HBS for up to 3 weeks. The National Aeronautic Space Agency (NASA) Extreme Environment Mission Operation (NEEMO) is a space-flight analog mission conducted within AURL, with similarities to space-flight missions, including mission task load and demands, communication logistics, geographical isolation, extravehicular activities (EVAs), associated risks, and others. Consequently, the NASA NEEMO AURL provides an operational relevant research environment for elucidating the multiday adaption impact of real-world HBS on free-living, highly trained operators , allowing for advanced analyses into the multiday adaptive impact of HBS on human psychology and integrated physiology, allowing potential insight into comparative medical pathologies, and/or aiding in the development of mitigative strategies or augmentics to extend human exploration .To investigate human objective and subjective adaptation to multiday HBS, we evaluated each aquanaut’s psychological, cardiac, respiratory, autonomic, thermic, peripheral and central hemodynamics, sleep, and body composition parameters while living at saturation for 9–10 days across NASA NEEMO 22 and 23. We hypothesized that aquanauts would experience psychological burden, reduced mood and work satisfaction, increased heart rate (HR), decrease respiratory rate, increased sympathetic and reduced parasympathetic modulation, elevated skin temperature, hemodynamic dysfunction, reduced sleep quality and quantity, bodyweight and fat-free mass loss, without change in adipose tissue, due to accumulated exposure to AURL HBS multi-stressors. However, we discovered that humans living at HBS for 9–10 days experienced intrapersonal physical and mental burden, sustained good mood and work satisfaction, decreased HR and respiratory rates, increased parasympathetic and reduced sympathetic modulation, lower cerebral blood flow velocity, intact cerebral autoregulation and maintenance of baroreflex functionality, and losses in systemic bodyweight and adipose tissue . These dn = 7; female, n = 4) composed of astronauts (n = 4), research scientists (n = 4), and/or professional habitat technicians (n = 3) underwent a multistep verification process including, medical and physical evaluation, confirmation of accredited national or international diving certification and experience, and mission training at NASA Johnson Space Center (JSC) to participate in NASA NEEMO 22 and 23. Upon selections, aquanauts underwent continual internet-based planning, training and familiarization along with a week-long mission training and evaluation program at JSC involving equipment, timeline, research projects, and personnel familiarization. 7 days before entering HBS, participants arrived at FIU On-Shore Mission Control Facility in Islamorada, Florida where they underwent pre-mission preparation involving training, planning, dives, etc. JSC and/or ARB training week provided an analogous physical and mental task load environment to within habitat HBS and was used as a normobaric pre-saturation control timepoint. Following 7 days of ARB training, aquanauts dove into the HBS environment (AURL) where they lived and executed full-day mission objectives for 9–10 days which included extravehicular activities, continual communication with FIU On-Shore Mission Control, research implementation, and other activities, analogizing space-flight operational demands and inherent risks. Throughout HBS, aquanauts were supported by Navy Medical Divers and additional support divers to ensure safe mission completion. Within 24 h of resurfacing, aquanauts underwent a 14–18 h desaturation protocol to lower arterial PN2. Following resurfacing, participants underwent 2-days of mission debriefing. This period was used for the normobaric post-saturation recovery timepoints. Intrapersonal physical burden, mental burden, mood, work satisfaction, cardiac function, autonomic function, sleep quality, subjective sleep, respiration, and body temperature measurements were taken twice pre-saturation (ARB Pre-Saturation Day 4&5), twice in saturation (Mission Day 4&8), and once post-saturation (Debriefing Day 1). Peripheral and cerebrovascular hemodynamics were measured once pre-saturation (JSC Pre-Saturation), three time during saturation , and once post-saturation (Debriefing Day 1). Body composition was measured 24 h prior to saturation and within 24 h of resurfacing. All crewmembers gave informed consent, as approved by the Institute for Human and Machine Cognition, Johns Hopkins University, NASA JSC, and/or European Space Agency Institutional Review Boards. Aquanaut baseline anthropometric characteristics are described in NASA NEEMO 22 and 23 missions planning began in January 2017 and January 2019, respectively. Eleven aquanauts  was pumped into AURL via high pressure air compressors from a sea level life support buoy into high pressure gas cylinders. AURL is positioned at 19.2 m sea water depth (2.92 ATA/2219 mmHg) where aquanauts lived and operated primarily at ∼13.4–14.3 m sea water depth (2.34–2.43ATA/1778–1847 mmHg) depending on oceanic tide with intermittent multi-hour EVAs at ≥ 19.2 meters depth (≥2.92 ATA/ ≥ 2219 mmHg). Throughout EVAs, aquanaut buoyancy was set to replicate Mars microgravity and they were monitored for sea water depth to ensure individuals remained within “no decompression” limits under continuous saturated diving conditions. PPgas in each environment was calculated as gas fractional concentration multiplied by total pressure. Corresponding PPO2, PPN2, and PPCO2 are indicated in 2 was recorded at 0.3–1.5% surface equivalence value throughout mission, similar to PPCO2 observed within the international space station  complied with all timepoints and were included in analyses.Aquanauts completed a multi-question intrapersonal mental burden, physical burden, mood, and work satisfaction survey to evaluate the busy/hectic, physically strenuous, mentally taxing, post-mission day fatigue, tenseness, and stiffness, headache and pain using a 1–10 Likert scale at different timepoints, always at the end of each day as previously described . Intensin = 5; female, n = 2) complied with all timepoint measurements, did not obtain errors in data collection, and were included in analyses.Aquanauts cardiovascular and autonomic function were measured throughout rest. Polar V800  Watch and H7 Chest Strap technology was utilized to gather electrocardiographic signals  across rest and activity . Brieflytrans-temporal window with a 2 MHz probe  as previously described (n = 4) and three completed the prone-to-stand protocol .Aquanauts were assessed during waking hours to determine beat-by-beat middle cerebral artery flow velocity (MCAv), systolic blood pressure (SBP), diastolic blood pressure (DBP), mean arterial pressure (MAP), and estimated HR. Transcranial Doppler measurements were obtained at the escribed . The Dopescribed , which un = 6).Aquanauts were evaluated for sleep quantity, quality, respiration and skin temperature throughout rest (sleep) using OURA which is a 15 g multisensory technology . This ten = 5; female, n = 2) complied with all timepoints and were included in analyses.Immediately before and upon completion of sleep, participants completed a modified Pittsburg Sleep Diary adapted to evaluate subjective sleep pre-, during, and post-saturation for trouble falling asleep, sleep quality, sleeplessness, pre-mission day fatigue, and alertness in extreme environmental conditions , using a2), body fat (%), fat-mass (kg), fat-free mass (kg), and bone density (g/mL) were calculated from anthropometric data  were available within 24 h post-resurfacing for body composition pre- to post-analyses. Of the nine subjects, six subjects gathered multiple muscle thickness measurements and were included in analyses .Aquanauts underwent anthropometric assessment to determine both systemic and region-specific alterations in tissue composition. Subject daily routines were pre-assigned throughout. Anthropometric data were gathered across the same environment and routine at similar daily timepoints (10:00–14:00) which illustrated overlapping pre-saturation and post-saturation subject timepoints to control for lifestyle-induced variability. Anatomical landmarks were indicated with a marking pen to ensure measurements were gathered at the same anatomical sites. Weight (±0.1 kg) and height (nearest cm) were gathered using a digital scale and stadiometer, respectively. The dominant extremity was used for gathering remaining anthropometric data. Waist circumference was measured between the right and left iliac crest using standard measuring tape. Leg circumference was measured halfway between the greater trochanter and patella. Arm circumference was gathered between the lateral humerus greater tuberosity and olecranon. Three-point sex-specific caliper skinfolds . Repeated Measures One-Way ANOVA followed by Tukey’s post hoc was used for >3 comparisons and Fisher LSD post hoc for ≤3 comparisons to control for family wise error. Pearson correlation analyses were conducted where overlapping timepoints and measurements were reported between OURA and Polar devices (Mean HR and RMSSD). Significance is indicated where p < 0.05. Data are reported as mean ± SD.GraphPad Prism 8 software was used for all statistical analyses. Within subject parameters were averaged within timepoint . The Shapiro–Wilk test was performed to determine normal distribution. Logarithmic transformation was performed for all parameters determined to not be normally distributed. Paired p = 0.026). Physical strenuousness was mild at pre-saturation and saturation but reduced significantly to none post-saturation compared to pre-saturation (p = 0.012) and saturation (p = 0.004). Mental taxation was mild at pre-saturation and saturation, and very mild post-saturation, but was not significantly different across timepoints (p = 0.005). No stiffness, headaches, and pain were indicated across timepoints , while trending lower from during saturation compared to post-saturation (p = 0.089). Lowest resting HR was trending lower during saturation compared to pre-saturation (p = 0.069), while significantly lower during saturation compared to post-saturation (p = 0.008). Respiratory rate was significantly lower in saturation compared to pre- (p = 0.012) and post-saturation (p = 0.039). R–R Interval, RMSSD, Total Power, VLF, LF, HF, Poincaré Perpendicular Standard Deviation (SD1), Poincaré Parallel Standard Deviation (SD2), SD1/SD2, and PNS index were significantly higher during saturation compared to either pre- or post-saturation timepoints (p < 0.050). Alternatively, nLF, LF/HF, Short-Term Detrended Fluctuation Analysis (DFAα1), Long-Term Detrended Fluctuation Analysis (DFAα2), and Stress Index were all significantly lower during saturation compared to pre- and post-saturation timepoints (p < 0.050). SNS Index showed trends for reductions during saturation compared to pre- (p = 0.077) and post-saturation timepoints (p = 0.071). While 5/6 aquanauts observed a reduction in skin temperature during saturation, this was not significant across timepoints  with similarly elevated, but more variable ΔHR in post-saturation (p = 0.056). There were no significant differences in ΔMAP or ΔMCAv when standing, but there was a 16% reduction in MCAv during saturation compared to only 8% reduction during pre-saturation. There were no significant differences in cerebral autoregulatory index when assessing cerebral autoregulatory response during transition from prone to standing. There were also no differences in the magnitude of the drop in MAP or MCAv across timepoints.Baseline middle cerebral artery velocity (MCAv) when standing significantly decreased 11.8% during saturation compared to pre-saturation (p = 0.061) and REM/total (p = 0.093) trended higher during saturation, compared to pre-saturation and post-saturation timepoints. 4/6 aquanauts experienced reduced total sleep along with increased REM and deep/total sleep.No significant differences were found across sleep parameters between the timepoints . HoweverAquanauts reported no trouble falling asleep at pre-saturation and saturation, and very mild trouble falling asleep post-saturation . Total np = 0.0004), BMI , fat mass , mid-thigh skinfold , and chest skinfold  due to well characterized difficulties and limitations assessing psychological and physiological responses in aqueous and pressurized gas mediums . We foun2 and N2 inhaled. While not assessed in the present analyses, one of the few reports in multiday HBS confirmed the elevations in inflammatory and nitrogen/oxidative stress (RNOS) in this setting  (2 and/or pressure induced vasoconstriction (baroreflex), increased cardiac efficiency due to elevated PPO2 exposure in cardiac tissue, prolonged Q-T interval, and/or carotid chemoreflex inhibition , frequency domain , Poincaré plot/non-linear , detrended fluctuation  and composite metric (Stress and PNS Index) analyses. While LF was elevated, elevations in both LF and VLF are likely explained by increased global autonomic modulation in saturation, as LF controlled for total power (nLF) along with LF/HF were both decreased. These findings are in line with current evidence in acute HBS environment which have been shown to stimulate vagal modulation, reduced HR, and increased R–R interval variability and HF . The inct of O2) . Mechanihibition .2 availability in multiday HBS to cerebral and systemic tissues reducing tissue-driven blood flow demands. In support of this, 2 despite no change in arterial PCO2  all the way to post spaceflight, HR and respiratory rate have been shown to progressively elevate over time , which iAquanauts experienced increased trends in sleep latency and sleep quality  resulting from losses in systemic adipose tissue without detectable changes across muscle tissue metrics. While adipose tissue and skeletal muscle adaptations to multiday HBS have not been previously reported, bodyweight alone has been previously demonstrated to decrease by 2, 1.38, and 1.36 kg after 10, 45, and 60 days of saturation, respectively . Taken tad libitum food intake without exercise had unchanged fat mass , EVA exposures, and daily mission tasks were observed. In some instances, individual data points and sub-cohort analyses could not be reported to ensure compliance with NASA LSAH privacy policy.While NASA NEEMO provides one of the only research environments to evaluate hyperbaric habituation over multiple days and is an ideal space flight mission analog replicating the mission demands, strict laboratory conditions across some study parameters (indicated in each section) were unrealistic due to NASA NEEMO ecological research environment and the prioritization of safe mission completion. Additionally, mission logistics and safety inherently limited participant Aquanauts living at HBS for 9–10 days during NASA NEEMO 22 and 23 experienced intrapersonal physical and mental burden, sustained good mood and work satisfaction, decreased HR and respiratory rates, increased parasympathetic and reduced sympathetic modulation, lower cerebral blood flow velocity, intact cerebral autoregulation and maintenance of baroreflex functionality, and losses in systemic bodyweight and adipose tissue. Together, these findings illustrate multiple novel insights into the unique human adaptation response to multiday HBS with clear overlap and distinction between other environmental extremes. Future studies should include the exploration of the molecular bases for these multi-component changes, and how these adaptations affect human response across other parameters of operational performance ,b,c.mean data sets, not individual subject data points. Requests to access the datasets should be directed to AK, akoutnik@ihmc.org or akoutnik@usf.edu.Datasets presented in this article are not publicly available. NASA IRB protocol required the reporting of The studies involving human participants were reviewed and approved by Institute for Human and Machine Cognition, John Hopkins University, National Aeronautics and Space Agency, and/or European Space Agency IRB(s). The patients/participants provided their written informed consent to participate in this study.AK, MF, CA, AP, JS, and DD led mission organization, development and implementation. All the authors were involved in mission organization, development and implementation, reviewed, revised, and approved final manuscript draft, agreed to accuracy and integrity of manuscript. AK led survey experimental design, sleep experimental design, sleep data processing, body composition data processing, and body composition experimental design and developed the original manuscript draft. AK and SM led survey data acquisition and processing. AK and MS-G led cardiorespiratory and autonomic experimental design and data acquisition. AK, MS-G, and BG led autonomic data processing. MF and JS led hemodynamic experimental design. MF, BS, and JS led hemodynamic data acquisition. MF, AK, and BS led hemodynamic data processing. AK, SM, and MM led sleep data acquisition. AK, KN, SM, JD, and CR led body composition data acquisition. AK, MF, and MS-G led statistical analyses. All authors contributed to the article and approved the submitted version.The authors declare that this study received funding from Ketone Technologies LLC. Ketone Technologies LLC is an entity which supports research on extreme environments for which CA and DD serve as CEO and CSO, respectively. CA and DD received no personal compensation for these studies and were involved in project development, logistical implementation, and final manuscript development. These interests have been reviewed and managed by NASA in accordance with its Institutional and Individual Conflict of Interest policies. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


TORNADO on top of general-purpose big data technologies for intelligent video big data analytics in the cloud. The proposed framework acquires video streams from device-independent data-sources utilizing distributed streams and file management systems. High-level abstractions are provided to allow the researcher to develop and deploy video analytics algorithms and services in the cloud under the as-a-service paradigm. Furthermore, a unified IR Middleware has been proposed to orchestrate the intermediate results being generated during video big data analytics in the cloud. We report results demonstrating the performance of the proposed framework and the viability of its usage in terms of better scalability, less fault-tolerance, and better performance.In the recent past, the number of surveillance cameras placed in the public has increased significantly, and an enormous amount of visual data is produced at an alarming rate. Resultantly, there is a demand for a distributed system for video analytics. However, a majority of existing research on video analytics focuses on improving video content management and rely on a traditional client/server framework. In this paper, we develop a scalable and flexible framework called  Video data are acquired directly from real-world domains and meet the veracity characteristic. Handling large-scale complex video data is not worthwhile utilizing conventional data analysis approaches. Video big data pose challenges for video management, processing, mining, and manipulation. Therefore, more comprehensive and sophisticated solutions are required to manage and analyze large-scale unstructured video data. Furthermore, due to the large volume of video data, it requires a flexible solution to store and mine for possible decision-making. However, large-scale Intelligent Video Analytics (IVA) becomes a reality on the rise of big data analytics and cloud computing technologies.Videos are recorded and uploaded to the cloud regularly. Sources that are actively contributing to video generation include CCTV, smartphones, drones, and many more, which have resulted in a big data revolution in video management systems. Various leading industrial organizations have successfully deployed video management and analytics platforms that provide more bandwidth and high-resolution cameras collecting videos at scale and has become one of the latest trends in the video surveillance industry. For example, YouTube users upload more than 400 h of videos per minute , and morBig data technologies, such as Hadoop  or SparkTORNADO.Recently, the deployment of distributed computing technologies in the cloud for video big data analytics has been the center of attraction in academics and industry. In the literature, some efforts have been made to propose a Cloud-based Video Analytics System (CVAS) for online and offline IVA, for example, ,10,11. TTORNADO is many folds, but principally, it can be aligned with the deployment of IVA algorithms and services in the cloud for video big data analytics. The main contributions of the proposed TORNADO are listed below.We propose a distributed data curation framework for video big data analytics in the cloud and encompasses key components like role-based access controller, device-independent video stream acquisition and synchronization, lifelong video stream monitoring tool, and big data persistence.High-level abstraction on top of big data technologies have been developed and optimized for video big data analytics to hide the complexity of big data stack.TORNADO effectively resolves the data curation issues throughout the life cycle of the IVA pipeline by developing distributed data management modules both for real-time and offline analytics.The proposed TORNADO provides IVA algorithms and service creation and publishing APIs that enable developers and researchers to author and publish contextual and domain-specific IVA algorithms and services, which are made available to the developers while following as-a-Service (aaS) model [S) model , i.e., IS) model  and IVA-S) model . Under tS) model , the IVAWe propose and optimize a unified scale-out middleware called IR Middleware against issues like big dimensionality, intermediate results, and IVA pipeline orchestration.We implement the proposed framework and conduct extensive experiments to validate our claims.The contribution of The rest of the paper is planned as: Real-time video surveillance and unstructured batch video data analytics in the cloud is an emerging research domain, and has attracted the attention of researches and practitioners. The state-of-the-art literature is proof of the fact that there is an increasing interest in adopting big data technologies for video analytics in the cloud.In the context of CVAS, Hossain  pointed Zhang et al. ,22 stateIn literature, there are some more studies in which efforts are made to resolve CVAS related issues. Pereira, R. et al.  was motiTORNADO is an attempt to fill the research gap by proposing, implementing, and evaluating a novel cloud-based framework for video big data analytics. TORNADO is designed on top of state-of-the-art distributed computing and data management technologies. The proposed framework is composed of different types of components and APIs, which can be deployed on various types of computing clusters independently with the intentions of scalability, fault-tolerance, and extensibility. Efforts have been made to provide high-level abstractions to hide the complexity of the cloud and distributed computing technologies. These high-level APIs assist researches in focusing on distributed IVA logic instead of focusing on the platform. In this direction, the video data acquisition APIs allow external real-time video data sources to register and collect the batch video data from the registered users. The acquired data are then maintained securely through a distributed messaging system and a distributed file system. TORNADO provides role-based access to the video data through APIs that make the life of the developer easy while developing new video data mining algorithms and services. IVA experts can develop and register contextual distributed IVA APIs, which is then available aaS to the users. TORNADO assists the IVA life cycle by managing the high-level and low-level features through Intermediate Results (IR) Middleware during the IVAs pipelining. Furthermore, TORNADO develops anomaly detection on the video streaming instances based upon the contextual IVA algorithm.Unlike all of the above approaches, TORNADO is the foundation layer of an ongoing collaborative research project called SIAT [TORNADO is the base layer that allows the other layers to develop IVA algorithms and services. The Video Data Processing Layer (VDPL) is in charge of pre-processing and extracting the significant features from the raw videos and input to the Video Data Mining Layer (VDML). The VDML is accountable for producing the high-level semantic result from the features generated by the VDPL. The Knowledge Curation Layer (KCL) [TORNADO base layer under the aaS paradigm. The VDPL, VDML, and KCL are pipelined in a specific context and become an IVA service to which TORNADO users can subscribe video data sources under the IVAaaS paradigm. However, the scope of this paper is the TORNADO layer only.led SIAT ,30. SIATer (KCL)  deploys Video Source are the sources that either generate video streams from sources connected directly to real-world domains such as IP-camera or can be already acquired videos in the form of datasets. IVA in the cloud are performed either on the video streams or on video datasets and is referred as Real-time IVA (RIVA) and Batch IVA (BIVA), respectively. In the context of IVA, a video can be represented in a hierarchy, as shown in The process of IVA undergoes different phases, as shown in Video Source and IR, respectively. Once an IVA service is developed and deployed then TORNADO users can subscribe a Video Source under the IVAaaS paradigm. The User represents the stakeholder of the proposed framework, i.e., such as administrator, the consumer, and researchers/practitioners. A Domain. A Domain is a specific real-word environment, for example, street, shop, road traffic, for which an IVA service needs to be built. The combination of IVA service and hardware constitutes an IVA System where IVA solutions can scale-out and can run fast.The input and output of an IVA algorithm can be a sequence of frames or features. We call these features IR. Multiple algorithms are pipelined to build a domain-specific IVA service. The input and output of an IVA service is restricted to TORNADO in this section and the technical details in the next subsections. As illustrated in TORNADO is composed of six components, i.e., Real-time Video Stream Acquisition and Synchronization (RVSAS), Immediate Structured Big Data Store (ISBDS), Distributed Persistent Big Data Storage (DPBDS), ISBDS Representation and Mapping (ISBRM), TORNADO Business Logic, and TORNADO Web Services.Formally, we describe the main components of The RVSAS component provides interfaces and acquires large-scale video streams from device-independent video stream sources for on-the-fly processing. The video stream sources are synchronized based upon the user identification and the timestamp of the video stream generation. Then, it is queued in the form of mini-batches in distributed stream buffer for RIVA. The RIVA may vary in the context of a business domain, cross-linked with the video stream sources and user’s profile. RIVA services are deployed in a cluster of computers for distributed video stream processing to extract the value for contextual decision making. The video stream queued in the form of a mini-batch can be accessed while using Video Stream Consumer Services (VSCS). During RIVA service pipelining, the IR are maintained through Intermediate Results Manager (IR-Manager). Similarly, RVSAS is equipped with a Lifelong Video Stream Monitor (LVSM) to provide a push-based notification response to the client with the help of a publish-subscribe mechanism. The extracted values  and the actual video streams are then persisted into ISBDS and DPBDS, respectively.TORNADO Business Logic. The structured data is related to users, access rights, VDPL, VDML, KCL, metadata of the video data stream sources, batch datasets, big models, and service subscription information. This module also orchestrates the IR and anomalies being generated by a domain-specific IVA service. According to the demands of TORNADO, two types of operations are required to be performed on the ISBDS, i.e., random read-write operation against the real-time queries and bulk load and store operations for offline analytics. For such operations, we develop and deploy ISBDS Access Controller to access the underlying data securely against random and scan read-write operations.ISBDS component is responsible for storing and managing large-scale structured data in a distributed environment according to the business logic implemented by DPBDS is built on top of Hadoop Distributed File System (HDFS) and is responsible for providing permanent and distributed big data persistence to the raw video data, big models, and also supposed to maintain the actual IVA plugins deployed by other layers. During the contextual distributed offline video analytics, batch video data and models are needed to be loaded. Similarly, different HDFS file operations are required, for example, access permission, file creation. In this context, we exploit the services of DPBDS Access Controller.TORNADO. TORNADO Business Logic hides the complexity of the proposed system by establishing a well-defined set of uniform and consistent operations. TORNADO is built to provide IVAAaaS and IVAaaS over the web. Thus, we develop TORNADO Web Services on top of TORNADO Business Logic to allow the users to utilize the functionality of the proposed framework over the web. Further technical details of each component are described in the following subsections.The ISBRM component is responsible for validating and mapping the contextual data to the respective data stores according to the business logic of Handling a tremendous amount of video streams, both processing and storage are subject to loss . To handTopic Manager, Partition Manager, Replication Manager, Cluster Configurator, and Cluster Health Manager, as shown in We utilize Apache Kafka to collect video streams in the form of mini-batches from the producer cluster, buffer in the Kafka Broker, and then route mini-batches to the consumer’s cluster. To manage the Kafka Broker for large-scale real-time video stream management and according to the business logic of the proposed Framework, we develop a module for this purpose known as BCS. The BCS is composed of five sub-modules, i.e., Topic Manager sub-module is used to create new Kafta topics dynamically in the Kafka Broker Cluster on new RIVA service creation. When a new RIVA service is created then three types of Kafka topics are automatically created with name convention RIVA_ID, RIVA_F_ID, and RIVA_A_ID on the Kafka Broker Cluster. Here ID is the unique identifier of the service. These topics are used to hold the actual video stream, IR, and anomalies detected by the IVA services. The Topic Manager sub-module permits the admin role to list all topics, check its configurations, and can also override the configurations if required. Each topic consists of partitions to which the video stream records are distributed. In general, the degree of parallelism and height throughput is proportional to the number of partitions in a topic. We develop a Partition Manager sub-module to automatically adjust the number of partitions in a topic according to video streams from video stream data sources and according to the stream consumption of the consumer group. To ensure height throughput and parallelism, a formula has been established for choosing the number of Partitions in a topic ‘T’. Let us consider, we want to achieve throughput ‘t’ for a producer ‘p’ and a consumer ‘c’ on a single partition. Then we need at least Max partitions per topic.Replication Protocol, and topic partitions can be replicated across Kafka Brokers. The Replication Protocol carries a parameter called Replication Factor (RF). An optimal value three is recommended for RF, but it must not exceed then the number of Broker Servers. In the Replication Manager sub-module, we develop APIs on the top of the Replication Protocol to manage the replication factor accordingly. The Cluster Configuration sub-module is responsible for holding and managing the cluster configuration parameters which are used by the producer and consumer, respectively. Similarly, a Cluster Health Manager sub-module is provided to allow the admin role to manage the health of the cluster.For fault-tolerance, Apache Kafka provides Producer Cluster to acquire large-scale video stream. VSAS is composed of four sub-modules, i.e., Video Stream Acquisition Adaptor, Video Stream Event Producer, Frame Detector and Preprocessor, and Record Composed. The Video Stream Acquisition Adaptor provides interfaces for device independent video stream sources. If a particular video source is subscribed against a RIVA service, then the Stream Acquisition Service gets the configuration metadata from the Video Data Source DS in ISBDS and configure the source device for video streaming. Then the Video Stream Event Producer decodes the video stream, detects the frames, and forwards to the Frame Preprocessor for meta-data extraction and frame resizing. To communicate with the video stream source, a JavaScript Object Notation (JSON) object is defined. The contents of this object consist of five fields, i.e., data source id, number of columns and number of rows in a frame, the timestamp of the data origination at a data source and payload. This JSON object is known as Record, which is then forwarded to the Producer Handler, as illustrated in VSAS are client APIs and can be configured on the Producer Cluster. Its function is to receive the records from the VSAS module, serialize the records, form mini-batch and then send it to the Kafka Broker. If a video stream source is subscribed to RIVA service then the Producer Handler will rout the mini-batches to topic RIVA_1 in the Broker Cluster. Similarly, if subscribed to multiple RIVA services, then the stream will be routed to the respective topics, as shown in Topic Name, Partition Number, Key, and Value. The Topic Name is dynamically provided by the Subscription Data Source stored in ISBDS according to the video data source subscription. The Partition and Key fields are set to the video stream source ID (camera ID provided by the ISBDS Video Data Source meta-store). The input of the Value field is the Video Stream Record being composed and provided by the Record Composer. The Producer Recorded is then sent to the Serializer to convert the Producer Record to bytes array so that it can be sent over the network. The VSP collects the Producer Records and adds them to the batch of record, for sending to the Kafka Broker. We compress these mini-batches utilizing the snappy compression algorithm and then VSP sends these newly formed mini-batches to the Broker Server. We enable the ACK of the VSP to ensure the message delivery. Technically, the VSAS and VSP sub-modules are encapsulated in videoStreamEventProducer high-level API, as shown in Algorithm 1.The VSP are also client APIs and are deployed on the Video Stream Analytics Consumer Cluster. On each cluster, two types of client services are configured, i.e., RIVA services and VSCS. Each Video Stream Analytics Consumer Cluster has different domain-specific RIVA services whereas the VSCS are common for all. The VSCS assists a RIVA service to read the mini-batches of the video stream from the respective topic in the Kafka Broker for analytics, as shown in The acquired video streams are now residing in the Kafka Broker in different topics in the form of mini-batches. To process these mini-batches of the video stream, we have different groups of computer clusters known as Algorithm 1: videoStreamEventProducerRIVA_IR_ID in the Kafka Broker Cluster. Similarly, this module is also responsible for reading IR from the respective topic and persists to the IR data store for reusability  in order to avoid recompilation. The overall process is shown in RIVA services generate IR and need to be persisted. Video analytics are always expensive and to avoid the recomputation, we persist the IR to ISBDS (IR Middleware). The IR demands proper management in a distributed environment. Thus in this context, we develop IR-Manager, which sends and gets the IR to and from the topic RIVA_A_ID) by using the LVSM instance. To generate the notification base response, LVSM follows standard observer-based implementation [RIVA_A_ID and notifies the clients in near real-time and simultaneously persists to the ISBDS.A domain-specific RIVA service processes the video stream for anomalies or abnormal activities. If any anomalies are detected, then the same is sent to the Kafka Broker topic , as shown in step-4. RIVA operations  are carried out on the videoData as per the developer logic and requirements. The RIVA output in the form of IR and anomalies are handed over to the IRM (step-5) and LVSM (step-6) APIs, respectively.To understand the VSAS, IR-Manager, and LVSM, we create a template service (shown in Algorithm 2). This service is based on Apache Spark Streaming. The input to a RIVA service is the Algorithm 2: rivaServiceTemplateBig Data Persistence. The Big Data Persistence is the backbone of the TORNADO. The Big Data Persistence component is responsible for providing permanent and distributed big-data persistence to both the structured and unstructured data of the proposed platform. The Big Data Persistence provides two levels of abstraction on the acquired data, i.e., ISBDS and DPBDS, which are elaborated in the following subsections.The second component of the proposed framework is TORNADO, but it also has some limitations. It does not provide rich queries like SQL. Designing a complex system with HBase is a challenging task. Indexing can be challenging to design as a relational concept and is not implicit. An elegant solution to such issues exists in the form of Apache Phoenix [OLTP operations best by combining the power of SQL and the flexibility and scalability of NoSQL. Thus, we exploit HBase as an NoSQL distributed data store and Apache Phoenix as a query execution engine. The ISBDS hosts five types of data, as shown in ISBDS is provided to manage large-scale structured data in the distributed environment. Because of the data-intensive operation and according to the requirements of the other layer, technologically, we deploy a distributed NoSQL big data storage known as Apache HBase. The NoSQL fits well according to the need and demands of the  Phoenix . Apache TORNADO provides role-based access to users. User logs and the respective role information are maintained through the user catalog. All the sensitive user information is made secure by deploying an MD5 [g an MD5  encryptiData Source metastore, i.e., video stream sources and video datasets. The former one can be subscribed to RIVA services, while the latter one is eligible for BIVA services. The meta-information of these sources, along with access rights, are managed through the Data Source metastore.The proposed framework manages two types of video sources through Video Analytics Algorithm and Service meta-store, respectively. The metadata of an IVA algorithm are managed through ALGORITHM metastore, and the data structure is composed of eight different data fields, i.e., AID, UID, algoType, algoName, input, output, requiredResources, sourceCodeAddress, and description. The metadata of the IVA service are managed through SERVICE metastore and encompasses attributes like SID, UID, dsTypeID, serviceTypeID, serviceName, and description. The relation between the service and algorithm metastore is managed through a lookup table called The IVA algorithms and services are managed through TORNADO also demands a scale-out middleware to allow diverse types of IVA algorithms and services to communicate with each other. The produced IR also needs proper orchestration for reusability and to avoid recomputation. Against such diverse demands, we design a unified middleware called IR Middleware and it is the heart of the TORNADO framework. The IR Middleware is created on top of Hbase and is composed of two main parts, i.e., IVA algorithms and services generate heterogeneous types of results in the cloud and lead to the problem of big dimensionality. The IRID Design: The IR Middleware persists the extracted IR over time from videos. It has been optimized to support DateTime granularity-based queries of IR. This is accomplished through careful design of the row-key called ure data . Here ouColumn Families: The IR Middleware is composed of two types of column families, i.e., Distributed IR Middleware: By default, Hbase creates only one region for a table. Another split is created automatically when a table with one region gets large enough. Thus the deployment of IR Middleware will lead to the issues of hotspotting and load-balancing [TORNADO. Resultantly, the IR Middleware demands custom region creation and rows distribution across regions. To address these issues, let IR Middleware is split into R. Thus, some of the RS would not be used for alancing , which wR will handle exactly R, we design the In the interest of having Subscription metastore.Finally, users are allowed to subscribe video stream sources to a RIVA service. The subscription information are maintained through TORNADO. This sub-module is composed of five sub-modules, i.e., Connection Configurator, Schema Handler, Create, Read, Update, and Delete (CRUD) Operator, Active and Passive Data Reader, and Writer.This module is responsible for providing read-write access to the underlying data securely according to the business logic of the Connection Configurator is used to maintain the configurations parameters such as cluster configuration, connection information, drivers, and security protocol parameters. When a request is made for data access, this sub-module is used to establish and maintain the connection session with the distributed data store. The basic schema structure of the ISBDS is maintained through Schema Manager, and the same can also be used for portability purposes.CRUD Operator is primarily designed to provide low-latency real-time interactive CRUD operations over the distributed storage. The instances of this sub-module are used by the distributed RIVA engines and by TORNADO Business Logic. It hosts pre-defined generalized queries for CRUD operations. Upon receiving the request for clients, CRUD Operator matches the request parameters with the available set of queries. In the case of a parameter match, the selected query is executed with a real-time response back to the client.CRUD is designed and provided for CRUD operations on data persisted in ISBDS. TORNADO is designed to support offline analytics over the bulk of videos while using distributed in-memory processing engines like Apache Spark [Furthermore, he Spark . The Pashe Spark ) and perTORNADO, a formal User Space is created on the top of HDFS. The User Space is managed through a proper hierarchical directory structure, and special read and write access is granted to the owner. All the directories are synchronized and mapped according to the user identification in the user profile logs. In HDFS, under each User Space three types of distributed directories are created, i.e., Raw Video Space, Model Space, and Project Space, as shown in The DPBDS component built on the top of HDFS and is responsible for providing permanent and distributed big-data storage. The data are stored and mapped systematically according to the business logic of the proposed system. The DPBDS component is designed to effectively and dynamically manage the data workload in the life cycle of IVA algorithms. Upon new registration with the Raw Video Space is used for the management of the video data. The Raw Video Space is further divided into two types of videos. The first type is the batch video, which has been uploaded for batch analytics, where the second type is acquired and persisted from the video stream sources. The entire acquired stream is timestamped on persistence. The granularity level of raw streaming videos is maintained through video data sources. The IVA life cycle may need different models for training and testing purposes. The Model Space is provided to facilitate the users to manage the training and testing models according to the deployed IVA algorithm. Similarly, the developer can develop a new algorithm or service. The Project Space is provided to manage the source code of the respective developer.User Space, and the respective data in the DPBDS, we develop a DPBDS Access Controller module and it consists of three sub-modules. These sub-modules are HDFS Handler, Active DRW and Passive DRW. The former one is designed over low level APIs of HDFS with the aim of managing the file operations and permission control according to the business logic of TORNADO. The Active and Passive DRW are provided for real-time and offline read-write operations.To manage and operationalize the TORNADO Business Logic component and the Big Data Persistence (BDP). This component is responsible for mapping the contextual data to the respective data stores according to the business logic of TORNADO. In TORNADO Business Logic module. The ISBRM is composed of three modules, i.e., ISBDS Model Selector, Instance Mapper, and Data Validator.The ISBRM component works as a bridge between the ISBDS Model, which encapsulates the attributes of the respective ISBDS data stores. The ISBDS Model is an implementation of Facade and Adopter Design Pattern [TORNADO Business Logic and ISBDS is possible because of the ISBDS Model Selector.As stated in  Pattern . The actTORNADO communicates with the external word over web by following standard data exchange formats such as Extensible Markup Language (XML) [Instance Mapper sub-module extracts the structured attributes from the input XML/JSON data as resources which are then mapped to the respective model instance. It identifies and maps the classes for the extracted resources and places the instances accordingly using a deep copy method [Data Validator sub-module validates the data according to the ISBDS schema and also ensures the compatibility with the defined constraints of the respective big-table in ISBDS.ge (XML)  and JSONge (XML) . The Insy method . SimilarTORNADO Business Logic is a high-level abstraction on BDP and RVSAS and is provided to customize the data access according to the design philosophy of TORNADO. TORNADO Business Logic APIs utilization can be found in The BDP and RVSAS components are designed to support large-scale video data acquisition and domain-specific IVA in the cloud while exploiting an advanced IVA algorithm. The TORNADO Business Logic components. The TORNADO Business Logic is subject to extendibility depending upon the future requirements. However, the current release implements and provide six types of modules, i.e., User Manager, Source Manager, IVA Algorithm and Service Manager, IVA Service Discovery and Subscription Manager, Ontology Data Manager, and Notification Manager.In this context, the current version implements different modules in the User Manager module encapsulates all the user-related operations such as new user account creation, access role assignment, and session management. TORNADO provides role-based access to the user and currently provides three types of user accounts, i.e., admin, developer, and consumer. The user roles and access rights are shown in The Data Source Manager module, the user can manage the data in a TORNADO deployed cloud. The users can manage three types of data sources, i.e., video stream sources, video datasets, and models. The IVA Algorithm and Service Manager modules are built to manage, develop, and deploy new IVA algorithms and services on the top of Spark. The former one is provided aaS, i.e., IVAAaaS to the developer, while the latter one is provided IVAaaS to the consumer. The developer can create and publish new IVA algorithms. These IVA algorithms are then made available as IVAAaaS to other developers for utilization. Once IVA services are created and published by the admin/developer, then TORNADO users are allowed to subscribe to video sources against the provided IVA services. The Ontology Manager allows the developer to get the IR for decision making. The Ontology Manager provides a secure way of getting the IR and maps it according to the video ontology deployed by the KCL. Similarly, Cluster Management are provided for administration purposes in order to allow the system administrator to monitor the health and functionality of the system.Thorough TORNADO framework over the web, it incorporates top-notch functionality into simple unified role-based web services. The TORNADO Web Service Curation is built on the top of TORNADO Business Logic. The order and the intercommunication among different components of the proposed system are shown in Finally, to provide the functionality of the proposed TORNADO allows the participating layer to register distributed RIVA and BIVA services. The TORNADO follows the lambda architecture style [Speed Layer Execution Scenario, and Batch Layer Execution Scenario. The data of both scenarios are managed through a common layer called BDP, also known as Serving Layer [re style  and the ng Layer . Figure TORNADO deploys a pool of RIVA services that are made available to the user for subscription. The IVA services are deployed by the supporting layers . Once a video stream source is subscribed to a RIVA service in the pool of IVA services then the life cycle of Speed Layer Execution Scenario encompasses through different stages while using distinct TORNADO components. For the ease of understandability, these components are deployed on six types of computing clusters in the cloud, which are labeled explicitly as ‘P’, ‘V’, ‘K’, ‘S’, ‘N’, ‘I’, as shown in ‘P’ hosts VSAS and provides interfaces to an external video stream source. On configuration, the video streams are loaded to the respective RIVA_ID, in the cluster ‘K’. The cluster ‘K’ deploys Kafka, where the acquired video streams, IR, and anomalies produced by LVSM are buffered. In this context, the cluster ‘K’ is composed of RIVA_ID, RIVA_IR_ID, and RIVA_A_ID, as described in ‘K’ to ensure high throughput. The RF should be less than or equal to the number of workers in the cluster ‘K’ [RIVA_ID need to be persisted to the DPBDS and ISBDS data stores. For this purpose, the cluster ‘V’ deploys three types of TORNADO’s modules, i.e., VSCS, Video Processor and Persistence. The first module allows the cluster ‘V’ to read the video stream mini-batches from RIVA_ID topics in the cluster ‘K’. The cluster ‘V’ then processes, encode and extract the metadata from the consumed video data. Finally, the video stream persistence module saves the video data and the respective metadata to the DPBDS and ISBDS, respectively.The cluster ster ‘K’ . The min‘S’ is responsible for processing video stream in near real-time while using RIVA services. Different stream processing engines, for example, Apache Spark Stream, can be used for the development of RIVA services. The cluster ‘S’ deploys four types of modules. The first module is VSCS and is used to consume the video streams from the RIVA_ID in the cluster ‘K’. The second type of module is the actual RIVA service that analyzes the video streams. The RIVA service is loaded according to the RIVA services subscription contract made by a user. The IR producer/subscriber is used to send and receive the IR according to the application logic to and from the topic IR in cluster ‘K’. The fourth type of module is the LVSM producer. A RIVA service instance deployed in the cluster ‘S’ should have some domain-specific goal and can produce anomalies if analyzed any. The TORNADO supports a real-time anomalies delivery system. The RIVA service sent the anomalies continuously to the LVSM producer and the LVSM producer to the respective anomalies topic RIVA_A_ID in the cluster ‘K’. The cluster group ‘I’ read the IR from the topic RIVA_A_ID in cluster ‘K’ continuously and sent it to the ISBDS’s IR Middleware for indexing. The final type of cluster in the active view is cluster ‘N’ and is known as Anomalies Notification Cluster. This cluster aims to read anomalies from the topic RIVA_A_ID in cluster ‘K’ and send the same to the ISBDS for persistence and also delivered in real-time to the video stream source owner in the form of alerts.The cluster TORNADO framework is also equipped with BIVA services. These are available aaS for batch video analytics. The batch video datasets are analyzed in an offline manner, where the execution time is proportional to the video dataset size and to the subscribed BIVA services. The Passive Execution life cycle undergoes three types of cluster, i.e., ‘R’, ‘M’, ‘B’. The cluster ‘R’ allows the user to upload a batch video dataset to the TORNADO cloud and configure three types of TORNADO libraries. The first type of service is Batch Video Acquisition Service, which is used to acquire a batch video dataset. Once uploaded to the node buffer, the batch dataset is processed by the activated Video Processor to extract the metadata from the batch videos. The batch video data and the extracted metadata are then persisted to the DPBDS and ISBDS, respectively. Similarly, the cluster ‘M’ works the same way as that of cluster ‘R’, but this one is responsible for model management.The In the batch video analytics, the supporting layers deploy BIVA services. This cluster loads the instance of BIVA services as per user contract and processes the videos in an offline manner. Once subscribed, this cluster loads the batch video dataset and model from the DPBDS. Similarly, the IR and anomalies are maintained in the ISBDS. The acquired video stream residing in the DPBDS is also illegible for offline analytics.TORNADO is a complex system and is composed of many components that are deployed over different types of clusters. Thus we evaluate various aspects of the proposed system. Before the evaluation, we explain the distributed cloud environment, which has been used for TORNADO evaluation.TORNADO, we set up an indoor distributed cloud environment called TORNADO Cluster while deploying Hortonworks Data Platform (HDP) version 3.1.0 [TORNADO Cluster consists of eleven nodes, as shown in For on 3.1.0 . The TORTORNADO Cluster consists of five types of nodes, i.e., TORNADO Web Server, HDFS Server, Worker Agents, TORNADO Broker Servers, and HBase Server. The TORNADO Server hosts TORNADO Web Service, VSAS and video stream processing modules. This server also hosts Ambari Server [TORNADO components  and RIVA services (Face Detection and Action Recognition service). These nodes perform the actual near real-time analytics. We configure the TORNADO Broker Servers [The i Server . The Agei Server , Zookeepi Server , Yarn Rei Server  and Spari Server . Worker  Servers  on agent Servers . In FiguFace Detection and Action Recognition [TORNADO, three types of topics are automatically created by the Topic Manager (sub-module of BCS) as shown in three (as we have three Broker server) and set the number of partitions to 140 per topic . Further, we set the value of different parameters in the Cluster Configurator sub-module, ISBDS, and DPBDS, as shown in We have developed two RIVA services, i.e., ognition ,48. Thesognition ,48. ThesTORNADO Broker Server at the rate of 12 MS. These two modules are configured on TORNADO Server to acquire and send the stream to the TORNADO Broker Servers . The results of the performance testing are shown in As the VSAS module is device-independent, we register different types of heterogeneous devices and offline video stream sources. The heterogeneous video stream sources includes IP camera , depth cTORNADO to scale per system with the increase in video stream sources. We increase the number of video stream sources on the TORNADO Server from 5 to 140. This test case stresses the VSAS and VSP sub-components with an average of 54 messages per second per video stream source. The respective results are shown in To evaluate the ability of TORNADO stream acquisition component is then tested in a production environment while subscribing to the RIVA services. Initially, we register and subscribe 30 and 40 cameras with the Face Detection and Action Recognition services, respectively. The VSAS and VSP successfully acquire and sent the messages to topic TORNADO Broker servers. Furthermore, after 1.5 h, we unsubscribe 10 cameras from Face Detection service and re-subscribe the same to Action Recognition service. Up to three hours, we receive 95.63 million messages on the TORNADO Broker servers as shown in The TORNADO Broker servers. In Case-1, we can achieve the best performance in 20 threads while receiving 814 messages per second. Similarly, in Case-2, 3, and 4, the optimal performance was achieved in 13, 9, and 7 Threads, as shown in In this section, we evaluate the performance of VSCS module. VSCS acquire the video streams from the TORNADO Broker server in the form of mini-batches for analytics. The TORNAOD’s user in near real-time after the detection of some abnormal activity in the video stream by domain-specific RIVA service. We subscribe Camera-1 and Camera-2 to Face Detection and Action Recognition services, respectively. The rules of these anomalies are shown in The LVSM component is responsible for notifying the The monitored lifelong of Camera-2 is shown in Action Recognition service monitors the lifelong video stream of Camera-1 according to rules given in Similarly, the Face Detection service and Camera-2 to Action Recognition. In Case-1 the notification delay, on average for Face Detection and Action Recognition, 73 and 113 milliseconds were recorded. In Case-2, the average notification delay is 69, and 70 milliseconds was achieved. Similarly, in Case-3, we get 36 and 38 milliseconds on average. The delay comparison of LVSM is shown in Face Detection and Action Recognition generates 30,458 and 11,841 notifications and sends the same to the topic The performance of LVSM is based on its timely generation of notification for the real-time domain-specific video analytics service. For the LVSM evaluation, we introduce three types of delay cases, i.e., Case-1, Case-2, and Case-3. The life cycle of Case-1 considers the total time from frame acquisition to processing and then the notification. The delay of the notification, in this case, is proportional to the execution time of the RIVA service. In Case-2, we exclude the execution time, whereas in Case-3, only the delay between the occurrence of anomaly and publication of notification is evaluated. The LVSM services are configured on the agents of the work. For this evaluation, Camera-1 was subscribed to TORNADO is supposed to provide support for the IVA life cycle and to manage the IR. Once the features are extracted from the mini-batches of videos, the same is then produced for the respective topic. Face Detection produces bounding boxes, whereas Action Recognition services generate low-level features, i.e., VLBP [TORNADO Broker server. On average, each mini-batch of the video stream generates a feature vector of 100 KB against the VLPB algorithm, which becomes 40 KB after compression while using snappy compression. A single node using a single thread can process 80 messages per second. When using four consumers and four producers with 25 threads each, we get up to 1645 messages per second, as shown in Broker servers. As we are using the RF three; thus, the data are accurately replicated to the three broker servers which ensure the fault-tolerance.As the e., VLBP . These aIn this section, we evaluate the performance of IR Middleware. We utilize the IR-Manager to read the IR from The write performance of the IR Middleware is shown in TORNADO Server. The Active Data Writer consumes the video stream from the topic RIVA_ID (TORNADO Broker server) and persists the video stream to the HDFS. The performance result of the Active Data Writer, i.e., blocks that are written to the data node, is shown in To evaluate the DPBDS performance over HDFS, we have performed experimentation both on Active and Passive Data Reader and Writer. The HDFS instances are configured on the worker agents (Data Nodes) and HDFS Server (Name Node) as shown in This section discusses the features of the proposed TORNADO that are related to state-of-the-art CVAS. Furthermore, it also highlights features of the proposed system that leverage it from existing systems. The comparison of the proposed system with commercial and scholarly work is shown in Some of the features of the proposed system are similar to the existing commercial and scholarly systems. Commercial CVAS like Google Vision , and IBMTORNADO framework is intended to provide a service-oriented echo-system while utilizing open-source big data technologies to facilitate IVA developers and scientists. Users can develop and deploy new IVA algorithms under IVAAaaS. The IVA algorithms can then be pipelined to create a domain-specific IVA service. The users can then subscribe to video sources to the available IVA services.Unlike existing CVAS, the In the context of IVA, the proposed system provides higher-level abstractions on the low-level APIs on top of big data solutions, which assist users to focus more on the IVA solution.The proposed system is equipped with IR-Manager and LVSM to maintain and manage the IR and anomalies.TORNADO.Unlike existing solutions, the proposed framework is facilitated with scale-out IR Middleware, which addresses the issue of big dimensionality. The IR Middleware allow the integration of diverse types of IVA services with The proposed system is based on the lambda architecture and can easily be extended to the fog based solution, i.e., RIVA services can be deployed near to the video stream data sources and BIVA service on the cloud resources.The principal dissimilarities between the proposed CVAS and the state of the art solutions are listed below.TORNADO framework, which focuses on curating large-scale video data in the cloud environment while deploying IVA algorithms and services under an aaS model. TORNADO is a pluggable and scale-out framework, which is designed to bridge the gap between IVA algorithms and service developers. It provides high-level abstractions on top of big data stacks to allow researchers/developers to focus more on the IVA services in the cloud. TORNADO efficiently manages the IR during the IVA life cycle. Furthermore, IVA algorithms can easily be pipelined to create a domain-specific IVA service. The data source-independent implementation of TORNADO makes it more scalable and IoT compatible. TORNADO is also facilitated with LVSM, which monitors the video stream against the rules defined in the IVA service by domain experts. Distributed Big Data Persistence is provided to support a large volume of raw video data, models, structured data, and the IR, which enables the TORNADO to support data-driven knowledge generation, descriptive and predictive analysis, and visualization.In this paper, we presented the TORNADO framework performance, accuracy, and scalability have been successfully evaluated against the demands of the TORNADO for real-time and offline IVA. From the results, it is clear that the proposed framework performs efficiently and effectively in the production environment.The TORNADO against in-memory computing engines like Apache Spark.The current version has been validated against RIVA services. In the future, we will evaluate and optimize the IR Middleware against tree data structures, and will provide more IVA services, i.e., offline video analytics, and deep-learning-based approaches. In the future, we are also investigating to optimize the