We assessed surgical workflow recognition and report a deep learning system, that not only detects surgical phases, but does so with high accuracy and is able to generalize to new settings and unseen medical centers. Our findings provide a solid foundation for translating AI applications from research to practice, ushering in a new era of surgical intelligence.AI is becoming ubiquitous, revolutionizing many aspects of our lives. In surgery, it is still a promise. AI has the potential to improve surgeon performance and impact patient care, from post-operative debrief to real-time decision support. But,  To put this in context, during an average 85 year lifespan it has been estimated that the average American will undergo more than nine surgical procedures2. It is also well-established that the quality of surgeon performance has a critical impact on the quality and overall cost of care, with poor performance resulting in substantial increases in morbidity, reoperation, and mortality5. Yet, much of surgical training and credentialing is still performed as it has been for the past 100 years—namely, through mentored one-on-one graduated training7.Surgery is an indispensable element to the treatment of disease. It is estimated that 312.9 million major surgical procedures were performed worldwide in 2012, including 36 million in the United States alone3 these metrics can be used to predict patient outcomes. Taken together, this suggests that AI-based systems that are able to provide real-time recommendations during surgery have great potential to support surgeons’ training and their decision-making process8 potentially reducing the risk for complications and improving outcomes10.Recent advances in artificial intelligence (AI) hold potential to enhance surgical training, improve surgeon performance and ultimately surgical outcomes. This is particularly true for minimally invasive surgery (MIS). In general surgery for example, the MIS surgeon operates by viewing the surgical field via a live video stream acquired by a laparoscope, inserted into the abdomen. In these, as well as robotic procedures, most essential aspects of surgery are captured by video feed. It has also been established that certain critical parameters of surgical performance can be measured from video by human observers10. As a result, nearly every study of machine learning applied to surgical performance has been on a small  video recording procedure set15. To date, the largest publicly available dataset is Cholec80, which contains 80 videos of laparoscopic cholecystectomies performed by 13 surgeons from a single medical center11. Given that it is well established that even human surgeons require well more the 100 repetitions of the same surgery to reach expert performance levels18 and the fact that typical video classification models in other domains are trained on more than 100,000 videos20, it seems unlikely the results reported on a few dozen videos represent the true potential of AI for surgery.A key challenge for machine-learning (ML)-driven AI is the need for correctly labeled, representative data to drive the learning process. Medicine in general, and surgery in particular, present unique challenges to the capture, labeling, and sharing of dataHow many surgical video recordings are necessary to train an AI system to recognize the major phases of a surgical procedure? and (2) How robust is the learned model to new sources  of data? To answer these questions, we report the results of state-of-the-art ML on a dataset that contains 1243 videos of laparoscopic cholecystectomy—an order of magnitude (10 times) larger than all previously published studies.In this study, we focus on answering two key questions fundamental to the application of AI to surgery: (1) 21. The task of surgical phase recognition is similar to the task of general action recognition23 but is more challenging due to the need to correctly label every second of surgery with a phase of the task, e.g. Dissection, Division, Separation, Packaging, and so forth. It is essential for predicting the next surgical task, identifying critical events timing during surgery and rapid navigation within a video, which facilitate various use cases such as post-operative debrief. Despite its benefits, routine post-operative debrief is not common. Mainly because in today’s work environment, reviewing a lengthy procedure, that just took hours to complete, is a time-consuming task that no surgeon has time for. The ability to quickly summarize a video into a palatable version, in which every major element can be quickly viewed, is a step in the right direction. Overlaying this, the recognition of particular adverse events, such as bleeding and bile spillage, and linking them to the particular phase of the procedure, will provide important analytics that can be the basis for targeted debriefing and educational interventions to enhance safety.We evaluated our ability to train a system to learn surgical context using the benchmark task of surgical phase recognition—that is, segmenting a video of surgery into the major steps that constitute the procedure. Phase recognition is a foundational step in automating surgical analysis applications and was used previously in surgical modeling challenges24 surgery videos are limited due to privacy and regulation issues. Additionally, the process of annotating surgical videos requires skilled personnel, making the curation process complex, slow and as a result—expensive.The core of our algorithmic approach is the application of recent advances in deep learning to video sequences. The dramatic impact deep learning has had on AI in general, and computer vision in particular, is strongly related to the availability of large, representative datasets. In contrast to publicly available images and videos in online repositories that contain millions of samples19 and the work of Abu-El-Haija et al.20, have nonetheless enabled the development of state-of-the-art methods in video classification and action recognition tasks, where the goal is to predict a single label for a given video23.The success of deep learning on image-based tasks has led to growing interest in research applying similar ideas to video-based problems. Even though the two share many key elements, the main challenge when handling videos is the addition of a temporal dimension, thus increasing the dimensionality and size of the data to be processed. A growing collection of large-scale video datasets, such as the work of Kay et al.Compared to general action recognition in video, the task of surgical phase recognition benefits from the fact that different procedures of the same surgery are similarly structured. The procedural flow, the tools used, and the surrounding anatomic context are similar no matter where the surgery is performed and who is performing it. However, in contrast, in surgical phase recognition we need to detect relatively small differences across time—a change in tool, a change in anatomic target, or difference in how a tool is used. These details are sometimes difficult to discern, even for an expert surgeon. Further, the system must do this correctly for every second of video rather than computing a single class label for an entire video. This makes it difficult to predict both the amount of data necessary to train a model for surgical phase recognition, and the robustness of the model to natural variations that will occur in practice, and thus motivates our study.25. The vast majority of gallbladder resections are done in relatively universal and predefined steps. These steps are the standard workflow a surgeon goes through during surgery and are known as surgical phases.Gallbladder removal, or cholecystectomy, is performed more than 750,000 times annually in the US alone, mainly for benign gallstone disease which affects 10–15% of adultsReviewing a surgical video and classifying each of its parts into the correct phase requires a set of guiding rules. These rules define the transition between phases and the sub-tasks performed in each phase. Determining the right phase anthology is a challenging task. It is important to establish a generic set of rules which eventually reduce the potential for subjective judgment when labeling phases during surgery.Phase 0: Preparation obtaining pneumoperitoneum, trocar placements, optimizing exposure through patient positioning and retraction. Generally, this will be the first phase, starting once the procedure begins or as the camera entered through the trocar. However, some videos lack this phase due to a delay in starting the recording and thus start at a later phase.Phase 1: Adhesiolysis dividing adhesions. It begins when the surgical tools (like scissors or monopolar hook) start separating the adhesions. This phase is not mandatory and only performed when adhesions exist.Phase 2: Dissection dissecting hepatocystic triangle or separating gallbladder lower part from the liver bed. It begins when related surgical tools start the dissection and complete after achieving optimal visualization and skeletonization of key anatomical structures, e.g. cystic duct and cystic artery.Phase 3: Division performing cystic structures division. It starts with the introduction of a related surgical tool, such as a clip applicator, a ligature to occlude the cystic duct and artery, scissors, stapler, or even an energy device.Phase 4: Separation gallbladder dissection from the liver bed. This phase begins when a separating tool enters or, in case the tool is already present as part of the previous phase, when it starts dissecting and separating the gallbladder from the liver bed.Phase 5: Packaging gallbladder packaging. Phase starts when an endobag enters or first shown in the video stream. This phase is not mandatory as surgeons might extract the gallbladder using a strong grasping instrument.Phase 6: Final inspection a closing step when the surgeon is completing the operation by reviewing the abdominal cavity, extracting the gallbladder, removing instruments and trocars. This phase begins when any act of procedure closure is done.In this work, we define seven laparoscopic cholecystectomy phases: Preparation, Adhesiolysis, Dissection, Division, Separation, Packaging, and Final inspection. Although phase definition is not standardized, phases were defined based on discussions with expert surgeons and AI researchers, and validated for relevancy by an iterative process of hands-on annotations experience, taking into account clinically-relevant and algorithmically-meaningful considerations. In defining phases, we attempted to simulate a surgeon’s common workflow, focusing on the goal or action performed in each phase, and using surgical tools as cues (when possible) to support the beginning of each phase.Although these phases aim to follow the progression of a typical laparoscopic cholecystectomy procedure, the order of appearance might change and vary due to different reasons, e.g. phase 5 can occur after phase 6 in case the gallbladder is packed only at the end of the procedure. Moreover, the surgeon might alternate back and forth between phases, e.g. when cystic structures division is performed, the surgeon might choose to keep doing the dissection phase in order to further clear the cystic structures surrounding.11, but we did modify a few phases after reviewing our dataset. This is done in order to provide a more generic set of guidelines. We separate phase 2  of Twinanda et al.11 into two different phases: Adhesiolysis and Dissection. First, there is an application difference between adhesiolysis and dissection in terms of the method used by the surgeon, the patient’s condition, the excessiveness of adhesions, etc. And second, the fact that adhesions are not limited to the gallbladder and might include other organs not related to the Calot triangle. In addition, after reviewing procedures within our dataset we found that in a large number of cases gallbladder retrieval does not necessarily occur at the end of the procedure as in Cholec8011. This led to unnatural phase jittering between the cleaning phase and the retraction phase. In order to support this variation, we decided to combine the final visualization of the abdominal cavity and the extraction of the gallbladder as part of the final inspection phase.Phase definition in this work shares some common ground with the work of Twinanda et al.We underline that these sets of guidelines were defined independently of any model training or performance analysis in order to avoid any biases.Supplementary Table All videos in this study are processed in the same manner. Initially, videos are processed using FFmpeg 3.4.6 on Ubuntu 18.04 and all video streams are encoded with libx264, using 25 frames per second (FPS). The video width is scaled to 480 and the height is determined to maintain the aspect ratio of the original input video. The audio signal is removed from all videos.Since videos are recorded during surgery and are not edited by the staff, the original video has non-relevant segments at the beginning and end of the video file. We trim off these segments in order to avoid noise segments that are not relevant to the procedure. In order to achieve the trimmed video version, we use a background detection model. This model was trained to identify non-relevant frames that were captured outside the body. The non-relevant frames are then utilized to identify the actual start and end of the procedure in the full video and trim it down.Preprocessing steps and the final verified video files are automatically processed and stored in a compliant, private, and secure cloud environment.26 and increase the generalization of our approach, we train on a diverse dataset and evaluate the models' robustness against several key factors.Our method, further detailed below, relies only on surgical phase labeling. As a first step, it learns from short surgery clips, taking advantage of the temporal context instead of using static 2D images, then the entire procedure is analyzed as a time series in order to learn long dependencies. In order to overcome overfittingThe proposed surgical workflow phase detection system is a two-step framework that consists of two modules, short- and long-term temporal context Fig. . The fir22. In practice, this is done by adding an additional temporal dimension to the original network. In addition, in order to capture long-range temporal dependencies, we use non-local blocks within the I3D architecture23. The I3D model is obtained by converting a 2D image classification model. In this work, we follow Wang et al.23 and inflate a ResNet-50 model27. We use a pre-trained ResNet-50 weights, trained on ImageNet24 to initialize our I3D model weights, such initialization was shown to be beneficial when converting 2D models to their 3D counterpart22. Next, we train the model for the video action recognition task on the Kinetics-400 dataset19. This pre-training step is done in order to achieve a temporal baseline model on a large-scale video dataset. Finally, we finetune the baseline model by training on the surgery video dataset, replacing the last layer which is a 400-way classification layer with a randomly initialized 7-way layer to match the number of phases in laparoscopic cholecystectomy surgery28.The short-term model is based on inflating a Deep Convolutional Neural Network (DCNN) designed for 2D images into a 3D ConvNet model (I3D) designed for video action recognitionThe short-term model is trained to predict a single phase for each second of the surgical video. We consider each second as an independent sample and the model input is a short 2.56 s clip around that second. As videos are encoded at 25 FPS this yields a clip of 64 images. We use a mini-batch size of 16 clips and train on a 4-GPU machine. Each mini-batch samples are randomly formed during training by randomly selecting a video and one of its seconds from the training set.Each sample clip is first resized with its shorter side to 256 pixels. The input spatial size is randomly cropped during training and center cropped during evaluation to a 224 × 224 crop. Following a per channel mean and standard deviation normalization using preset ImageNet values . Further augmentation is done during training, as part of producing the sample’s clip, by randomly selecting one anchor (middle) frame from the 25 possible frames within a second. Optimization is done using cross-entropy loss and Stochastic Gradient Descent (SGD) with an initial learning rate of 0.01 and a momentum of 0.9. The validation set is evaluated every 250,000 training samples and the learning rate is reduced three times by a factor of 10 after 10, 20 and 25 validation iterations and stop after 30 iterations. This translates into 2.5 million, 5 million, 6.25 million and 7.5 million training samples.The model prediction errors can be divided into near errors, e.g. phase 3 is mistakenly predicted as phase 2, and distant errors, e.g. phase 0 is mistakenly predicted as phase 6. In order to overcome the latter one, we use a progress ratio and calculate a Second Duration Ratio (SDR) by dividing the sample second value with the overall video length. The SDR value is concatenated to the final layer before the classification layer. We found this addition to significantly reduce distant errors.The final output of the short-term model is obtained from a SoftMax layer, which produces a probability vector per second, assigning a probability value for each phase. These vectors are then sequentially stacked together to create an L*7 matrix representation for each video, where L is the video length.29.The long-term model is based on a Long Short-Term Memory (LSTM) network. This type of Recurrent Neural Network (RNN) uses special hidden units to maintain and accumulate memory from previous samples, enabling the model to learn long-term dependencies between samples30. We use a relatively high temperature value, T = 11, which produces a softer probability distribution. The probability vector is used to create a categorical distribution from which we randomly select a phase per second. The combination of categorical distribution phase selection and SoftMax with temperature considerably contributes to the model’s robustness and generalization.Here, a single input sample is considered as a sequence of phases values from an entire video. The fact that the input samples are an entire video sequence limits the training process to a relatively small number of samples. In order to avoid overfitting and apply data augmentation to such input, we use a SoftMax with temperature technique on the phases probability vectorsThe phases sequence is first processed with an embedding layer that works as a lookup table mapping each phase in the input to an embedding vector of size 32. Next, a single layer LSTM network with 128 hidden state features takes the embeddings as input. We use a bidirectional LSTM which uses past and future information to produce a final representation for each second in the input sequence. Finally, a linear layer operates as the classification layer and maps the output from the LSTM hidden space, back to the phases space. Cross-entropy loss is calculated for each second in the input video and model optimization is done with SGD using a fixed learning rate of 0.1 and a momentum of 0.9. We use a mini-batch of one since each video has a different sequence length and train the model for 30 epochs on a 1-GPU machine (an epoch is a single pass on the entire training samples).Both models’ hyperparameters are first explored for the largest training dataset and remain fixed for all other experiments. For the short-term model, in case the training set number of available seconds is less than 250,000 the evaluation step is done every epoch and learning rate reduction is done in the same manner as before at 10, 20 and 25 validation iteration and stop at 30 iterations.In the finetune for unseen medical center experiments we use only 15 training epochs in order to train the baseline model, reducing the learning rate after 5, 10 and 13 validation iterations. Although the number of training samples was 263,494 we ran an evaluation step every epoch. Fine-tuning on a different amount of training videos from medical center 1 is done in the same way, only initializing the models’ weights using the baseline model.31.Our system is implemented with Python 3.6 using PyTorch (1.1.0) as the main deep learning framework19, this publicly available dataset contains about 240,000 training videos and about 20,000 validation videos. It was developed to facilitate research for human action classification and perform as a benchmark for building video classification models. Each video clip in Kinetics was curated from a different YouTube video and lasts around 10 s. It is labeled by a specific task, such as playing the clarinet, riding a bike, shaking hands, etc.The short-term temporal model is first trained on the Kinetics-400 dataset22 is first trained as a 2D model27 for static image classification. This is done on the publicly available ImageNet dataset24. ImageNet contains about 1.3 million training images and 50,000 validation images from 1000 different classes.The backbone for the 3D temporal model11 and the rest from our internally proprietary dataset. Supplementary Tables In order to train the short- and long-term models for the phase detection task, we use a large laparoscopic cholecystectomy dataset. Our complete labeled data set consists of 1243 surgery videos curated from several sources, six different medical centers and more than 50 surgeons. It includes 80 videos from Cholec8032, in which an unbiased group of surgeons reviewed large portions of the cases in our dataset and reported high agreement with our annotations.Each video undergoes a rigorous annotation process by two different annotation specialists. Annotators are a group of medical students and surgeons who underwent thorough training on labeling the phases. A video is randomly assigned to a first annotator that reviews the video and manually assigns a surgical phase for each second of the video. The second annotator then validates the labels to reduce manual errors and enforce an aligned annotation process. In the case of unclear workflow or non-typical events, we also consult a group of surgeons in order to maintain high-quality labels. Annotation validity was confirmed by Korndorffer et al.Although laparoscopic cholecystectomy is a relatively linear structured surgery, not all phases are mandatory to take place. We do force full covering of each video, i.e. each second is labeled with one and only one phase.33 which is not a typical approach in our dataset, or those surgeries which eventually converted to open surgery. We tried to keep videos of both high and low quality, only excluding those in which the video is corrupted.We apply several criteria for rejecting a video. We excluded videos in which surgery is done in a retrograde approachDuring the training and evaluation process, the video dataset is analyzed on a single second basis. Thus, the 1243 videos translate into more than 2.3 million second samples. Supplementary Table 15. The limitations mentioned in these previous studies are addressed in our work. We use a common practice in ML of dividing our dataset into a training set, composed of hundreds of videos, a different group of videos as a validation set to tune our models’ hyperparameters and another separate subset as an independent test set to further verify the results26. First, we randomly allocate 25% of the videos to serve as the independent test set. This test set is never seen during the training and hyperparameter tuning process and provides an unbiased evaluation set for the final model. The remaining videos are divided again into train and validation sets by an 80/20% ratio. Both short- and long-term models learn from the same train set and their hyperparameters are tunes on the same validation set. This setup yields a train, validation and test sets of 745, 187 and 311 videos, respectively. All subsets are randomly created on the video file level, thus each video takes part in only one subset. Both the short- and long-term models are trained on the same training set and hyperparameters are tuned on the same validation set.The task of achieving an automated system that is able to recognize surgical phases with high accuracy has been explored in recent studies. However, all are limited to a small number of samples and lack the ability to generalize well to new samplesThe main performance metric throughout this work is per-second accuracy. Each second of video footage is annotated with a single surgical phase. Accuracy is calculated by comparing the ground truth annotations with the system phase predictions Fig. A–5. AccuOur system has 90.4% accuracy and 86.1% mean phase accuracy on the validation set. On the test set it achieves 91.7% accuracy and 87.5% mean phase accuracy.Classification errors can be generally grouped into three types: (1) A temporal shift where a predicted phase starts or ends early or late compared to the true labeled phase, 2) Adding or dropping entire phase segments, and (3) Random flips of a phase label. In order to better understand the nature of our system errors, we examine the confusion matrix  between the prediction and the true start time of each phase for different τ values . Table The best performing hyperparameters established on the validation set were used for all model training and testing throughout this work. Initially this was done in order to enable a fair comparison between models, but we found this set of hyperparameters an important result of our work, making it possible to use our approach and train models on relatively small publicly available datasets, without the need of a complex and long hyperparameters exploration on large train and validation sets.Our first question was how many surgical videos are needed to reliably recognize the phases of surgery. We approach this by examining the change in the system accuracy on the test set as a function of training data size. The validation and test sets are kept fixed in order to maintain a fair comparison. Models’ hyperparameters were first optimized for the largest training dataset and remain fixed for all other experiments.We start by examining the short-term context model, training the model on a different number of videos, ranging from a single video up to 745 videos Fig. A. The moFigure The question of how many videos are needed for surgical phase recognition can also be examined as a generalization question. Namely, how effectively does the system generalize to various data sources—i.e. surgeons or medical centers.We first consider the effect of data imbalance related to medical centers (Supplementary Table 11 (80 videos) which includes 13 additional surgeons (without a surgeon labeling per video) and an additional group (121 videos) missing surgeon labeling (unknown group). In the analysis, we focus on 11 different surgeons, the Cholec80 group and the unknown group. These 13 subsets have more than 10 videos in the dataset , Cholec80 groupWe measure the accuracy of each surgeon separately within the validation and test sets Fig. C,D. Most28. In the scope of this work, as the dataset is skewed toward MC1, which has 1002 videos, it is interesting to examine the effect of discarding MC1 samples during training. We explore how the largest medical center affects the overall performance and whether there is a bias toward that specific medical center.Model adaptation is an important capability that enables fast deployment in new surgical departments and medical centers. Adapting models to a new unseen medical center can be done using a finetuning approach, in which the learning process continues on a relatively small number of new samplesThe original dataset is divided into two subsets. The first is formed by discarding all MC1 videos which yields a training set of 95 videos and a test set of 119 videos. The second subset contains only videos from MC1 and has 650 training videos and 192 test videos.Next, new baseline models were trained on the first subset. The short- and long-term models achieve 83.15% and 89.6% accuracy on the first subset test set, respectively. However, these models produce only 73.9% and 79.2% accuracy on the second subset (which has only MC1 videos). This shows degradation of about 10% compared to the first subset, and is consistent with what we expect to see on smaller than 100 training videos (Fig. We adapt the baseline models to the MC1 by continuing the learning process of both models. In order to evaluate how many samples are needed, we use an increasing number of training samples from MC1, ranging from 5 to 200, fine-tuning the baseline models and measure the performance on the second subset test set (Fig. The short-term model achieves more than 80% accuracy after only 50 new videos and was able to reproduce the original result at the 200 videos Fig. . The lonThis experiment further supports the fact that training with only tens of videos is not sufficient to achieve a generalized model and handle unseen samples. The fine-tuned model reaches 90% accuracy only after training on hundreds of new videos from MC1.34. Our study aims to support the transition from bench to bedside and help translate research into tools that aid surgeons, practically, in their daily routine. The viewpoint by Wang et al.35 discusses the challenges of integrating deep learning applications into the clinician’s workflow. Challenges such as data quantity and quality and model generalizability are addressed in this study. Since our dataset is more than an order of magnitude larger than any previous study it provides in-depth knowledge of surgical workflow characteristics. In addition, as ML systems are often considered as a “black-box”, we train our approach based on predefined phase mapping, agreed to by a group of experts in the field. This ensures that healthcare professionals understand and can interpret the results.In this work, we follow previously suggested guidelines for ML to overcome “barriers to deployment” and have an actual impact on healthcareIn summary, the contributions of this work are threefold. First, we leverage a large and diverse surgical video dataset and apply state-of-the-art methods for video analysis to introduce a deep learning system that achieves more than 90% accuracy in detecting the correct surgical phase. In order to support future research in this field, we report the set of hyperparameters learned and optimized with our dataset, allowing to train models on smaller but publicly available datasets. Second, we report a comprehensive analysis, assessing the likely asymptotic performance of our phase detection system and evaluating the generalization of our approach to different medical centers and a diverse surgeon population. Third, given a set of unseen videos from a new medical center, we demonstrate that by fine-tuning on a relatively small number of new samples, our model can converge to a similar high performance.Examining the number of videos needed in order to learn the surgical workflow shows high return in performance when going from tens to hundreds of training videos, where a diminishing effect in accuracy is starting to appear. We do note that our performance assessment was done using a fixed model capacity. Increasing the available data to thousands or even tens of thousands of videos may offer additional improvements by also increasing the model capacity.As opposed to elite athletes who can review performance and analyze every roll, pitch and yaw post-competition, surgeons lack objective tools to routinely debrief and analyze performance. The current level of accuracy allows for the creation of surgical highlight reels that can be returned to a surgeon quickly after exiting the operating room. Higher levels of accuracy will be required in order to leverage such capabilities for real-time decision support.The results in this report are limited to one specific surgical procedure, laparoscopic cholecystectomy, which has a relatively linear phase progression. Further analysis is needed in order to ensure the transferability of the ideas presented here to less structured or lengthy procedures. In addition, both modules do not assume any causality constraints and design to operate offline on the entire surgical video. However, enforcing such constraints can easily be done by adjusting the models to “look” at past inputs only. The evaluation of such models is not within the scope of this work. Although procedures from MC1 were curated over decades and capture high surgical variability, the dataset is still skewed towards this single center. Our generalization exploration focuses on high-level biases, which might occur due to different medical centers, surgeons and techniques. Once sufficiently labeled, future exploration should be done to assess both patient-level bias factors such as age, BMI, ethnicity, sex, anatomy variance, etc. and medical-center-level bias, e.g. by grouping procedures based on visualization hardware, instruments, or on the period or date of performing the surgery. Our models are also limited in handling outliers, such as very low video quality or in case a new surgical tool, never seen before, is used.11 and EndoTube36. The ability to detect surgical phases with high accuracy can promote the development of other surgical applications and support continuous research in this field. In addition, robust phase detection models can be the catalyst for studies of intraoperative event detection or applying transfer learning on other types of laparoscopic procedures.The hyperparameters described for the short- and long-term approach enable training robust models on relatively small, but publicly available, datasets such as Cholec8010. Regardless of the use case, a foundational step in integrating AI systems into routine surgeon workflow is the ability to analyze and successfully discern between different surgical phases. Thus, surgical phase detection is a key benchmark problem to assess the ability of AI to successfully learn surgical context.There are many opportunities for ML-based AI in the operating roomWe believe our analysis can further accelerate computer vision-based research and applications for laparoscopic surgery to the point of integrating AI systems in the surgical workflow routine, assisting in the decision-making processes and ultimately improving surgeon experience and patient care.Supplementary Information.


Acute malnutrition is a nutritional deficiency resulting from either inadequate energy or protein intake. Children with primary acute malnutrition are common in developing countries as a result of inadequate food supply caused by social, economic, and environmental factors. Secondary acute malnutrition is usually due to an underlying disease causing abnormal nutrient loss, increased energy expenditure, or decreased food intake. Acute malnutrition leads to biochemical changes based on metabolic, hormonal, and glucoregulatory mechanisms. Most children with primary acute malnutrition can be managed at home with nutrition-specific interventions . In case of severe acute malnutrition and complications, inpatient treatment is recommended. Secondary acute malnutrition should be managed by treating the underlying cause. Acute malnutrition is a nutritional deficiency resulting from either inadequate protein or energy intake. In 1959 Jelliffe introduced the term “protein calorie malnutrition”, which has been replaced by “acute malnutrition”. Olsen et al. .According to the American Society of Parenteral and Enteral Nutrition (ASPEN) , pediatrPrimary acute malnutrition in children is the result of inadequate food supply caused by socioeconomic, political, and environmental factors, and it is most commonly seen in low- and middle-income countries ,5. RespoSecondary acute malnutrition is usually due to abnormal nutrient loss, increased energy expenditure, or decreased food intake, frequently in the context of underlying, mostly chronic, diseases like cystic fibrosis, chronic renal failure, chronic liver diseases, childhood malignancies, congenital heart disease, and neuromuscular diseases ,5.Although there may be a lack of consensus on the use of terminology and definition, there is agreement that acute malnutrition should be diagnosed using anthropometrics only  5,7].,7.5,7].The aim of this review is to describe the pathophysiology and main clinical aspects of acute malnutrition in childhood, and to provide an overview of the current recommendations on management based on acute malnutrition type, cause and severity.Acute malnutrition is responsible for almost one third of all deaths in children <5 years of age and causes intellectual or cognitive impairment among those who survive [In studies using various methods of assessing malnutrition, the prevalence of acute malnutrition among hospitalized children in developed countries ranged from 6 to 51% ,10,11,12Inadequate energy intake leads to various physiologic adaptations, including growth restriction, loss of fat, muscle, and visceral mass, reduced basal metabolic rate, and reduced total energy expenditure ,5,6. TheOrgan systems are variably impaired in acute malnutrition ,15. CellVillous atrophy with resultant loss of disaccharidases, crypt hypoplasia, and altered intestinal permeability results in malabsorption. Other common aspects are bacterial overgrowth and pancreatic atrophy resulting in fat malabsorption; fatty infiltration of the liver is also common . Drug meCardiac myofibrils are thinned with impaired contractility. Cardiac output is reduced proportionate to weight loss. Bradycardia and hypotension are also common in severe cases ,16. The Acute malnutrition has been recognized as causing reduction in the numbers of neurons, synapses, dendritic arborizations, and myelinations, all of which resulting in decreased brain size . The cerAcute malnutrition pertains to a group of linked disorders that includes kwashiorkor, marasmus, and intermediate states of marasmic kwashiorkor. They are distinguished based on clinical findings, with the primary distinction between kwashiorkor and marasmus being the presence of edema in kwashiorkor .The term “marasmus” is inferred from the Greek word “marasmus”, correlating to wasting or withering. Marasmus is the most frequent syndrome of acute malnutrition . It is dThe term “kwashiorkor” derives from the Kwa language of Ghana and its meaning is equivalent to “the sickness of the weaning” . Cicely Marasmic kwashiorkor is represented by mixed features of both marasmus and kwashiorkor. Characteristically, children with marasmic kwashiorkor have concurrent gross wasting and edema. They usually have mild cutaneous and hair manifestations and an enlarged palpable fatty liver.An adequate nutritional assessment includes detailed dietary history, physical examination, anthropometric measurements  using appropriate reference standards, such as the WHO standard growth charts , and basQuestions regarding mealtimes, food intake and difficulties while eating should be part of routine history taking and give a rapid qualitative impression of nutritional intake. For a more quantitative assessment, a detailed dietary history must be taken by recording a food diary or (less commonly) a weighed food intake. This would usually be performed in association with an expert dietician. When considering whether intakes are enough, dietary reference values provide estimates of the range of energy and nutrient requirements in groups of individuals .Accurate measurement and charting of weight and height  is essential if malnutrition is to be identified. Clinical examination without plotting anthropometric measurements on growth charts has been shown to be very inaccurate . For preLaboratory investigations may be helpful in the diagnosis of primary acute malnutrition and are essential in driving treatment in secondary malnutrition . BecauseThe validity of individual anthropometric measurements may vary based on the population of children. Hence, a combination of measurements obtained by a trained individual in combination with other clinical parameters should guide assessment of nutritional status . Serial Management strategies for acute malnutrition depend on the type of malnutrition, identification of its cause, and its severity .In primary moderate acute malnutrition, management at home is recommended, including counseling of parents, with emphasis on continuing breastfeeding and appropriate complementary feeding (nutrition-specific interventions). Ideally, these children should receive 25 kcal/kg per day of energy in excess of what their healthy peers receive, and their diets should contain animal-source foods rich in essential fatty acids and micronutrients including vitamin A, iron and zinc ,32.Children with severe acute malnutrition without any complications can be managed in the community with ready-to-use therapeutic food  . ChildreThe stabilization phase of treatment for complications includes : (i) treThe refeeding syndrome is due to the sudden availability of glucose, leading to inhibition of gluconeogenesis and an insulin surge. This causes rapid intracellular influx of potassium, magnesium, and phosphate and therefore low serum levels and poor myocardial contractility ,34. ThisChronic malnutrition needs nutrition-sensitive interventions scaled up at the national or regional level, including ensuring household food security, safe water, proper hygiene, female education, creating proper livelihoods, social protection schemes, etc. ,35. GrowFor the management of secondary acute malnutrition, it is crucial to identify the underlying disease by history taking, examination and laboratory investigations . ExclusiThe principles of management of severe malnutrition resulting from the underlying diseases mentioned above are similar to those for primary severe acute malnutrition.In humanitarian crises, supplementary feeding is considered the main strategy for preventing and treating moderate acute malnutrition. According to the Sphere guidelines , two typBlanket supplementary feeding programs are recommended where food safety is low and there is a need to expand interventions beyond only moderate acute malnutrition cases. They should be accompanied by general food distribution targeting affected households. Defined impact indicators for blanket supplementary feeding programs do not exist, but it is important to check coverage, adherence, acceptability and rations provided.The indicators for managing moderate acute malnutrition mainly refer to targeted supplementary feeding. The main aim of a targeted supplementary feeding program is to prevent the moderately malnourished becoming severely malnourished and to rehabilitate them. These types of program usually provide a food supplement to the general ration for moderately malnourished individuals, for pregnant and nursing mothers, and other at-risk individuals.In conclusion, acute malnutrition is a nutritional deficiency resulting from either inadequate energy or protein intake, with variable clinical presentation. Clinical examination and careful measurement of growth status and reference to standard growth charts is essential in order to identify children with acute malnutrition. Most children with primary acute malnutrition can be managed at home, while those with severe acute malnutrition and complications require treatment in a hospital; those without complications can be treated at home with ready-to-use therapeutic food. The management of secondary malnutrition is mainly based on treating the underlying cause .


These isoforms are highly overexpressed in many types of cancer, where they contribute to the acidification of the tumor environment, promoting tumor cell invasion and metastasis. In this work, in order to identify novel dual hCA IX and XII inhibitors, virtual screening techniques and biological assays were combined. A structure-based virtual screening towards hCA IX and XII was performed using a database of approximately 26,000 natural compounds. The best shared hits were submitted to a thermodynamic analysis and three promising best hits were identified and evaluated in terms of their hCA IX and XII inhibitor activity. In vitro biological assays were in line with the theoretical studies and revealed that syringin, lithospermic acid, and (-)-dehydrodiconiferyl alcohol behave as good hCA IX and hCA XII dual inhibitors.The tumor-associated isoenzymes  These enzymes are grouped into eight distinct and not related genetic families  and are a typical example of biological convergent evolution directed toward the catalysis of essential biochemical processes [hCAs) belong to the α-class and exist in 15 different isoforms, which differ for tissue distribution, subcellular localization, and catalytic activity. In particular, hCA I-III, hCA VII, and hCA XIII are cytosolic isozymes; hCA VA and VB are located in the mitochondria; and hCA IV, hCA IX, hCAXII, and hCA XIV are membrane-bound isozymes [The Carbonic Anhydrases CAs, EC 4.2.1.1) are metalloenzymes that reversibly catalyze the hydration of carbon dioxide  [hCAs inhibitors, special attention should be given to their selective profile. In fact, several drug interactions and side effects have been linked to non-selective hCAs inhibition, thus, a selective profile over the appropriate isoforms is mandatory [hCA IX and hCA XII inhibitors could be considered as potential anticancer drugs [Nowadays, cancer is ranked as the second cause of death worldwide, therefore, the development of new anticancer therapies is an urgent need ,23. In prodrugs) ,25,26, prodrugs) ,28. Furtrodrugs) ,30,31,32andatory ,33. Undeer drugs .hit compounds. The in silico studies are able to speed up the identification of bioactive compounds, thus, reducing the cost and time of research-associated activities [hCA IX and hCA XII inhibitors, we performed a SBVS using a database of natural occurring compounds. The FooDB database was used to build up a chemical library of natural occurring compounds virtually screened towards several hCAs. The most promising compounds, based on their theoretical binding energy, were further submitted to in vitro assays to point out new promising hCA IX and hCA XII inhibitors. The experimental results confirmed the computational predictions, providing the rationale behind the ligands biological activity and selectivity.In the drug discovery process, the application of computational methods has demonstrated to be important for the identification of new tivities . In parttivities ,39,40,41tivities ,43. FooDtivities  is the mStarting from the three dimensional structure of the human Carbonic Anhydrase IX in complex with 5-thiophene-2-sulfonamide and the crystal structure of the human Carbonic Anhydrase isozyme XII with 2,3,5,6-tetrafluoro-4-(propylthio)benzenesulfonamide, deposited in the Protein Data Bank (PDB) with the PDB codes 5FL4  and 5MSAhCA IX and XII catalytic binding site, equal to 0.545 and 1.059 Å for hCA IX and hCA XII, respectively.With the aim to evaluate the reliability of our molecular recognition approach, we performed redocking calculations by using the Glide Standard Protocol (SP) algorithm, which was able to reproduce the experimentally determined binding modes, obtaining Root Mean Square Deviation (RMSD) values, calculated between the best docking pose and the ligand co-crystallized into the The receptor structures were energy optimized by means of the Protein Preparation Wizard tool implemented in Maestro, using OLPS_2005 as the force field. Residual crystallographic buffer components and water molecules were removed, hydrogen atoms were added, missing side chains were built using the Prime module, and side chains protonation states at pH 7.4 were assigned ,48.For the Virtual Screening (VS) studies, the FooDB database, containing 26,680 compounds, was used . The libhCA isoforms IX (2–5) and XII (1–5) inhibitors  [hCA isoforms IX and XII inhibitors, respectively, were used to further filter the obtained hits.With the purpose to select the scored compounds according to their Glide score (G-score) value, we performed molecular docking studies of already approved and investigational hibitors , such ashibitors . Startinl v10.8) ,52 and thCA isoforms IX and hCA XII were investigated by visual inspection. Then, considering their commercial availability, we purchased the 3 best hits, which were subjected to in vitro assays.Finally, the shared compounds between both 2 hydration activity of the compounds reported was assayed by means of the Applied Photophysics stopped-flow instrument [2SO4 was added for keeping constant the ionic strength, to follow the initial rates of the enzyme-catalyzed CO2 hydration reaction for a period of 10–100 s. The CO2 concentrations for the determination of the kinetic parameters and inhibition constants ranged from 1.7 to 17 mM. For each potential inhibitor, at least six traces of the initial 5–10% of the reaction have been used for determining the initial rate.The CA-catalyzed COstrument . Phenol IS) were obtained by the nonlinear least squares methods using PRISM 3 and the Cheng−Prusoff equation, as reported earlier [in-house [The uncatalyzed rates were determined in the same experimental manner and subtracted from the total observed values. Stock solutions of the potential inhibitor (0.1 mM) were prepared by using distilled deionized water, and the dilutions with the assay buffer were prepared up to 0.01 nM. The solutions containing the inhibitor and the enzyme were preincubated for 30 min at r.t. prior to performing the assay, in order to form the E−I adduct. The inhibition constants  at 10 µM concentration. After 24, 48, and 72 h, 20 µL of 3--2,5 diphenyl tetrazolium  was added on the plate at the final concentration of 5mg/mL, and incubated for two more hours [The human colorectal cancer cells Caco-2 were maintained in Dulbecco’s Modified Eagle Medium  integrated with 10% of FBS  and 1% P/S . Cells’ culture conditions were 5% COre hours . CulturehCA IX and XII isoforms. Starting from the active set   . Therefohits (6–14) were selected  could be purchased and submitted for biological assays. Among them, compound 12 was found to have a good inhibitory activity on hCA XII [hCAs involved in tumors pathogenesis.Finally, after a careful visual inspection analysis of the best poses, nine selected . Structuisoforms . Unfortu hCA XII , thus, wAll the VS steps are summarized in 6, 12, and 14 were evaluated in vitro for their inhibition potencyagainst the most relevant hCAs such as the cytosolic and widely expressed hCAs, I and II, and the tumor-associated hCAs, IX and XII. The obtained data reported below in hCA inhibitor acetazolamide (AAZ).The selected compounds hCAs I and II with KIs >100 µM. Interesting results were obtained for the remaining isoforms:(i)14, the data reported in hCA XII isoform was 3.5-fold more potently inhibited when compared to the hCA IX  with a selectivity index  of 3.5.As for compound (ii)6, although enhanced enzymatic SIs were observed. The tumor-associated hCA XII resulted inhibited 27.0-fold more potently when compared to the IX .The same kinetic profile was also recovered for (iii)12 was already investigated as a potential inhibitor hCA on the widely expressed hCAs, I and II, and on the tumor-associated IX and XII from some authors of this manuscript and the data reported in hCA XII selective inhibition was obtained from such a substance, thus being the most potent and selective against such a tumor-associated isoform .Compound Overall, the compounds resulted ineffective inhibitors against the widely expressed 6, 12, and 14 being effective inhibitors of the tumor-associated hCA IX and XII and in particular, the latter was preferentially inhibited with KI values in the low micromolar range. Among the substances tested, compound 12 was a particularly potent inhibitor of the hCA XII, having a KI value close to the reference drug of the sulfonamide type AAZ .The kinetic data reported here clearly showed compounds hits were well recognized in the catalytic binding site of both hCA IX and XII, as shown in hCAs by means of hydrogen bond (H-bond), salt bridges, electrostatic, and cation interactions. Moreover, the better affinity of all three hits towards hCA XII could be rationalized by the higher number of good contacts and H-bonds with respect to the hCA IX isoform .6) to approximatively 50% of cell viability (12 and 14)  .Considering the aforementioned results, those compounds can have a potential anticancer activity, thus, encouraging their further development.hits. Such a scaffold is known to be incorporated in several antioxidant compounds and, more recently, was found to have a key role in inhibiting hCA I and II [The molecular modeling studies presented evidence of the presence of the phenolic moiety in all best I and II .6, 12, and 14 are food constituents, thus, confirming the strong importance of natural components as a source of lead and bioactive compounds, able to affect several biological processes in our organism. In fact, many studies in the literature have demonstrated that natural products with anti-inflammatory, antimicrobial, and antioxidant activity could be useful for multifactorial diseases [Compounds diseases ,43.hits, compound 6 is known as (-)-dehydrodiconiferyl alcohol, and belongs to the class of organic compounds known as 2-arylbenzofuran flavonoids. It has been detected, but not quantified, in coffee and coffee products and green vegetables. In the literature, it was found to exert anti-Helicobacter pylori [Among the three r pylori , antiadir pylori , and antr pylori . Recentlr pylori . Moreover pylori .6, the lithospermic acid 12 is a member of the class of 2-arylbenzofuran flavonoids and it is also known as lithospermate. Our approach highlighted it as the most interesting dual inhibitor of both hCAs, in line with a previous work related to its hCA XII [As with compound  hCA XII  proven a hCA XII ,65.14 is syringin, also known as eleutheroside b or β-terpineol. It belongs to phenolic glycosides, which are organic compounds containing a phenolic structure attached to a glycosyl moiety. Syringin was first isolated from the bark of lilac (Syringa vulgaris) by Meillet in 1841. Moreover, it can be found in caraway, fennel, and lemon, which makes syringin a potential biomarker for the consumption of these food products. Several pharmacological actions of syringin include plasma glucose reduction, antioxidation, anticancer activity, antidepressant effect, and immunomodulation [Finally, compound dulation ,67.In vitro results demonstrated that all the three hits are able to inhibit both hCA IX and XII isoforms in the micromolar range, although with a preference towards the XII target, thus, confirming their potential anticancer activity.hCA IX and XII isoforms. In particular, we proposed (-)-dehydrodiconiferyl alcohol, lithospermic acid, and syringin as good hCA IX and hCA XII dual inhibitors with a potential anticancer activity, thus, encouraging their further development. Specifically, lithospermic acid and syringin, with KI values in the low micromolar range, showed a better cytotoxic effect seventy-two hours after treatment than (-)-dehydrodiconiferyl alcohol. Interestingly, all the best hits are characterized by a phenol moiety, which has recently aroused considerable interest due to its potential antioxidant effects on human health.In this work, in silico and experimental techniques were combined to identify natural bioactive compounds present in food, such as coffee, green vegetables, common thyme, peppermint, caraway, fennel, and lemon, and are endowed with inhibition properties against both


We present an object labelled dataset called SFU-HW-Objects-v1, which contains object labels for a set of raw video sequences. The dataset can be useful for the cases where both object detection accuracy and video coding efficiency need to be evaluated on the same dataset. Object ground-truths for 18 of the High Efficiency Video Coding (HEVC) v1 Common Test Conditions (CTC) sequences have been labelled. The object categories used for the labeling are based on the Common Objects in Context (COCO) labels. A total of 21 object classes are found in test sequences, out of the 80 original COCO label classes. Brief descriptions of the labeling process and the structure of the dataset are presented. It is one of the datasets used in the MPEG-VCM (Video Coding for Machines) standardization group.•The dataset can be used to study the impact of video compression on object detection, or for developing and analyzing systems that perform video compression and object detection simultaneously, as in MPEG-VCM.1SFU-HW-Objects-v1, which contains bounding boxes and object class labels for High Efficiency Video Coding (HEVC) v1 Common Test Conditions (CTC) video sequences We present a dataset called Object labels are organized into separate folders, one for each sequence. Data file structure is shown in The remaining four numbers in the row specify the bounding box of the corresponding object. Of these four numbers, the first two specify the center coordinates of the corresponding bounding box, relative to the top-left corner of the video frame, normalized by the resolution, and the last two elements are the width and height of the bounding box, again normalized by the resolution. An example showing how these are computed in presented in 2Raw HEVC v1 CTC video sequences in the YUV420 format are maintained by the Joint Collaborative Team on Video Coding (JCT-VC).The final labels and bounding box coordinates were saved to a file corresponding to the current frame. As mentioned before, the annotation file is a text file whose name contains the sequence name, resolution, frame rate, and frame index.The work did not involve any human or animal subjects, nor data from social media platforms.Hyomin Choi: Conceptualization, Methodology, Software, Data curation, Writing - original draft. Elahe Hosseini: Methodology, Data curation, Writing - original draft. Saeed Ranjbar Alvar: Conceptualization, Investigation, Validation. Robert A. Cohen: Supervision, Writing - review & editing. Ivan V. Bajić: Supervision, Writing - review & editing, Project administration, Funding acquisition.The authors declare that they have no known competing financial interests or personal relationships which have or could be perceived to have influenced the work reported in this article.


Studies have been carried out on the association between frailty and malnutrition, but the similarities and divergence of the relationship remain debatable. This study aimed to explore the prevalence of malnutrition risk and frailty as well as the overlapping constructs. The associations that emerged were assessed independently of other risk factors. A total of 301 community-dwelling older adults with a mean age of 66.91 ± 5.59 years old were randomly recruited. Fried Criteria and Mini Nutritional Assessment-Short Form (MNA-SF) were used to assess frailty status and malnutrition, respectively. Other related nutritional assessments were assessed (body mass index (BMI), circumference measures, body fat % and skeletal muscle mass). The prevalence of frailty was 14.6% and prefrail was 59.7%; 29.6% were at risk of malnutrition, and 3.3% were malnourished. Malnutrition risk was significantly associated with a higher number of chronic diseases, BMI, circumference of mid-upper arm (MUAC), and calf, (CC)and skeletal muscle mass (SMM) and frailty, whereas frailty was significantly associated with higher number of chronic diseases, SMM and malnutrition. Frailty syndrome can be predicted with increasing age, body fat, lower skeletal muscle and malnutrition. Those who were frail were found to be five times more likely to be at risk of malnutrition. Results suggested that frailty and malnutrition shared considerable overlap, which emphasised the interrelated but discrete concepts. Therefore, the assessment of malnutrition is imperative and could be used as a practical implication in assessing frailty syndrome.  Frailty is a multidimensional syndrome that has been a crucial aspect of research within the geriatric field. It is generally considered synonymous with disability, comorbidity and other syndromes under the giant umbrella of the geriatric syndrome. From a clinical interpretation, frailty state is critical in instigating severe complications which can lead individuals to a higher risk of adverse health outcomes, such as falls, regress in mobility and independence, hospitalisation, disability and death . The comThe relationship between malnutrition and frailty has been widely discussed in the literature ,3, whichCurrent evidence hypothesised that frailty and malnutrition are two concepts that share common determinants and several pathophysiological pathways. Some of these included reduced body tissue, presence of chronic inflammation, pernicious sociodemographic background, impaired physical and cognitive functions as well as dependence in bodily functions ,7. HenceThe relationship between frailty and malnutrition were widely explored using a well-validated instrument, the mini nutritional assessment (MNA), regardless of the simple adaptation, or the original version. The assessment was validated and well accepted in screening malnutrition risks among older adults. In general, two out of three malnourished older adults were physically frail, while only about one in 10 of the population that was physically frail was malnourished . A systeLoss of skeletal muscle mass and skeletal muscle wasting would be evident in cases of malnutrition and frailty when the muscle circumference in the arms and calves is noticeably reduced. Previous studies suggested that reduced muscle mass was a definitive feature in loss of manual pressure strength ,11,12. NThe purpose of this study was to compare the differences in demography, body composition, nutritional status and malnutrition among community-dwelling older adults residing in the capital city of Kuala Lumpur. Prevalence of malnutrition risk and frailty and overlapping constructs were explored to assess the association of other risk factors independently.This cross-sectional study was conducted on older adults aged 60 years and above who were living in 10 randomly selected People’s Housing Projects (PPR) flats at Kuala Lumpur, the capital city of Malaysia. A cluster sampling method was applied, whereby all blocks from each chosen flats were included in the study. Census data on the residents in each apartment complex were obtained from the City Hall of Kuala Lumpur (DBKL), and the number of subjects recruited was based on proportionate sampling. Subjects were screened for inclusion and exclusion criteria selection before participation. The inclusion criteria included older adults aged 60 and above who resided in flats and were able to ambulate. Subjects were excluded if they showed a severe sensory deficit which hindered them from performing a physical functioning assessment, had an unstable medical condition or were in palliative care. Three-hundred-and-ten (310) older adults who were eligible for this study were recruited, but nine (9) were excluded due to missing data. Approval and permission in conducting the study were granted by the Ethical Research Committee, which involved the Human Research Department at Universiti Putra Malaysia and Community Development and Urban Wellbeing Department of Kuala Lumpur City Hall (DBKL). The consent form was then disseminated to the eligible subjects and signed for confirmation of their participation. The interview-based survey was conducted by trained researchers at the community centre of each apartment complex. The subjects who agreed to participate were invited to the interview sessions and reminded a day before the allocated sessions to be present. Upon inviting respondents to participate in the study, information sheet and consent forms were distributed. Further details on personal information and contact number were obtained from respondents who agreed to participate. Two days prior to data collection, respondents were reminded through phone calls to fast overnight for a minimum of eight to 12 hours. In the event whereby respondents failed to fulfil the requirement to fast overnight, a subsequent visit attempt was made where they were reminded to fast overnight a day before. Sociodemographic data included age, gender, race, education, marital status, living arrangement, working status and monthly household income. The self-report of chronic diseases diagnosed and treated by a physician was recorded for ten (10) identified diagnoses and other disorders.Frailty syndrome was indicated based on the well-established, standardised phenotype of frailty, proposed by Fried et al. (2001). The phenotypes evaluated five components of frailty syndrome, which denoted one point for each criterion met. The first phenotype on (1) shrinking was based on the subjective report of unintentional weight loss of more than 5 kg for the past six to 12 months. The second phenotype focused on (2) exhaustion, which was assessed based on two questions of self-reporting on fatigue from the CES-D scale of depression that was proposed in the original method. (3) Weakness was identified if the handgrip strength was lower than the cut-off point proposed by Asian Working Group of Sarcopenia (AWGS). Handgrip strength was measured using the Jamar Hydraulic Hand Dyanmometer. In a sitting position, respondents were asked to hold the dynamometer vertically using their dominant hand. They were required to compress the handgrip to their maximum strength for three attempts. (4) Slowness was measured by the gait speed of walking four meters at the usual speed and those who took less than the cut-off point at 0.8 m/s is considered slow. Finally, (5) low physical activity was identified by the low score of the lowest tertile in the physical activity scale for the elderly (PASE). The Malay version of PASE was used instead of the components mentioned in the original Fried Criteria , which wAnthropometric measurements that were included in this study were weight, height and body mass index (BMI). The circumferences of the crucial body parts were also considered in the study, including waist circumference (WC), mid-upper arm circumference (MUAC), and calf circumference (CC). The invasive method using bioelectrical impedance analysis  was used to assess the body composition parameters, namely body fat percentage, visceral fat and skeletal muscle mass. All respondents fasted for at least eight to 12 hours prior to the administration as the assessment of body composition using BIA is highly dependent on hydration status. Respondents were required to void the bladder and to avoid any vigorous activity beforehand. Participants stood bare-footed on the footplate electrodes on the analyser holding the handgrip electrodes with both hands. This device applies single-frequency  with eight sensors on both hands and feet for complete body measurement. The machine is calibrated daily and the measurements were taken twice in accordance with the standard procedure. The mini nutritional assessment (MNA) was employed to identify older adults who were at risk of malnutrition. Both MNA and MNA-SF (MNA-short form) are validated and effective tools for malnutrition screening . Other aSince the original version of MNA that was used was found to be lengthy, which would be impractical for the implementation of this study and less effective in a community setting, a shorter version was used. This shorter version was also always preferred among scholars . Particun) and percentage (%) in the cross-tabulation. Subsequently, the chi-square test of association and the Pearson product moment correlation was used to compare the difference of covariates between subgroups of frailty status  and nutritional status . In cases where the variables were not normally distributed, a nonparametric test of Spearman correlation was used instead. Additionally, the chi-squared test was used to observe the frequency differences between the categorical covariates with the observed variable. To determine the correlation between various risk factors with the risk of frailty syndrome, variables which were found to be significant in the bivariate analysis were further included in the regression analysis. The multivariate logistic analysis was used to determine the significant predictors for frailty syndrome. Data with a p-value less than or equal to 0.05 were considered to be statistically significant.Data were analysed using Statistical Package for Statistical Package for Social Science (SPSS) (Version 22) All variables were assessed for homogeneity of variance before any statistical analysis was attempted. Exploratory data analysis (EDA) was used to check for data normality of all variables. In the descriptive analysis, continuous variables were expressed in mean and standard deviation (SD), while categorical variables were presented as frequency  based on the mean monthly income of these subjects, which was at RM959.96 ± 817.10 (approximately USD231.65).p < 0.001). However, there was no significant difference in frailty status observed across the ethnicities. The mean monthly income among subjects who were frail was found to be significantly lower when compared to their counterparts (p < 0.001), suggesting that frail syndrome was greater with reduced financial income. Subjects with frailty status were found to have at least one chronic disease (p < 0.05), greater BMI and waist circumference. In addition, a higher percentage of body fat, visceral fat, lower skeletal muscle mass and index (p < 0.001) and at risk of malnutrition (p < 0.01) were also prevalent among frail individuals. The baseline characteristics of the subjects according to their frailty status are presented in p < 0.05), presence of weight loss (p < 0.05) and immobility (p < 0.01) were significantly dominant among frail individuals. No significant differences were observed for psychological stress and neuropsychological problems for subjects with frailty status. Further observation of the MNA indicators with the frailty status revealed that signs of reduced food intake (p < 0.05). No significant difference was found between malnutrition across gender and ethnicity. However, those who were at risk of malnutrition earned less monthly income, though the result was proven to be insignificant (p > 0.05). Based on results for circumferences of body parts, both MUAC and CC were found to be significantly low compared to well-nourished individuals (p < 0.05). Skeletal muscle mass and index both showed a significantly lower mean among subjects identified as at risk of malnutrition. The malnutrition status of the subjects was measured using MNA-SF and is presented in p < 0.05). The prevalence of shrinking and exhaustion was significantly higher among subjects who were at risk of malnutrition. Nonetheless, there were no significant differences found in phenotypes on weakness, slowness and physical inactivity with malnutrition.Further exploration on the phenotypes of frailty with malnutrition risk showed that there were significant differences observed for shrinking and exhaustion phenotypes with malnutrition status , for which those who were frail were five times more likely, and those who were prefrail were twice as likely to be at risk of malnutrition. Results from the multivariate analysis in relation to malnutrition risk  showed tp < 0.05). Similarly, those who are at risk of malnutrition have a higher number of chronic diseases, lower body weight and BMI, a lower circumference of arm and calf as well as lower skeletal muscle mass and frailty (p < 0.05). Despite the extensive literature in this field of research, this study is currently the only study that explores the relationship between malnutrition and frailty among community-dwelling older adults in Malaysia. Therefore, this study provides new insights into the nutritional status of the frail community-dwellers in Malaysia. Results show that those who are physically frail are found to be of higher mean age, have lower household income and SMI and are at risk of malnutrition (p < 0.01), which has been deliberately reported previously [Hence, this present study demonstrates that there is a strong association between malnutrition and frailty. Among those who are frail, almost half of them are found to be at risk of malnutrition. The proportion of individuals suffering from poor nutritional status increased gradually with the growing level of frailty . Thus, the phenotype of shrinking is found to reflect similar constructs in the component on the presence of weight loss in MNA, suggesting a strong correlation between these two components. This finding reiterates the previous reports that the most apparent overlap is the mutual assessment of weight loss [In this study, further explorations of the components in the criteria of frailty and MNA have revealed that both exhibit considerable overlapped constituents. We showed that there was a strong relationship between each of the MNA-SF items and frailty status. Both frailty and malnutrition are significantly related to each other, particularly for the phenotype of shrinking and exhaustion in the frailty criteria with reduced food intake, presence of weight loss and immobility in the components of malnutrition (ght loss ,24. The On the other hand, the phenotype of slowness is partly emulating the component of immobility in MNA. The assessment of immobility is dependent on the gait speed, while in the MNA component, it is dependent on self-report. The frailty criterion of “exhaustion” is evaluated by self-perception of “everything was an effort” or “I could not get going”, which inevitably shows some overlap with the depressive state in the neuropsychological item of MNA . Though The pathophysiology of malnutrition and physical frailty is known to share common pathways and has been described as un-interchangeable . NonetheFrom another perspective, the roles of muscle mass and sarcopenia are vital in understanding malnutrition and frailty better. In this study, there is a significant relationship between the muscle mass with frailty and malnutrition, with reduced skeletal muscle mass when both constructs worsen. The finding from this study is supportive of the proposed vicious cycle of frailty, which highlighted sarcopenia as one of the main contributors of frailty . As an iStudies have included the assessment of muscle function and quality as a linking substrate between malnutrition and physical ability ,31. ReduThis study has identified some limitations. Firstly, the sample population is urban-living older adults, which may not represent the general population when extrapolated. This cross-sectional study design is also unable to signify the cause and effects of the tested variables. Nevertheless, this study is able to determine the frailty incidence and establish the association of malnutrition among the elderly population in a primary health care setting.In conclusion, the use of MNA is practical in the identification of frailty syndrome among older adults in a community care setting. As such, the adaptation of MNA as an indirect tool to screen for frailty syndrome is promising. Given that the prevalence of malnutrition is much higher than frailty, this result strongly supports that malnutrition and physical frailty are interrelated, but possess distinct syndromes. The present study has addressed the relationship between malnutrition and frailty among community-dwelling older adults using past studies within this area of research. Since frailty in older adults is of rising concern, there is a need to prevent and manage frailty by concentrating on the essential factors, such as maintaining good nutritional status and physical function to decrease the escalation of frailty syndrome. The implementation is especially important for primary health care to be amenable in governing this epidemic at an earlier stage. More insights are warranted to account for potential biases that may have been present in this study.


Research on the prospective relationship between older adults’ alcohol consumption and their subsequent risk of dementia and cognitive impairment, no dementia (CIND) has been limited by inconsistent definitions of “moderate” drinking, use of short follow-ups, and an exclusive focus on either amounts of alcohol, or history of drinking problems, as predictors. To overcome these limitations we analyzed a longitudinal, 18-year Health and Retirement Study cohort  to determine how older adults’ baseline membership in one of six drinking categories ; Within-Guideline Drinker, Without and With a HDP; and Outside-Guideline Drinker, Without and With a HDP) predicted dementia and CIND 18 years later. Among participants with No HDP, 12.6% of Non-Drinkers, 5.2% of Within-Guideline Drinkers, and 8.8% of Outside-Guideline Drinkers were classified as having dementia at the 18-year follow-up; among participants With HDP, 14.1% of Non-Drinkers, 8.9 % of Within-Guideline Drinkers, and 6.9% of Outside-Guideline Drinkers were classified with dementia. Being a baseline Within-Guideline Drinker with No HDP reduced the likelihood of dementia 18 years later by 45%, independent of baseline demographic and health characteristics; being a baseline Within-Guideline Drinker With a HDP reduced the likelihood of dementia by only 13% (n.s.). Similar patterns obtained for the effects of baseline drinking group membership on likelihood of CIND at follow-up. These findings suggest that consuming alcohol at levels within validated guidelines for low-risk drinking may protect against dementia and CIND, but only among older adults who have no history of drinking problems.


Mycobacterium tuberculosis and Mycobacterium marinum. It has recently been shown that PDIMs work in concert with the M. tuberculosis Type VII secretion system ESX-1 to permeabilize the phagosomal membranes of infected macrophages. As the zebrafish-M. marinum model of infection has revealed the critical role of PDIM at the host-pathogen interface, we set to determine if PDIMs contributed to phagosomal permeabilization in M. marinum. Using an ΔmmpL7 mutant defective in PDIM transport, we find the PDIM-ESX-1 interaction to be conserved in an M. marinum macrophage infection model. However, we find PDIM and ESX-1 mutants differ in their degree of defect, with the PDIM mutant retaining more membrane damaging activity. Using an in vitro hemolysis assay—a common surrogate for cytolytic activity, we find that PDIM and ESX-1 differ in their contributions: the ESX-1 mutant loses hemolytic activity while PDIM retains it. Our observations confirm the involvement of PDIMs in phagosomal permeabilization in M. marinum infection and suggest that PDIM enhances the membrane disrupting activity of pathogenic mycobacteria and indicates that the role they play in damaging phagosomal and red blood cell membranes may differ.Phthiocerol dimycocerosates (PDIMs) are a class of mycobacterial lipids that promote virulence in  Mycobacterium tuberculosis, the causative agent of tuberculosis (TB), is an obligate human pathogen; its ability to survive and spread is dependent upon its ability to survive within its human host. During the early stages of infection, this requires manipulation of the innate immune system, allowing for survival of the pathogen within the microbicidal environment of host macrophages [M. tuberculosis and the pathogenic M. marinum use their lipid coats and Type VII secretion systems to evade and co-opt ordinarily lethal host defenses within these monocytes.rophages . M. tubeM. marinum and M. tuberculosis. PDIM-deficient mutants in M. marinum and M. tuberculosis are attenuated in animal models of infection [The virulence lipid phthiocerol dimycocerosate (PDIM) is an outer membrane virulence lipid in both nfection –4 and menfection , 6. Contnfection , 8. ConsThe ESX-1 Type VII secretion system was identified as essential for virulence when its loss was determined to be the primary cause of attenuation for the live BCG vaccine strain –12. ESX-M. tuberculosis [M. marinum has provided new insights into PDIM’s role in pathogenesis [M. marinum. In this paper, we confirm that as in M. tuberculosis, proper PDIM localization is required for M. marinum to effectively permeabilize macrophage phagosomes.Incorporation of PDIM into host membranes has been proposed to rigidify them, enhancing lysis by ESX-1 . Supportrculosis –26. As togenesis , 7, we sM. marinum and M. tuberculosis to permeabilize macrophage phagosomes during infection [M. tuberculosis [M. marinum by using an attenuated ΔmmpL7 M. marinum mutant defective in PDIM localization to the mycomembrane [M. marinum. M. marinum is capable of cleaving the dye with its endogenous mycobacterial β-lactamase BlaC, which causes a loss of FRET and an increase in blue fluorescence (450 nm). We used this dye to determine the relative ability of wildtype, ΔESX-1, and ΔmmpL7 M. marinum to permeabilize their phagosomes  , but higher than ΔESX-1. Highlighting its intermediate level of staining, galectin-8 staining was seven times greater in ΔmmpL7 than in ΔESX-1 infection,  . ΔmmpL7 fection, . These rM. tuberculosis and M. marinum both show ESX-1 dependent hemolytic activity [M. marinum ΔESX-1 mutant can be rescued by complementing with the M. tuberculosis ESX-1 locus [M. marinum has been regarded as a correlate of ESX-1 mediated virulence, although recent work suggests that hemolysis can be lost with minimal effects on intramacrophage growth or virulence [M. marinum ΔmmpL7 were defective in hemolysis as well. As expected, ΔESX-1 had reduced hemolytic activity as compared to wildtype. We saw that ΔmmpL7 had significantly more hemolytic activity than ΔESX-1  . M. marinum ΔESX-1 and ΔmmpL7 were generated as described previously [Strains are described in eviously , 14. tdT5 THP-1 cells were seeded on 24-well optical bottom tissue culture plates , treated with 100nM phorbol 12-myrystate-13-acetate (PMA)  for two days. PMA-containing media was then removed and replaced with fresh media. Two days later, adherent cells were washed twice with PBS and infected with antibiotic-free media containing single-cell suspensions of tdTomato-expressing M. marinum at a multiplicity of infection of ~1.5 for 4 hours and maintained at 33°C, 5%CO2. 22 hours later, cells were fixed in 4% (wt/vol) paraformaldehyde in PBS at room temperature for at least 30 minutes.2.5 x 10Galectin-8 staining was based on the method described by Boyle and Randow . Fixed cA Nikon A1R laser scanning confocal microscope fitted with a 20x Plan Apo 0.75 NA objective was used to generate 2048 x 2048 pixel 8μm z-stacks consisting of 1μm optical sections. Image acquisition was carried out using a galvano scanner, 488nm and 561nm lasers, and a GaAsP multi-detector unit. Maximum intensity projections were generated in NIS Elements (Nikon) and used to calculate the percentage of THP-1 cells containing mycobacterial foci labeled with galectin-8 per imaging field.6 THP-1 cells were seeded on 12-well tissue culture plates and treated with 33 nM PMA for three days. Cells were then washed with media and then infected with single-cell suspensions of tdTomato-expressing wildtype, ΔESX-1, or ΔmmpL7 M. marinum at a MOI of 1 for 4 h at 33°C in EM medium. 24 hours post infection cells were stained with Fixable Viability Dye eFluor660 (eBioscience). Cells were then harvested and stained for 1h at room temperature with 8 μM CCF4-AM (Invitrogen) in EM medium supplemented with 2.5 μM probenecid. Finally, cells were fixed overnight at 4°C in 4% (wt/vol) paraformaldehyde. Cells were analyzed in an LSRFortessa II cytometer, using FACSDiva software (BD Biosciences). At least 40,000 events per sample were collected. Data were analyzed using FlowJo (Treestar). Permeabilization percentage among infected monocytes was calculated using a region defined by an increased 450 nm signal  relative to uninfected cells in the same sample. Events in this region were designated “permeabilized”. Percent permeabilization was then calculated as a ratio of events in the permeabilized field over total live and infected cells.CCF-4 assay was conducted as previously described , with mi600 ~1.5, washed twice with PBS, and resuspended to ~30 OD units/mL. 100 μL M. marinum were mixed with 100 μL 1% RBC in a micro-centrifuge tube, centrifuged at 5000 x g for 5 minutes, then incubated at 33°C for 2 hours. Pellets were then resuspended, centrifuged at 5000 x g for 5 minutes and the A405 measured of 100 μL supernatant. PBS treatment was used as a negative control (background lysis) and 0.1% Triton X-100 (Sigma) as a positive control (complete lysis). Hemolysis was calculated as percentage of detergent lysis after subtracting background lysis.Hemolysis assays were conducted as previously described . BrieflyS1 Data(XLSX)Click here for additional data file.


Contactless physiological monitoring can be important for animal health and well-being. The current study investigated whether heart rate in pigs can be extracted automatically from videos without disturbing the pig and showed that this was possible with 4.69 beats per minute in mean absolute error. The study also tested different body regions and found that the abdomen was a better region to measure heart rate from videos compared to the front leg or the neck. However, future studies are needed that include videos with different light conditions, different housing systems and multiple pigs to enable real-time on-farm monitoring of heart rate from videos.3.5). The sensitivity of the algorithm was then tested on the video of a non-anaesthetized resting pig, as an animal in this state has more fluctuations in HR than an anaesthetized pig, while motion artefacts are still minimized due to resting. The GS HR measurements for the resting pig had a mean value of 161.43 bpm and SD of 10.11 bpm. The video-extracted HR showed a performance of 4.69 bpm in MAE, 6.43 bpm in RMSE and 57% in PE3.5. The results showed that HR monitoring using only the green channel of the video signal was better than using three color channels, which reduces computing complexity. By comparing different regions of interest (ROI), the region around the abdomen was found physiologically better than the face and front leg parts. In summary, the developed algorithm based on video data has potential to be used for contactless HR measurement and may be applied on resting pigs for real-time monitoring of their health and welfare status, which is of significant interest for veterinarians and farmers.Heart rate (HR) is a vital bio-signal that is relatively easy to monitor with contact sensors and is related to a living organism’s state of health, stress and well-being. The objective of this study was to develop an algorithm to extract HR (in beats per minute) of an anesthetized and a resting pig from raw video data as a first step towards continuous monitoring of health and welfare of pigs. Data were obtained from two experiments, wherein the pigs were video recorded whilst wearing an electrocardiography (ECG) monitoring system as gold standard (GS). In order to develop the algorithm, this study used a bandpass filter to remove noise. Then, a short-time Fourier transform (STFT) method was tested by evaluating different window sizes and window functions to accurately identify the HR. The resulting algorithm was first tested on videos of an anesthetized pig that maintained a relatively constant HR. The GS HR measurements for the anesthetized pig had a mean value of 71.76 bpm and standard deviation (SD) of 3.57 bpm. The developed algorithm had 2.33 bpm in mean absolute error (MAE), 3.09 bpm in root mean square error (RMSE) and 67% in HR estimation error below 3.5 bpm (PE Cardiac activity variables have been widely used in animal health and animal welfare research. For instance, heart rate (HR) brings valuable information in relation to an animal’s disease status, physiological functioning, psychological stress and in assessing their individual characteristics, e.g., temperament and coping strategies . Farm anIn previous studies, HR of animals was monitored using two types of technologies, namely implantable transmitters and external body-mounted sensors . AlthougPresently, HR extraction based on different video analysis techniques extract the subtle changes (color or motion) caused by the pulsatile activity of the beating heart. Some of the recently developed algorithms are based on blind source separation (BSS), i.e., separation of the source signals from a set of mixed signals without prior information about the source signals ,11, suchRecently, the video-based HR monitoring approach has been explored in animal research including cattle , primateAnother vital aspect of accurate monitoring of HR is the identification of a region of interest (ROI). The subtle blood flow changes caused by the heart beats have to be detected from continuous frames and, thus, an adequate body part of the animal has to be assessed. Regions with large capillaries near the skin surface and low hair covering can be chosen as an ROI. In this sense, domestic pigs present a large area of exposed skin with low hair coverage, and commercially housed pigs spend about 60–80% of the daily life resting ,29, faciInvestigate the combination of bandpass filtering and short-time Fourier transform (STFT) with sliding windows for extracting HR from noisy input data in a continuous fashion;Explore different regions of interests (ROI) from different anatomical parts of the pig’s body to find the most suitable ROI for signal extraction of cardiac activity;Optimize the different heart rate extraction processing steps to minimize the computational complexity of the algorithm for implementation of real-time monitoring applications.The main goal of the present work was to develop a video-based HR monitoring algorithm for pigs with minimum computational burden for real-time applications. The specific objectives were to:The datasets used in the analysis were obtained from two different experimental set-ups. In the first experiment, video recordings of one anaesthetized pig was made and its data used to develop the algorithm, as the pig had very limited movement (breathing) with no motion artefacts. In the second experiment, a non-anaesthetized resting pig was video recorded. The pig from the second experiment presented larger variations in HR, and its data were used to test the developed algorithm. All the raw videos were stored on an exchangeable, external 4 TB hard drive . The authors are aware of the limited number of animals and variation included in the data and what limitations this has for the interpretation of the developed algorithms. However, this was evaluated as acceptable, as the study presents the initial work on a new method for HR monitoring.The first dataset included one anaesthetized two-year-old Göttinger Minipig a, weighiTM  experimental pen with enough space for an individual animal (1.12 m2) under ambient light. The animal wore a wearable sensor (Zephyr BioHarness-3) in order to measure its movement with a 3D accelerometer  and its HR , which was used as the GS. Video recordings of the pig were made during the whole experiment using a Sony HandyCam HDR-SR5 camcorder. The camera was positioned on a Manfrotto Autopole at a height of 2.5 and 2 m from the center of the pen. The resolution of the video was 1440 × 1080 pixels, and the frame rate was 30 fps. One period of the video of about 450 s was chosen and used to test the algorithm, as the pig during this period was constantly resting with minimal movements. The second dataset included one resting, individually housed Large White × York pig weighing 20 kg b. The exThe different steps taken in the HR extraction algorithm are represented in The first challenge in developing the algorithm was to identify the ROI for pigs. Subtle changes in the video signal are used to extract heart rate. Thus, in human applications, places with large capillaries near the skin surface and low hair covering are frequently chosen as the ROI . The skiThe finite impulse response (FIR) bandpass filter has been widely used in suppressing frequency components outside the HR bandwidth ,25. CompAlgorithms 1 Computation details of HR extractionInput: single colour signal Output: Estimated heart rate hr1:2:3:For i in  do4:5:end for6:For k in  do7:8:9:10:11:12:end forThe short-time Fourier transform (STFT) is a variant of the classic Fourier transform, where a window function is convolved with the original signal to only transform a short part of the signal into the frequency domain. Hence, a simultaneous representation in time and frequency can be achieved. The length and shape of the window function can control the time and frequency resolution. Due to the uncertainty principle, there is always a trade-off between time and frequency resolution. If the window size is too large, almost all the frequency information will be captured, which is similar with using fast Fourier transform (FFT) over a long window. In this case, the time resolution would be lost and vice versa for small windows. In order to find the optimum setting, we tested different window sizes and window functions. Specifically, the signal was first split into pieces by the windows. Then, a window function was adopted to time the data points in every piece. After that, Fourier transformation was performed in each piece, and the amplitude of each frequency was computed. Finally, the HR was computed based on the amplitude and the predefined frequency range (based on the range of the GS HR). The frequency that holds the highest amplitude is considered as the HR frequency. Note that the overlap between two continuous windows was set to three quarters of the window, and f1 and f2 were 0.6 and 6 Hz, respectively. The computing details were as follows:3.5) beats per minute (bpm) were used to make the comparison with the GS HR. Assume that the estimated and reference HR are 3.5 are given by:N is the total number of windows, and i is the i-th window. To investigate the quality of HR estimation, mean absolute error (MAE), root-mean-square error (RMSE) and percentage of HR error below 3.5  were also tested. By comparing the MAE, RMSE and PE3.5 from different window sizes in 3.5 of this window was not the highest. This might result from some extreme large or small HR estimations when using window size 68.27 s. Thus, considering the highest PE3.5 and relatively low MAE and RMSE, the best window size for the anesthetized pig’s HR extraction was 34.13 s. Further, by comparing the MAE, RMSE and PE3.5 from different window functions of window size of 34.13 and 68.27 s in the G channel, hamming and hanning worked better than rect and blackman, and there was not much difference between hamming and hanning. If we compare the MAE, RMSE and PE3.5 from window size 8.53 and 17.07 s, we can see that rect performed better than hamming and hanning; this may be caused by the small data sample size. More experiments should be conducted in future to further investigate this matter. For further analysis, only window function hamming will be used.hamming). 3.5 as well as computation time regarding the comparison of the different methods. Note that the computation time included the reading of the video and all the processing time. It can be concluded that in the current study, the result obtained by the single G channel is better than for the other methods. Regarding the extraction accuracy and computing complexity of using the single G channel, the current study made a comparison with three-channel-based methods that includes combining three channels in different proportions of GRD, aGRD, CHROM and POS ,13,35,36hamming window based on the results described above.The resting pig has larger variations in HR than the anaesthetized pig, which was used to test the sensitivity of the algorithm, using only the G channel, the 34.13 and 68.27 s window sizes and 3.5 of face, front leg and abdomen in 3.5 of pigs are more or less the same as for primates, but worse than for humans. We know that animals have more uncontrollable factors than humans, and in this case, the monitoring of HR in pigs on a non-moving subject can be considered acceptable. This study continuously monitored the HR of a pig by using the single green channel of a video signal. In order to evaluate the monitoring effectiveness on pigs, the current study made a comparison with other species, and the results can be found in 3.5 , it can be seen that the proposed algorithm was reliable in the currently used setting. Compared to previous studies using three channels, this work decreased the computing complexity by using a single channel. Additionally, from the comparison with other methods, we can conclude that using a single channel also presents the advantage of enhancing accuracy to extract HR from the video of pigs. These advantages may also have the potential to help in the monitoring of other physiological parameters, e.g., respiration rate [In the current study, the videos of an anaesthetized and a resting pig were used to develop and to test an algorithm to monitor the HR of pigs. From the evaluation parameters of MAE, RMSE and PEion rate ,38. The 3.5, values of 2.33, 3.09 and 67% were obtained from the video of an anesthetized pig and values of 4.69, 6.43 and 57% from a resting pig. The skin area of the abdomen proved to be the most sensitive body region of the three tested for monitoring the HR of the resting pig. The monitoring results obtained from the single green channel presented higher accuracy and needed less computation time than other methods, including combing three color channels in different proportions  and ICA. The experimental results indicate that the developed algorithm based on RGB video analysis was capable of monitoring the HR of pigs under the used conditions. It has the potential to be used for contactless heart rate measurement and may be applied on resting pigs for real-time monitoring of their health and welfare status, which is of significant interest for veterinarians and farmers. A FIR band pass filter combined with short-time Fourier transform (STFT) based on a single green channel signal allowed us to successfully monitor the HR of pigs in a contactless way from video. Analyzing MAE, RMSE and PE