Automatic surgical workflow analysis (SWA) plays an important role in the modelling of surgical processes. Current automatic approaches for SWA use videos (with accuracies varying from 0.8 and 0.9), but they do not incorporate speech (inherently linked to the ongoing cognitive process). The approach followed in this study uses both video and speech to classify the phases of laparoscopic cholecystectomy, based on neural networks and machine learning. The automatic application implemented in this study uses this information to calculate the total time spent in surgery, the time spent in each phase, the number of occurrences, the minimal, maximal and average time whenever there is more than one occurrence, the timeline of the surgery and the transition probability between phases. This information can be used as an assessment method for surgical procedural skills. Minimally invasive surgery (MIS) has evolved as the gold standard for highly accurate, sensitive and less invasive surgical procedures . These pAdvanced performance in MIS depends on the combination of various skill sets ,5, for wResidents and educators alike have been particularly concerned about advanced cognitive skills’ acquisition and have been looking at strategies to combat this lack of experience . AdvanceOne of the most extended strategies for the training and assessment of advanced cognitive skills  is the analysis of surgical processes targeted at the automatic recognition of surgical phases. Surgical processes are defined as “a set of one or more linked procedures or activities that collectively realise a surgical objective within the context of an organisational structure” . The keySurgical processes can be modelled at any level . These levels are known as granularity levels. A granularity level is defined as the level of abstraction at which the surgical procedure is described . MacKenzThe highest level is the procedure itself, which is composed by a list of phases (defined as the major types of events occurring during surgery). Each phase is composed of several steps or tasks . An activity is a physical task and is composed by a list of motions . Each granularity level is assumed to describe the surgical procedure as a sequential list of events, except for the surgical procedure itself and lower levels of granularity  where information may be continuous .For instance, Uemura et al.  employedOn the other hand, Loukas et al.  used a dSPM was first introduced for supporting whole surgical interventions, but soon their applications in surgical training gained popularity amongst the surgical educational community . FirstlyOne of the steps involved in surgical process modelling (SPM) is surgical workflow analysis (SWA), which is the deconstructing process by which a surgical process is divided into a list of different phases, activities or tasks .Currently, SWA focuses on analysing the laparoscopic video to recognize surgical phases and activities using artificial intelligence. Probabilistic methods have proven to be of great interest in video-based SWA ,15. HoweA limitation of video-based analysis is that it is based on outcomes  rather than on the mental process that triggers it. Thus, our current line of research considers monitoring audio cues during surgical procedures from the attending surgical staff, motivated by the Think Aloud theory by which verbalization of tasks are inherently linked to the ongoing cognitive processes . To our We explored this avenue in a previous study , where wThe main goal of this study is to implement an intelligent system to train and assess surgeons’ procedural skills by means of SWA. The system will integrate a surgical phase classifier . To do this, we propose to create a desktop application to open videos and obtain and display metrics using the results from the surgical phase classifier. This system could also serve as a training tool to guide surgical residents in the procedure.In this study, we present both the application for procedural skills’ assessment and the algorithm for surgical phase classification based on speech and video, including preliminary results obtained from a test video.Laparoscopic cholecystectomy (LC) was selected as the main procedure for both studies to analyze since it is a widespread and convenient procedure for SWA (due to the unique characteristics of each of its phases), as well as one of the interventions with the highest training rate ,26. To train the endoscopic video model, the dataset from the “Surgical Workflow and Skill Analysis” sub-challenge of the Endoscopic Vision Challenge  was usedAdditionally, to account for those audio signals including sentences that are not related to any of these phases , we included a pseudo-phase in the audio-based model.To train the speech model, a database was created using 15 surgical educational videos found online and taken in different hospitals, in which the steps of the LC were verbally disclosed in Spanish by a single trainer surgeon equipped with a clip-on microphone . The audTo evaluate the performance of the application, two educational videos were used. These videos, similar to the ones in which the speech dataset is based, consisted of high-quality endoscopic video feeds found online on top of which the surgeon verbally discloses the ongoing procedure. One of them contains a total of 51 audio samples and 367 video samples (frames), whereas the other contains 28 audio samples and 269 frames. All samples were then manually tagged independently following the same guidelines as for the individual datasets.The implemented algorithm for the recognition of surgical phases uses two different classification models: one for the endoscopic video feed, and one for the surgeon-in-training’s voice (obtained through of a clip-on microphone). The audio and image channels are separated using FFmpeg . The speech signals are preprocessed to be divided into fragments and automatically transcribed into text using Google transcriber . Next, tThe endoscopic video is classified by means of a Convolutional Neural Network (CNN) with a Long Short-Term Memory (LSTM) unit . SpecifiThe first training stage learns the spatial information using transfer learning methodologies. This first stage takes frames as input and outputs the predicted class. The model resulting from this stage is used as a feature extractor, followed by an LSTM unit, such that the temporal information is learnt. LSTM units require sequences instead of individual frames to properly analyze the temporal dependencies. To accomplish this, sequences of nine frames in length, including the frame of interest and their four previous and four following frames, were extracted. The features from the individual frames included in these sequences were extracted from the model resulting from the first training stage and used to train the LSTM unit. Predictions from speech- and video-based models are synchronized in time. Specifically, the video samples are continuously processed until there is an audio sample obtained at the same moment. At this point, the predictions from both models are averaged until the audio signal ends. For instance, if an audio fragment took place between seconds 1–5, the phase predictions of the frames corresponding to these seconds were combined with the phase predictions of said audio fragment. Predictions from both audio and video models are given equal weight . Thus, whenever there is more than one frame for a single audio fragment, the weight for the frames is equally divided to add up to 50%, and the other 50% is reserved for the audio fragment. These weights were selected to ensure the prediction power was the highest possible.In addition, whenever the predictions of the audio-based model correspond to the pseudo-phase used to contain sentences not related to the procedure itself, they are eliminated from the analysis. Lastly, if no speech signal is captured , only the video prediction is used as the final prediction.The implemented intelligent system consists of a desktop application capable of obtaining audiovisual information, processing it accordingly, predicting the corresponding phase of the procedure, and obtaining a set of metrics related to procedural skills. The application was implemented using Python and tkinter. It first allows the user to select the source from which to obtain the assessment metrics. The source can be the URL of an online video, a video in the PC or a webcam connected to the PC to perform real-time predictions. The video and URL options are included in case recorded interventions want to be assessed retroactively using this application.If the URL is selected, the video will be downloaded and processed in the same manner as a local video. The local video (or the downloaded video from the URL) is read and the channels (audio and image) are separated. Then, images are cropped and saved locally to be processed by the image model, while audio is transcribed into text to be processed by the speech model. Images and text samples are classified according to their corresponding phase and averaged to obtain the final prediction. This prediction is printed on the screen together with the image corresponding to the video frame . In the case of real time predictions, the camera is opened, and images are fed to the video classification model, while the application uses the microphone to capture live conversation in the operating room (OR).x-axis represents the time in seconds and the colours correspond to the different phases. This allows for seeing the amount of time spent in each phase, the number of occurrences of phases and the percentage of time spent in each phase.Once the video or streaming is over, the metrics are calculated. Specifically, the total time spent in surgery, the time spent in each phase, the number of occurrence of phases and the average time whenever there is more than one occurrence are calculated in an attempt to characterize the surgical process in a timewise way. These metrics have already been validated for the assessment of procedural skills in the study by Uemura et al. . MetricsLastly, to evaluate the assessment capability of the application, we have modeled an average workflow of the process using the samples in the video database , since iAll metrics and the workflow of phases are processed and printed in a PDF file. This can be used as a report of the performance of the student, both for the student to reflect on and for the teacher to follow their progress.In order to assess the algorithm for phase recognition, performance was analyzed based on the F1 score, which is the harmonic mean of precision and recall. The F1 score of the complete algorithm was compared with that obtained by the video-based model and the speech-based model alone.We analysed the processing time of different modules of the application , which is essential for real-time feedback, by carrying out the process in the test video 10 times and averaging the processing times. Additionally, as a proof of concept, the overall workflow of the procedure (as obtained from the video database ) was comThe combination of audio and video models resulted in a higher F1-score than for both individually (0.875). In addition, the average error per phase was found to be 11.86% and the average error, 17.89%, improving with respect to the individual models and the state of the art. This suggests that the algorithm we created to combine the two models was effective in its purpose and yields a more accurate prediction of surgical phases. This can be indicative of the ability of visual information to reinforce speech predictions (and vice versa), reducing the possibility of errors introduced by one of the models alone.With respect to the video analysis alone, in the challenge in which the database we use was presented (HeiChole-19) , the resAs seen in The time processing analysis resulted in an average 0.007 s to calculate the audio prediction (per speech sample), 0.057 s to calculate the video prediction (per frame) and 0.094 s to calculate the average predictions (per speech sample and frame). The PDF is generated within an average 2.164 s.The time analysis shows that the video samples take more time to be classified than the speech samples. This may be due to the preprocessing associated to the video sample, which is necessarily longer than in the video case . Nevertheless, the total time can be considered low for real time predictions, even more so taking into account that the variations between frames are usually small. In addition, the PDF greatly increases the processing time when embedding the plots into the document, although it does not affect the actual prediction and processing time. This could be solved by optimizing the plot creation, especially in the case of the workflow, which uses graphs to connect one phase to another. Moreover, the desktop application could be converted to a web application in which an HTML displays the graphs and the user decides whether or not to save it as a PDF.The timeline of phases is especially useful to check whether the procedure was successfully followed. For instance, in the case studied in this work c, we obsThe time and occurrences’ analysis allows for having a better idea of the time spent in each occurrence of phases. The time is represented as hours, minutes and seconds . This coAccording to the resulting workflow from the integrated system, the surgeon in training successively goes from preparation, Calot’s triangle dissection, cystic duct and artery clipping and cutting and gallbladder dissection. This dissection was interrupted by the clipping (possibly because improved visibility was necessary). However, we know from the actual footage this was not the case and in fact, it is a misclassification of the integrated system. This stresses the need to improve the performance of the integrated system. The gallbladder dissection was followed by the gallbladder packaging, which in turn was followed by cleaning and coagulating. However, as pointed out before, while packaging, it may have been necessary to clean the anatomical surface and then continue packaging, which is why the surgeon in the test video went back to packaging from cleaning. The last phase is the gallbladder extraction (directly after cleaning and coagulating).The main limitation of this study corresponds to the high computing times that it currently conveys. This makes the possibility for real-time applications (both for training and assessment) difficult. Thus, the first goal short-term is to optimize the algorithms to ensure the processing time is shortened and real-time applications are indeed possible. This can be done by introducing more threads to the application while respecting the main thread of the front-end.On the other hand, although the model yields fairly accurate results, it fails to recognize phases, especially when first transitioning from one phase to another or whenever the field of view is too similar . In these cases, we aim at incorporating a model capable of recognizing instruments to incorporate additional information to help the model differentiate accurately between phases. We also plan to study a lower granularity of the procedure. Specifically, we will analyze the main actions in LC and annotate the speech samples accordingly. These new annotations will be used to train new models to classify the individual actions within the procedure (instead of the phases), so that we are able to explore the usefulness of this granularity level with respect to the assessment of declarative knowledge and advanced cognitive skills. Another limitation to our study was the lack of databases combining speech and video, which resulted in few videos available to test our models. To tackle this problem, we aim to create a database of speech and endoscopic video in surgical educational environments, which will likely be published to allow other researchers to work on this issue. This database will be used not only to adjust the weights of our models, if necessary, but also to test the validity of the proposed assessment system with surgical residents in the near future.Furthermore, as mentioned in the introduction, we aim for this system to have training functionalities as well, such as alerting students in real time of missed steps and suggesting different pathways. Such a system may even be upgraded with a dictionary of possible solutions to complications encountered. Lastly, we could convert the desktop application to a web application, including a login system which allows for a database of different procedures performed by a student, such that not only they can have access to it, but will also allow for their teachers to easily track their progress.In this study, we developed a pipeline to assess surgical procedural skills by means of a desktop application which uses a surgical phase classification algorithm to gather metrics from the procedure and present them visually to surgical trainers and trainees.The feasibility of the algorithm and application were demonstrated as part of a preliminary analysis of the functionalities of the application. In the near future, we aim to test the validity of the proposed application as part of the assessment process of procedural skills , as well as to test the feasibility to conduct real-time phase recognition to explore the potential of the application, and to provide residents with hints whenever disruptions from the normal workflow occur. We believe the application could have a major impact on the surgical educational community, since the human resources employed for the assessment of procedural skills will be reduced by much. In addition, the integrated SWA system could be used to provide formative feedback in real time and prevent mistakes in the overall procedure. Moreover, designers of procedural surgical simulators could benefit from the automatic workflows calculated in order to replicate different complications in the OR.


Safe donning and doffing of personal protective equipment (PPE) are critical to prevent transmission of infectious diseases. Novel strategies to improve infection prevention and control (IPC) adherence can optimise safety. We describe and quantify video surveillance of doffing at an outdoor hotel quarantine facility led by the Australian Medical Assistance Team in the Northern Territory, Australia.Motion-activated video cameras were installed in seven areas where personnel doffed PPE upon exit from an area dedicated to quarantined residents. Video footage was reviewed daily and compliance issues were identified using a standardised checklist and risk graded to initiate feedback. We collated audit data from 1 February to 18 April 2021 to describe trends by month, staff group, doffing component and risk.In 235 h of video footage, 364 compliance issues were identified, of which none were considered high-risk compromising to PPE integrity. Compliance issues were low risk  or moderate risk  and the most common issue was missed or inadequate hand hygiene . Compliance issues per minute of video footage reviewed decreased following introduction of the activity, from 24 per 1000 in February to 7 per 1000 in March and April.Video surveillance with feedback supported rapid response to improve IPC adherence in a challenging ambient environment. The activity focused on perfection to identify compliance issues that would go unreported in most healthcare settings and contributed to a suit of activities that prevented any high-risk PPE breaches or compromises to safety.The online version contains supplementary material available at 10.1186/s13756-022-01155-2. Safe donning and doffing of personal protective equipment (PPE) are critical to protect patients and staff from transmitting infectious diseases. To ensure PPE is donned and doffed in the correct sequence, with appropriate hand hygiene between each step and disposal of equipment, competency-based training and audits with regular and timely performance feedback are recommended , 2. HoweVideo surveillance with feedback has successfully improved hand hygiene, donning and doffing in the hospital setting –8. VideoHoward Springs International Quarantine Facility at the Centre for National Resilience was the Australian Medical Assistance Team (AUSMAT) operation responsible for the repatriation of Australian citizens and permanent residents unable to return to Australia by commercial flights from 23 October 2020 to 23 May 2021 . Over thIn mid-January 2021, following an increased number of residents accepted to quarantine at the facility, motion activated video cameras were installed at five doffing stations. The video surveillance had been successfully piloted at two doffing stations for the previous month. In March, two additional doffing stations were established and video monitored. All personnel were informed of the monitoring daily, and entry signage of video monitoring in progress was advised. The cameras were positioned to view the whole doffing station to record personnel doffing PPE upon exit from an area dedicated to quarantined residents. Personnel had donned PPE required for activities considered either low or high risk. High-risk activities was classified as face-to-face resident contact or terminal room cleaning, which required PPE for airborne precautions, including P2/N95 mask, eyewear, face shield, gloves and gown. All other activities were low risk and required gloves, surgical masks and eyewear. All personnel operated in a buddy system for all activities at the quarantine facility.Video footage was collected from each camera’s Secure Digital card and reviewed daily by a trained security officer through a compliance checklist developed by AUSMAT, based on the World Health Organization minimum guidelines  were moderate risk, 15% (55/364) were low risk and none were high risk. Compliance issues per minutes of video footage reviewed decreased from 24 per 1000 in February to 7 per 1000 in March and April  and accounted for 88% (135/153) of compliance issues reported in February. Despite an overall decrease in compliance issues over time, the proportion of compliance issues per staff group decreased for the cleaning contractors to 28% (28/102) in April and increased for all other staff groups except waste management contractors. The most frequent compliance issues were hand hygiene not performed between each step of the doffing sequence or for the required 20-s duration , mask removal   and incorrect doffing sequence or incorrect method of PPE removal for one step . Compliance issues became increasingly low risk (Fig. We demonstrated high performance of personnel from various professions to avoid any high-risk PPE doffing breaches in 235 h of video footage reviewed in an outdoor hotel quarantine facility with compounding hazards. Following introduction of the audit activity, compliance issues decreased between February and March, and remained low in April. The compliance issues identified were low or moderate, which would go unnoticed in most healthcare settings but contributed to the avoidance of any high-risk PPE breaches or true compromises to safety.We report a strong adherence to PPE doffing procedures, which may be explained by the operations prioritisation of creating a safety culture . The hypAn evaluation of the opportunities reviewed indicated PPE doffing compliance issues were initially by contract cleaners, however this increased for other staff groups, predominately other contractors, in April, this may be explained by the higher incident of contractors on site and increased tempo at the facility during this period. There is mixed evidence comparing PPE procedural adherence of non-clinical and clinical staff, however most evidence supports that clinical staff performed better –20. NotaWe report the most common compliance issue identified related to missed or inadequate hand hygiene, followed by mask removal. Existing literature reports most doffing procedural errors relate to insufficient hand hygiene, contact with contaminated surfaces or gown removal technique –24. HoweOur study has some limitations. Firstly, only one third of all video footage was reviewed, however, in emergency operations 11 weeks is a substantial period, which produced a large amount of performance data for immediate action. Secondly, we were unable to present total doffing events monitored, type of PPE doffed per event or PPE doffing events per positive resident, although we note daily staff entrances to the resident zone and disease prevalence as a proxy. Finally, the use of dedicated staff to perform ongoing surveillance may not be feasible in all settings, however the activity required inexpensive infrastructure and preserved clinical resources through use of a security officer trained to use a structured checklist.Video surveillance with feedback supported rapid response to improve IPC adherence and contributed to the prevention of SARS-CoV-2 transmission at the AUSMAT-led quarantine facility. The implementation of novel mitigation strategies should be considered to improve IPC adherence in high-risk settings.Additional file 1. The compliance checklist.Additional file 2. Detail on the intervention.Additional file 3. Detail on methods used to estimate the daily number of staff entrances to the resident zone.


Metacognition monitoring is the ability to evaluate the cognitive process actively. L2 learners with high metacognition monitoring ability can better monitor reading processes and outcomes consciously, thus facilitating self-regulated learning and improving reading efficiency. Previous studies mostly used offline self-reports to examine the metacognition monitoring in static text reading by L2 learners. This study investigated the effects of different indicators of metacognition monitoring on L2 Chinese audiovisual comprehension by online confidence judgment and audiovisual comprehension tasks. Target measures of metacognition monitoring included absolute calibration accuracy based on video or test and relative calibration accuracy measured by Gamma or Spearman correlation coefficient. 38 intermediate-advanced Chinese learners participated in the study. Multiple regression analysis showed three main results. First, absolute calibration accuracy can significantly predict L2 Chinese audiovisual comprehension, while relative calibration accuracy has no significant effect. Second, the predictive effect of video-based absolute calibration accuracy is affected by the video difficulty, that is, the greater the video difficulty, the greater the impact on the performance of audiovisual comprehension. Third, the predictive effect of test-based absolute calibration accuracy is influenced by the language proficiency, specifically, the higher the L2 Chinese proficiency, the stronger the prediction on the performance of audiovisual comprehension. These results support a multidimensional view of metacognition monitoring by specifying how different indicators of metacognition monitoring may predict L2 Chinese audiovisual comprehension. The findings have important pedagogical implications for strategy training of metacognition monitoring and point to the necessity to take task difficulty and individual differences among learners into full consideration. In the Previous studies mainly focused on the role of metacognition monitoring on L2 reading comprehension of static texts. Related studies have also investigated the effectiveness of metacognitive abilities in Chinese L2 static text reading, mainly using think-aloud protocol , questio2.calibration as the accuracy of the learners’ perception of his or her own performance, that is, the ability to assess comprehension accurately. The researchers asked the participants to make a confidence judgment during the reading process to self-assess their reading situation, and calculate the calibration bias, which is the difference between the self-assessed correct rates and the actual ones  Absolute Calibration Accuracy  refers to the absolute difference between the learners’ self-assessment and actual performance; the greater the absolute value of the subtraction between the two, the greater the gap between the learners’ self-assessment and actual performance; (2) Relative Calibration Accuracy  refers to the extent to which learners’ self-assessment scores reflect their actual reading comprehension scores, by calculating the correlation between the two correct rates. Higher correlation coefficients indicate greater agreement between learners’ self-assessment and actual performance . These tThe present study focuses on the metacognition monitoring of L2 Chinese audiovisual reading comprehension and examines the following four measures: (1) video-based absolute accuracy, the absolute difference between learners’ self-assessment of video comprehension and their actual performance after watching the video and before taking the test; and (2) test-based absolute accuracy, the absolute difference between learners’ self-assessment of their performance on the test items and their actual performance after watching the video and completing the test; and (3) test-based relative accuracy measured by Gamma, the correlation coefficient between learners’ self-assessment of their performance on the test items and their actual performance after watching the video and completing the test; and (4) test-based relative accuracy measured by Spearman, the correlation coefficient between learners’ self-assessment of their performance on the test items and their actual performance after watching the video and completing the test. Considering the four indicators of metacognition monitoring, this study will further investigate the following two questions:Do the four indicators of metacognition monitoring have a significant predictive effect on the performance of L2 Chinese audiovisual reading comprehension?Is the predictive effect mediated by video difficulty and Chinese language proficiency?3.3.1.SD = 2.05). All the participants were native Korean speakers and had studied Chinese for a mean of 6.81 years (SD = 3.65). They were intermediate and advanced Chinese L2 learners, with an average score of 25.47  in the fixed-ratio cloze test . The Pearson correlation coefficient between the subjective and objective measures of Chinese language proficiency in this experiment was 0.602 (p < 0.001), indicating a strong correlation. This study followed In order to ensure the validity and immediacy of the test results, we did not use the participants’ acquired HSK scores as the basis for measuring their language proficiency, whereas we remeasured the participants’ Chinese language proficiency from both subjective and objective perspectives: (1) Language proficiency self-assessment questionnaire. For the subjective measures, we used the Chinese translation of the Language Experience and Proficiency Questionnaire LEAP-Q; . In addi3.2.2.Participants were required to make confidence judgments before and after the test to further calculate the metacognition monitoring of the video and the test dimension. (1) Prediction confidence judgments. After watching the video and before taking the test, participants were asked, “How accurate do you think you will be when responding to comprehension questions regarding the video you have just watched?” They made judgments using a five-point scale ranging from 0 to 100% . (2) Pos3.2.3.China Focus, a set of comprehensive language textbooks designed specifically for L2 Chinese audiovisual and speaking courses. Three videos were selected for this experiment as practice video (1 min), test video 1 (3 min), and test video 2 (3 min). To control the difficulty of the videos, we revised the vocabulary and sentence patterns of the original videos and analyzed the difficulty of the revised texts with the assistance of the Chi-Editor, which is designed to provide CSL teachers with difficulty grading and intelligent adaptation of reading texts and contains three core modules: text rating, word annotation, and word archive  of the participants thought that test video 2 was more difficult.Participants watched three videos in Chinese on a computer screen, each followed by five multiple-choice questions, which consisted of two detail questions, two inference questions, and one topic question . The videos and test questions were adapted from  archive . The resChina Focus: Inspirational Section , indicating that larger absolute accuracy seemed to be associated with poorer audiovisual comprehension performance. The effect of video difficulty was significant , with the more difficult the video, the worse the comprehension performance. Importantly, we found a significant interaction effect between video-based absolute accuracy and video difficulty , with the follow-up contrasts demonstrating the greater the video difficulty, the greater the effect of video-based absolute accuracy on audiovisual comprehension performance. The effect of Chinese language proficiency was not significant , but the interaction effect of video difficulty and Chinese language proficiency was significant ; specifically, the effect of Chinese language proficiency on audiovisual comprehension performance was enhanced with the increase of video difficulty.The results of the multiple regression model with video-based absolute accuracy as the independent variable are shown in β = −2.98, SE = 1.17, t = −2.55, and p = 0.013), with the higher the deviation value of the self-assessment, the worse the audiovisual comprehension performance. The main effect of video difficulty was also significant , which indicated the more difficult the video, the worse the audiovisual comprehension performance. Chinese language proficiency was not a significant predictor in the model , while the interaction effect of test-based absolute accuracy and Chinese language proficiency was significant . Further comparisons revealed that the effect of test-based absolute accuracy on audiovisual comprehension performance increased with the improvement of Chinese language proficiency. The model yielded no significant interaction between test-based absolute accuracy and video difficulty .A similar set of multiple regression analyzes were also conducted with test-based absolute accuracy as the independent variable see . Test-ba5.5.1.The Self-regulation Learning Theory suggests that learners need to monitor their learning in real time to see if they reach their predetermined goals , emphasiThe present study did not find significant predictive effects of the two relative indicators of metacognition monitoring on L2 audiovisual reading comprehension, which may indicate that the practical predictability of the relative indicators is less stable . Masson It is worth mentioning that the four indicators of metacognition monitoring were examined comprehensively in this study, and there were differences in the predictive effects on audiovisual comprehension performance. Explicitly speaking, test-based absolute accuracy could significantly be predicted audiovisual comprehension performance and video-based absolute accuracy had weaker predictive power, while the two relative indicators did not have significant predictive effects. The above results provide support for the multidimensional view of metacognition monitoring , demonst5.2.Previous studies have focused on the role of metacognition monitoring in static text comprehension. In contrast, this study examined metacognition monitoring in L2 audiovisual reading comprehension using real-time measures, demonstrating the effectiveness of metacognition monitoring for intermediate and advanced Chinese L2 learners. In real-life contexts, self-regulated learning by native or L2 speakers is online in real time, which requires them to monitor and regulate their learning process instantaneously. Metacognition monitoring reflected in this process not only motivates learners to adopt effective strategies but also enhances their self-efficacy, thus improving the effectiveness of online learning . Ackerma5.3.The Cognitive Effort Hypothesis holds that learners’ self-assessment accuracy of text comprehension is affected by text difficulty. To be specific, self-assessment of text comprehension is less accurate if the text being read is easy for the learner since they do not pay excessive attention to the details of easy texts . The preThe present study also found that the predictive effect of test-based absolute accuracy on L2 Chinese audiovisual comprehension was also moderated by Chinese language proficiency, i.e., the stronger the predictive effect of metacognition monitoring on audiovisual comprehension as learners’ Chinese language proficiency increased. The above results suggest that the predictive effect of metacognition monitoring on Chinese L2 audiovisual comprehension is complex. The prediction of video-based absolute accuracy was affected by the video difficulty . In contrast, the prediction of test-based absolute accuracy was moderated by L2 proficiency . Our findings also provide evidential support for a multidimensional view of metacognition monitoring . As stat6.The current study systematically examined four indicators of metacognition monitoring in L2 Chinese audiovisual comprehension from the perspective of monitoring objects and statistical methods by online confidence judgment tasks. Furthermore, we explored the predictive effects of different indicators on L2 audiovisual comprehension performance. The main results showed that the absolute accuracy of metacognition monitoring had a significant predictive effect on L2 Chinese audiovisual comprehension, while the effect of relative accuracy was not significant. Specifically, the predictive effect of video-based absolute accuracy was influenced by the video difficulty . In contrast, test-based absolute accuracy was moderated by L2 proficiency . Our findings support a multidimensional view of metacognition monitoring and oppose a one-dimensional view that simply makes no distinction among all four indicators.Pedagogically, the empirical findings in this study not only emphasize the importance of metacognition monitoring in L2 audiovisual comprehension, but also provide some implications for teaching Chinese as a second language. Firstly, both Chinese teachers and learners should recognize that reading comprehension is not only a complex cognitive process but also a complex metacognitive process, and learners with higher metacognition monitoring ability can effectively regulate their own cognitive processes in real time and thus make more accurate self-assessments. Therefore, instructional interventions aimed at metacognition monitoring are essential. Teachers should provide learners with a variety of strategies needed to conduct more accurate self-assessment, such as planning and prediction of text content, assessment of the characteristics of text genre, assessment of a text content, construction and modification of a mental model, supervision of comprehension and text understanding, revision, and self-correction . SecondlThe original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.The studies involving human participants were reviewed and approved by Academic Committee of School of Chinese as a Second Language, Peking University. The patients/participants provided their written informed consent to participate in this study.YZ and YW designed the study and contributed to the final manuscript writing. YW, JH, ZA, and CL performed the study and collected the data. JH and YW analyzed the data. YW, JH, ZA, and CL prepared the draft. All authors contributed to the article and approved the submitted version.This study was funded by National Social Sciences Project “Application of Chinese Proficiency Grading Standards and Strategies of Textbook Research and Development (21STA032)”.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


The use of wearable sensors in movement disorder patients such as Parkinson’s disease (PD) and normal pressure hydrocephalus (NPH) is becoming more widespread, but most studies are limited to characterizing general aspects of mobility using smartphones. There is a need to accurately identify specific activities at home in order to properly evaluate gait and balance at home, where most falls occur. We developed an activity recognition algorithm to classify multiple daily living activities including high fall risk activities such as sit to stand transfers, turns and near-falls using data from 5 inertial sensors placed on the chest, upper-legs and lower-legs of the subjects. The algorithm is then verified with ground truth by collecting video footage of our patients wearing the sensors at home. Our activity recognition algorithm showed >95% sensitivity in detection of activities. Extracted features from our home monitoring system showed significantly better correlation (~69%) with prospectively measured fall frequency of our subjects compared to the standard clinical tests (~30%) or other quantitative gait metrics used in past studies when attempting to predict future falls over 1 year of prospective follow-up. Although detecting near-falls at home is difficult, our proposed model suggests that near-fall frequency is the most predictive criterion in fall detection through correlation analysis and fitting regression models. InjurioThe Movement Disorder Society-Unified Parkinson’s Disease rating scale (MDS-UPDRS)  analyzesA few methods of characterizing mobility in patients with movement disorders have been proposed, including home movement diaries  and charTo address some of the problems with characterizing mobility in patients with movement disorders, we have developed an algorithm that can both accurately measure gait parameters and enable real-time detection of high-risk activities in the patient’s home environment using inexpensive and widely available wearable technology. The merging of cost-effective technology with deep learning techniques yields significant promise in the field of wearable sensor technology . The goa2.2.1.Nineteen movement disorder patients who were being clinically evaluated and/or treated for either normal pressure hydrocephalus or Parkinson’s disease and 10 age-matched healthy control participants were enrolled over a period of 2 years from the Minneapolis VA Health Care System (MVAHCS) and University of Minnesota (UMN). Enrollment was designed to enroll a variety of types of movement disorder patients with varying gait dysfunction and postural instability ranging from normal gait and balance (MDS-UPDRS gait and pull test item scores of 0) to moderate dysfunction (MDS-UPDRS scores of 3). Patient participants were excluded if they were non-ambulatory or if they were unable to give consent. Control participants were excluded if they had any movement, gait, or balance disorders. Demographic information was collected . This st2.2.The measurement sensors were customized and reprogrammed inertial measurement units . The board was equipped with a high-performance ARM Cortex-M4 processor powered by 500 mAh high-capacity Lithium battery . The meaThe sensor configuration is one IMU sensor on each shank (just above the ankle), one IMU on each thigh, and one IMU sensor on the chest. This five sensor configuration uses an angle-based method taking advantage of the geometry of human in-plane walking. Each IMU sensor measures the acceleration and angular rate of movement of different body segments. An LMI-based non-linear sensor fusion algorithm is designed to estimate the limb segment orientations by taking advantage of the acceleration for lower frequencies and gyro in higher frequencies. The details of our estimation algorithm and the accuracy of a variety of kinematic variables which can be calculated using this configuration compared to a gold standard infrared camera measurements has been previously described .2.3.Each participant was shown how to properly place the IMUs  in the c2.4.n = 10) to wear a small video camera  that have been used in prior studies such as the total ambulatory time each day and number of ambulatory bouts each day . We alsoFrom the 29 participants, we have collected fall diaries and have survival data for 17 subjects. We did not collect the fall diaries from the control subjects and hence excluded them for the fall prediction model. From the remaining subjects, nine are censored since they are either new patients, or their home data is missing, or stopped sending their fall diaries to us before week 52.3.3.1.Eight patients with NPH and 11 with PD were enrolled for a total of 19 patient participants. Ten healthy, age-matched control participants were also enrolled. Demographic characteristics are demonstrated in 3.2.Ten of our patients generated more than 40 h of video footage which was manually annotated see  inset.The sum total of video footage in the entire subset of patients resulted in more than 14,000 total events which were used for algorithm predictions. To compare our LSTM algorithm to other standard classifiers commonly used to make predictions on large datasets, we have created receiver operating characteristic (ROC) curves for 6 activities of standing, walking, sit-stand transitions, turning, bending and near-falls. Eight of the most correlated features with fall frequency come from values calculated solely using the IMUs while the patient moves around their home and surrounding environment. For example, time spent lying down per day , the total number of ambulatory bouts at home (totNumABs), the frequency of sitting at home , walking frequency at home , the peak acceleration of the chest at home  and the distribution of ambulatory bouts in time [alpha parameter for ABs of more than 8 s  ] at home4.Using a combination of domain specific knowledge and machine learning techniques, we developed an automatic algorithm for detection and characterization of near-falls and high fall risk activities of the patients. We created a validated, video annotated and quantitative dataset of movement disorder subjects wearing inertial sensors at their home environment. The statistical analysis of our algorithm shows >95% sensitivity in detection of activities apart from near-falls, which showed 80% sensitivity. The correlation analysis of the computed features in our dataset showed that our novel metrics based on near-falls are superior in terms of the highest correlation with patient fall frequency over an entire year of follow-up while clinic-based features were either not correlated or were among the most weakly correlated features.4.1.Using video to validate algorithm predictions based on IMU data is a necessary component to reliably interpret any wearable dataset, but is frequently lacking in studies employing wearable sensors . StudiesAs such, our video-validated home data represents an ecologically valid quantitative dataset that can then be leveraged to understand the factors relevant in producing falls with far more detail compared to the current standard solution which involves qualitative clinical examination or laboratory-based task assessments that may or not be related to real world balance perturbations. Since most falls occur at home and surrounding environment, a wearable dataset that is “validated” in the clinic or lab may not be ecologically valid for fall prediction or other uses. On the other hand, some activities such as walking, standing and bending may be able to be easily validated in the clinic setting and so datasets validated with video “in the wild” should be compared to datasets using clinic-based assessments in order to better understand what can accurately assessed in the clinic vs. which assessments need to occur at home.Our activity recognition algorithm uses a nonlinear switched-gain observer based on measurements from IMUs worn on leg segments and the chest in order to estimate body segment orientation. The observer estimates the tilt angles and measurement bias is estimated and removed. This has been measured in prior studies using infrared-based motion capture systems to ensure its accuracy . These e4.2.We have developed prospective, predictive falls risk metrics that integrate the patient’s postural response along with data that reflects the patient’s home environment based on near-falls detection. We have included all known gait parameters used in prior studies and current clinical standards in the study, however our proposed metrics showed superior performance in predicting falls in these patients. While there are simple measures which may be more easily measured with a smartphone or smartwatch , our study suggests these are inferior to the number or frequency of near-falls. Similarly, clinical tests are typically inadequate in describing the likelihood of the subject falling and in characterizing the extent of their postural instability . Inter-aDespite being the most relevant feature of the dataset for fall prediction, near-falls were the most difficult activity to accurately detect with a sensitivity of 80%. This was for several reasons. First, near-falls look similar to other activities (like sit-stand transitions and bending) when examining inertial sensor data. Second, some of the near-falls were so subtle that they could not even be detected in videos. Finally, the natural occurrence of near-falls is relatively rare and obtaining video-validated samples is difficult since most of the patients who are at risk of falling are usually less active or use a walking aid to avoid falling. As such, they were also among the least common events and our algorithm, like all machine learning algorithms, performs better with more samples. Continued data collection with more validated events will likely help increase the accuracy of the algorithm over time.4.3.We have collected our dataset using an inexpensive wearable system based on inertial sensors to provide kinematic data of PD and NPH patients at home. Typical uses of IMUs worn by movement disorder patients at home are detailed gait analysis and metrics on mobility/ambulation which can be used for a wide variety of purposes such as disease stage assessment, fall prediction, and treatment evaluation, among others . An advaOne limitation of this dataset is its practicality as the current setup with five sensors might not be practical for everyday patient use. Future research should develop algorithms to use as few sensors as possible in optimal locations on the body. Because we were interested mainly in postural instability and falls, we did not include IMUs on the upper limbs and so our dataset does not include hand or arm movements. Given the frequent presence of upper extremity tremor in PD, this is particularly relevant for these patients in their diagnosis, monitoring and treatment evaluation. In addition, many activities of daily living can likely be classified with an upper extremity IMU. Further research should plan to integrate IMU/smartwatch-based data to obtain the widest variety of activities with the best diagnostic accuracy. Wearable usage should also be tailored to the specific usage desired by the clinician and patient. Finally, we plan to further develop activity recognition algorithms using unsupervised and semi-supervised learning methods to increase their accuracy or discover new activities which might have been missed by the current methods.Even though near-fall detection is difficult to recognize and our algorithm shows 80% sensitivity, near-fall frequency at home was still the most predictive criterion in the linear regression model compared to any other metric. Our results showed that the detection of near-falls is a far more powerful way to examine home monitoring data compared to current methods and should be incorporated into fall prediction algorithms. This validated dataset of movement disorder patients engaged in daily living activities in their homes can serve as a valuable resource for researchers to provide a ground truth for IMU algorithm comparison that include the natural responses of patients at home.The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation upon request.The studies involving human participants were reviewed and approved by University of Minnesota IRB. The patients/participants provided their written informed consent to participate in this study.RM conceived the original idea. AN developed the algorithms, designed the sensors and measurement setup with the help and supervision of RR. AN, AJ, JH, LS, JJ, TL, ER, YM, SR, KN, and CS helped with the data collection and annotations. AN analyzed and interpreted the results with the help and supervision of RM. AN and RM wrote the manuscript with the help of AJ and LS. RM and RR supervised the project. All authors contributed to the article and approved the submitted version.This work was funded in part by a grant from the University of Minnesota MnDRIVE Neuromodulation Program and Institute for Engineering in Medicine (IEM) doctoral fellowship.The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.https://www.frontiersin.org/articles/10.3389/fnagi.2023.1117802/full#supplementary-materialThe Supplementary material for this article can be found online at: Click here for additional data file.Click here for additional data file.


An aged population, increasing care needs, and a lack of (in)formal caregivers represent major challenges for our society today. Addressing these challenges fuels efforts and developments in innovative technologies leading to various existing AAL applications aiming at improving autonomy, independence, and security in older age. Here, the usage of video-based AAL technologies is promising as detailed information can be obtained and analyzed. Simultaneously, this type of technology is strongly connected with privacy concerns due to fears of unauthorized data access or inappropriate use of recorded data potentially resulting in rejection and non-use of the applications. As privacy-preserving visualizations of video data can diminish those concerns, this empirical study examines the acceptance and privacy perceptions of video-based AAL technology applying different visualization modes for privacy preservation (n = 161). These visualization modes differed in their degrees of visibility and identifiability, covering different levels of privacy preservation  and are specifically evaluated based on realistic video sequences. The results of our study indicate a rather low acceptance of video-based AAL technology in general. From the diverse visualization modes, the “Avatar” mode is most preferred as it is perceived as best suitable to protect and preserve the users’ privacy. Beyond that, distinct clusters of future users were identified differing in their technology evaluation as well as in individual characteristics . The findings support the understanding of potential users’ needs for a successful future design, development, and implementation of video-based, but still privacy-preserving AAL technology. Tackling the rising challenges related to demographic change, efforts have been made to develop innovative approaches, such as Ambient Assisted Living (AAL) technologies, aiming at support and assistance of older people and people in need of care ,2,3. In Beyond the potential and the advantages the usage of AAL technologies might bring, barriers and concerns, in particular regarding an intrusion of privacy and data security, exist ,11,12. ITechnical developments and adaptions offer the option to address privacy concerns by applying diverse visualization modes that conceal visual details of the recorded footage . Thus faWith regard to the technical developments, there are rather fluent transitions between the areas of assisting systems, lifelogging approaches, assistive smart home technologies, and Ambient Assisted Living: these approaches share that they are conceptualized, developed, and designed for support in older age in terms of activity and behavior monitoring, detection of emergencies,  reminding of purposes, and relief in everyday life . InTo sum up, a variety of applications are aimed at the support and assistance of people in their private home environments based on intelligent monitoring and analyses by means of computer vision. Thereby, the technology is considered to be used to generate information about the users, i.e., especially older people in need of care, allowing risk detection and support services and simultaneously protecting the users’ privacy as best as possible. Addressing the latter, Padilla-López et al.  proposedReaching from wearable sensors in clothing, smart technologies to be taken wherever desired, up to ambient systems integrated into private environments and institutional care facilities, assisting technologies can now accompany us almost everywhere. But are we also willing to accept them? Do we want them around us everywhere and at all times? In which situations are they helpful, and when are they a nuisance? These questions are crucial when it comes to the use of technologies in different contexts.In the process of technology adoption, technology acceptance is an undenied key player. Research in this area that originates not only from the area of information systems but also from psychology and sociology has resulted in several theoretical models: the most established technology acceptance models are the Technology Acceptance Model  and the In addition to these generally acknowledged acceptance models and criteria, former research has also identified that context-related perceptions of benefits and concerns regarding a particular technology considerably shape its acceptance and long-term adoption ,26. JascOn the other hand, concerns associated with assistive technologies can significantly reduce the willingness to use them. The most important empirically elaborated barriers having a high contradicting impact on technology acceptance are the uncertainty regarding data security, stigmatization, lack of human interaction, loneliness, and costs e.g., ,27). One. One27])Thus, the acceptance issue when using assistive technologies in living environments is a decisive one, especially for technology generations that barely spent a (large) part of their lives dealing with digitization and data processing . In a prDue to its enormous role in acceptance, the sensitive issue of privacy in the use of video-based data is addressed in more detail below.Definitions of privacy are either value-based and described as a human right or economic commodity or cognate-based, referring to privacy as a behavior or a predisposition of the individual to behave . As partFurthermore, privacy plays a crucial role when it comes to personal well-being, positive human functioning and effective management of social interactions ,43,46,47Similarly, in the field of video-based AAL, privacy is mainly regarded as a concern comprising feelings of permanent surveillance, fear of access and misuse of personal information, and obtrusiveness, which all possibly hinder the adoption of such intelligent  systems ,50,51,52In general, the importance of privacy and its preservation within the context of video-based AAL cannot be denied and many technological efforts are being developed to adhere to user preferences and perceptions.In addition to the technical potential of privacy-preserving approaches, such as the development of visualizations allowing different levels of visibility and identifiability , the perRQ1: How are video-based AAL technologies perceived and accepted in general?RQ2: Which specific visualization modes are selected for different exemplary situations of everyday life (in older age)?RQ3: Are there differences in the acceptance of the different visualization modes?RQ4: Are the visualization modes and their characteristics evaluated differently?RQ5: Are there user groups differing in the acceptance of video-based AAL technology? How are the groups and their evaluation patterns characterized?To address these research gaps, it was the aim of this empirical study to reach a specific and realistic evaluation of video-based AAL technologies applying different visualization modes addressing adults of all ages. For this purpose, different visualization modes were evaluated based on real video sequences the participants watched prior to the evaluations of the single visualization modes. The underlying research questions were the following:In the following section, the empirical approach of the conducted study is presented, introducing the design and the concept of the online survey. Further, we describe the characteristics of the sample and the procedures of data analysis.The idea of this study was to enable the evaluation of video-based AAL technologies in general as well as specifically in terms of the assessment of different visualization modes being applied for video-based AAL technologies. To realize these evaluations, an online survey was conceptualized, aiming at reaching a broad sample of adults covering all ages. The survey’s structure with its constructs and their items are described in For the general evaluation of video-based AAL technologies, a descriptive scenario was prepared for the participants explaining the use and functions of a video-based AAL system. It was aimed at providing technically correct information in a simple, comprehensible, and transparent manner. The scenario served as a basis to introduce the participants to video-based AAL technologies enabling the evaluation of their perspectives and opinions on the general idea of video-based AAL technologies. The applied descriptive scenario was conceptualized as follows:The system described here aims at everyday support in older age and works with the help of cameras. The cameras are installed in the residents’ own homes and can record their daily lives. The cameras are about the size of a hand and are mounted at head height and in various rooms. They record only the image but no sound, so they cannot listen to conversations. The system and its cameras can be switched off and on by the user at any time. The cameras, or the system behind them, can offer users various health services. These include, for example, the detection of falls, support during rehabilitation , remote monitoring by doctors or nurses, or the early detection of signs of dementia or frailty. The cameras continuously evaluate the video material. In this way, the system can independently detect an accident or emergency and then trigger an emergency call. With the user’s consent, it is also possible to record important events to learn why they happened - for example, what caused a fall. The more videos that are recorded, the more likely it is that the system will be able to detect, for example, whether a person is suddenly performing unusual movements or actions. If this indicates a risk, e.g., in the case of persons with incipient dementia, family members or the family doctor can be notified.”“In the second step, the assessment of different visualization modes being applied for video-based AAL technologies was focused on. As described before, previous research has investigated the perception of different visualization modes based on pictures . To enabBefore the participants evaluated the visualization modes, a scenario was given to them in order to introduce the visualization modes and to enable the participants to better empathize with the situation of having a video-based AAL system in their own home environment. For this purpose, the following explanations and scenarios were used:“Using a lifelogging system in your own home comes with several advantages and disadvantages. For example, installing cameras may raise privacy concerns. Such concerns may arise because your family members, doctors, or caregivers may be watching you in situations where you do not want them to. To mitigate these privacy concerns, various video visualization modes have been developed to make people unrecognizable on the video. The following is a brief explanation of the filters and their characteristics. Please read this explanation carefully.Imagine that you have decided to use a video-based AAL system for support in older age at home. The system now provides you with different visualization modes, each offering a different level of privacy and visibility. Before installing the cameras, you can select a mode that will make you unrecognizable on the video footage. Once installed, the cameras now record what’s happening in your home around the clock. The moment someone wants to access your videos, the visualization mode you selected will be activated. The access can be from one of your relatives, your doctor or nurse, for example, because they want to check your status or because the system has sent a warning - like in case of a fall. The visualization mode you choose now processes the recorded video so that your caregivers can only see the video in the way you want them to. In doing so, the visualization modes are designed to protect your privacy while allowing your family members, doctors, or caregivers to assess whether something has happened to you. The stronger a visualization mode is, the more detail is obscured from the image. While with some visualization modes you can still make out colors, structures and shapes, other modes only show the person’s posture or figure. The more details are rendered unrecognizable, the less information can be extracted from the video by relatives, doctors, or nurses.The next step is to evaluate the visualization modes. For this purpose, short video sequences are shown below, which present the filters in different situations. As mentioned at the beginning, there are no wrong answers, you are asked for your personal feeling with regard to the videos and their visualization modes. We will start with a concrete application scenario of the lifelogging system and its cameras. Please watch the following video. Imagine that the camera is installed in your home and that you are the person being filmed here.”After the participants read the scenario and the instructions, they started to watch the videos for three situations of daily living: “Falling” as a concrete use-case and frequent situation in older age, “Eating” as a typical activity of daily living, and “Undressing” as an example for a sensitive situation in the everyday life. For each situation, first, the original video was shown. Afterward, four videos were presented, applying the different visualization modes in a randomized order.The evaluation of the visualization modes followed after the presentation of all videos for the three situations. Here, the participants evaluated their perception of the three situations and selected their most preferred visualization mode for each situation. Related to each visualization type, the participants evaluated their perception as well as their acceptance of the visualization modes in terms of an intention to use them.The applied scenarios, as well as the structure and items of the online survey described in the next section, have been pre-tested by three persons of different ages. These participants confirmed the comprehensibility, in particular, with regard to the scenarios and reported an average response time of 25 min.The online survey was set up in the German language and was structured into four parts: (i) demographics and user factors, (ii) evaluation of video-based AAL technologies in general, (iii) selection of specific visualization modes for different situations, and (iv) evaluation of the visualization modes. To support the understanding of the process, the key components of the online survey are illustrated in (i) Demographics and user factors. Following the introduction that provided an overview of the study’s aim and intention, the first part focused on demographic data and user factors. In addition to demographic data, such as age, gender and education level, various personal attitudes were collected. Among others, these were needed for privacy  Evaluation of video-based AAL technologies in general. This part of the survey started with a descriptive scenario explaining the use and function of a video-based AAL system (see stem see . Followi results . The gen(iii) Selection of specific visualization modes for different situations. In this part of the survey, the participants were first introduced to the privacy-preserving video-based technology; here, the different visualization modes were presented. Within a short introduction, it was described that privacy and surveillance concerns are well recognized and that video filters have been developed to diminish those concerns. In this regard, a scenario was provided to the participants putting them in the role of a potential user of the video-based AAL system  Evaluation of the visualization modes. The fourth part of the study aimed at the assessment of the single visualization modes independently. Therefore, each visualization mode was displayed again using an image as a reminder and evaluated by means of a semantic differential and a concrete intention to use the visualization modes. To avoid a valuation bias, the order of the visualization modes was again randomized. Starting with the semantic differential, nine pairs of adjectives were evaluated by the participants on six-point scales  vs. not acceptable (=6)). The semantic differential included aspects referring to the visualization modes’ acceptance, their potential for privacy preservation, technical characteristics, and appeal. All pairs of adjectives are illustrated in At the end of the online survey, thanks were expressed for sharing opinions and participating in the study, and an open comment field was provided for feedback, questions, and suggestions.For data acquisition, the link to the online survey was shared in social networks, i.e., groups whose topics indicated an interest in the subject of this study, such as interests of nursing, care institutions and the care of family members . In addition, a poster with a QR code leading to the online survey with a brief explanation of the study’s topic and purpose was hung up in places where many people spend their time, such as bakeries, cafés, and the waiting rooms of physicians’ office.Overall, 260 respondents took part in the study that was conducted in Germany in July 2020, and data collection was open for 24 days. After data cleaning , n = 161 data sets remained for further analyses (“N” refers to the whole population of the sample). In the following, “n”, is used to describe subgroups within our sample. the mean age of the participants was 46.80 , covering an age range between 18 and 86 years. While individuals aged 50 years and older were surveyed for the reason that they are the target users of the technology, younger people served on the one side as potential future users and on the other side as a control group to identify differences in their perception of the object of investigation. The sample consisted of 62.1% female (n = 99) and 37.9% male (n = 61) participants (one participant indicated a diverse gender). The educational level of the participants was high with 50.9% (n = 82) of the participants holding a university degree, 21.7% (n = 35) a university entrance qualification, and 5.0% a doctoral degree. Only 22.4% (n = 36) of the participants reported lower education degrees, i.e., diverse secondary school certificates. Considering health-related aspects, the sample was overall in a good state of health, as only 27.3% (n = 44) of the participants indicated suffering from a chronic disease, and only 13.7% (n = 22) indicated having a physical disability or restriction. Referring to the measured individual attitudes , the participants generally showed a moderately positive evaluation of their technology commitment  and a confirmation of their privacy perception —indicating existing needs for protecting and preserving their own privacy.p > 0.05) were interpreted as not significant (n.s.).All measured items referring to the evaluation of video-based AAL technologies were assessed on six-point Likert scales , whereas the value of 3.5 represented the mid-point of the scale. Hence, values < 3.5 indicated rejection, while values > 3.5 indicated acceptance of an item. Reliability analyses .The participants assessed the overall Acceptance of video-based AAL technology slightly negatively  based on the mean of the scale (M = 3.5). Operationalized as a behavioral intention to use, the participants tended to slightly reject the two positive items, whereas they simultaneously confirmed the negative item: …triggers an emergency call…  and …notifies someone who can help me… ). Other aspects, e.g., referring to living independently in my home  and …relieve my relatives of care , received agreement as well. In contrast, the item I can move around my home without any worries  was assessed almost neutrally, while the participants rejected liking to use the system in case they are frail and dependent on help .Beyond the acceptance construct, the participants also evaluated the perception of video-based AAL technologies in terms of Perceived Benefits  and Perc…feel disturbed if cameras were installed… , …I could do nothing without anyone noticing , …would violate my privacy , and …feel monitored by the system . Further, the participants confirmed that they would only use the system if they were dependent on care . Related to this, the participants further agreed with the statement that they do not feel in the condition to depend on such a system, indicating that necessity and dependency are relevant usage conditions. Further, the concern that the costs for the system are too high  was only slightly confirmed. Finally, concerns related to false alarms  were slightly rejected, indicating that these aspects do not represent severe barriers of using video-based AAL technology.Moving to the Perceived Barriers, overall, the results showed confirmations of the participants , indicating that almost all single aspects represent “real” barriers of using video-based AAL technology. Four aspects referring to an invasion of privacy received the highest agreements and thus represent severe barriers: Beyond an assessment of video-based AAL technologies in general, this study aimed at an investigation of different privacy-preserving visualization modes of video-based AAL technology, which were presented to the participants per video examples. The respective results are presented in the following sections.Eating as a typical activity of daily living, Falling as an example of a critical situation in older age, and Undressing as a presumably very private and intimate activity , moderately private , and rather not intimate . In comparison, Undressing was also not perceived to be dangerous , but to be clearly private  and intimate . Not surprisingly, Falling was perceived to be dangerous , private on a rather neutral level , and not to be intimate . These distinct evaluation patterns confirm the three activities to cover a range of differently perceived activities of daily living. Therefore, it is useful to investigate which visualization modes the participants would select and prefer in these situations.Starting with Eating, the majority of the participants selected the “Avatar” visualization mode , followed by the “Blurred” visualization . Compared to that, the “Pixel”  and “Grey”  modes were chosen by only a few participants. Beyond that, 19.5% (n = 31) selected the option that they do not want a camera to record this activity, while 15.7% (n = 25) selected the opposite opinion, namely that this activity can be recorded and shown without a privacy-preserving visualization .Undressing, the selection patterns changed and most of the participants  chose the option that they did not want a camera to record this activity. Only 6.3% (n = 10) selected the opinion that this activity can be shown by the original video . Related to the specific visualization modes, the majority of the participants selected the “Avatar”  mode again, followed by the “Pixel”  and the “Blurred”  visualization modes. In line with the evaluations of the activity Eating, the “Grey” mode  was selected the least frequently.Moving to the situation of Falling, the selection patterns changed fundamentally. Most of the participants selected that this activity can be shown without a privacy-preserving visualization, indicating that the original video would be accepted . In comparison, a clearly lower proportion of the participants  chose the option that they did not want a camera to record this situation. Related to the specific visualization modes, the “Grey”  and the “Avatar”  modes were preferred, followed by the “Blurred”  and the “Pixel”  visualization modes.Considering the situation of The specific selection of the visualization modes has shown that their evaluation and acceptance depend on the respective context. In the following, it is investigated to what extent the evaluation and acceptance of privacy-preserving visualizations of video-based AAL technologies also depend on the individual characteristics of the potential users.Acceptance of the Visualizations  vs. not acceptable (=6) (useful (=1) vs. not useful (=6)  vs. does not protect my privacy (=6)  vs. does not alienate me as a person (=6) (makes my face unrecognizable (=1) vs. shows my face clearly  vs. does not make me look good (=6) (is sympathetic to me (=1) vs. is unsympathetic to me (=6)  vs. allows misperceptions (=6)  overall acceptance of video-based AAL technologies, (2) the perceived benefits and (3) the perceived barriers, two clusters were identified as the optimal cluster solution in the data set. In addition to specific differences in the evaluation, both clusters were also characterized by differences in the individual factors, which are introduced first .The identified clusters did not differ regarding demographic characteristics, e.g., age, gender, or educational level. In addition, both clusters did not differ regarding health  or living situation. Instead, both clusters differed regarding attitudinal characteristics. Cluster 1 was characterized by a significantly lower perception of privacy needs, whereas Cluster 2 showed higher needs regarding protecting and preserving privacy. Beyond that, technology commitment was significantly higher in Cluster 1 compared to Cluster 2.Acceptance in terms of an intention to use video-based AAL technology  of this study referred to the general perception and acceptance of video-based AAL technology. In line with previous research, the results of this study confirmed that video-based AAL technologies are evaluated on a neutral, slightly rejecting level, indicating that visual approaches are perceived to be more critical than audio or rather binary technologies ,14, suchIn more detail, the participants acknowledged the potential benefits of using video-based AAL technologies, especially regarding emergency-related aspects, such as notifications in case of emergencies. It is therefore recommended that future information strategies regarding video-based AAL technologies should focus on communicating these highly relevant perceived benefits: thereby, emergency-related functions should be taken as an example and should be explained in a technically correct but transparent, comprehensible, and clear manner.In addition to the perceived benefits, the results of this study revealed high confirming evaluations of perceived barriers as well, in particular regarding privacy-related aspects, whereas the economic and technical concerns were comparably low. These high confirmations of the (privacy-related) perceived barriers are characteristic of video-based AAL technologies, as previous research clearly identified lower evaluations of these barriers and concerns related to AAL technologies in general  or in spMoving to the second research question (RQ2), it was aimed at an investigation of future users’ preferences when the specific visualization modes were used for different situations in their everyday life. In the present study, three exemplary situations of daily living in older age were prefixed, covering a broad range of situations of life in older age . In the first step, the results confirmed that the three situations  were clearly perceived differently in terms of the danger attributed to them—intimacy and privacy. These findings highlight that privacy perceptions are strongly context-bounded ,45,46, wNevertheless, the three selected situations first enabled concrete selections of the preferred visualization modes for video-based AAL technologies and resulted in different preference patterns. For the situation of “Eating”, the “Avatar” mode clearly represented the most preferred visualization type: presumably due to the highest level of privacy preservation enabled by this visualization, as it does not allow any conclusions about the colors, textures, and shapes of the monitored person. Further, a third of the participants did not select a visualization mode being of the opinion that this activity should either not be filmed by a camera or they would not permit the original video. These two opposite answers highlight the discrepancy in the trade-offs between protecting the privacy and disclosure of privacy in order to enable functions improving security, autonomy, or independence. For the situation of “Undressing”, the majority of the participants did not want this situation to be filmed by a camera due to the high intimacy and high need for privacy in this situation. However, the “Avatar” mode still represented the most preferred visualization mode. In particular, in this—very intimately perceived—situation, a visualization mode is desired that conceals individual characteristics, such as shape, colors, and textures. Considering the situation of “Falling”, the selection patterns changed in that the majority of the participants would permit the original video. Here, the trade-off between preserving privacy and increasing security turned in favor of increasing security, depending on this—very dangerously perceived—situation. Related to the visualization modes, the “Grey” and the “Avatar” modes were preferred. In this regard, it can be assumed that the “Avatar” mode was in part selected as it still provides the most protection of privacy out of the four alternatives. The “Grey” mode was presumably selected due to the fact that more details can be obtained from a video applying this visualization  increasing the security in terms of a higher probability of correct detection of falls and emergencies. The “popularity” (even with comparably low ratings) of the avatar across all three situations may be explained in terms of bodily privacy ,63 and dSummarizing these situation-dependent results, it is not that easy to derive generally applicable recommendations. However, the results show that situations of daily living are perceived very differently and that there is no single decision profile that applies equally to all future users of video-based AAL technologies. Therefore, video-based AAL technologies and their visualizations should be individually tailored to the users by enabling them to turn the cameras on and off and to select specific visualization modes depending on the respective context . This way, future users are free to decide whether to be recorded at all in certain situations or in which manner video footage can be recorded and seen. Beyond that, future communication and information strategies should focus on the trade-off between (i) protecting privacy by using visualizations with a high degree of privacy preservation and (ii) increasing security by choosing visualizations that allow more conclusions to be drawn about situations in terms of representing, e.g., shapes and gestures. Here, both directions should be disclosed and explained in a transparent and comprehensible way.The investigation of differences in the acceptance of the specific visualization modes represented the content of the next underlying research question (RQ3). In accordance with RQ1 and confirming previous research ,14 for sThe results of the fourth research question (RQ4), which focused on potential differences in the evaluation of the visualization modes, provide explanations for the evaluation differences regarding the acceptance (RQ3). Within the semantic differential, the evaluations showed large differences, in particular regarding privacy-related characteristics for which the “Avatar” mode achieved clearly higher ratings compared to all other modes. Hence, the high potential of privacy preservation of this mode led to its higher acceptance. However, the potential for allowing wrong perceptions was rated highest for the “Avatar” mode as well due to the fact that the concealed gestures, colors, textures, and shapes provide the least information compared to all other visualization modes. Again, these evaluations highlight the complexity of the trade-off between protecting privacy and enhancing security. In contrast to the “Avatar” mode, the “Grey” mode was characterized by the most negative evaluations referring to its acceptance but also to its appeal and sympathy. In terms of communication and information strategies, the characteristics of all visualization modes—their potential, their advantages, but also their drawbacks—should be focused on; this way, future users can be enabled to evaluate the trade-off between privacy and security and to make informed decisions about using or not using such video-based AAL technologies.how” they seek privacy—their privacy regulation management—not enabling them to reach their desired level of privacy. These cluster differences were also visible regarding the evaluation of the specific visualization modes, in which the rejecters clearly showed more negative assessments regarding the acceptance of the specific modes compared to the adopters. However, the adopters’ evaluations also did not result in positive ratings of the visualization modes’ acceptance. This shows that even for people with a positive attitude towards video-based AAL technology, the four visualization modes did not represent an optimal solution for applying video-based AAL technologies. Future research should, therefore, integrate future users right from the beginning to consider their opinions and ideas in the development of other, specifically tailored visualization modes focusing maybe not only on the way video information is presented but also on what exactly it is that is presented , following a skin segmentation approach [In the final step of our study (RQ5), we investigated if there were user groups that differed in their evaluations of video-based AAL technologies. Thereby, we identified two clusters that differed regarding their general perception of video-based AAL technologies. One cluster of participants was characterized as adopters because this group revealed having a positive intention to use video-based AAL technologies, a positive perception of the potential benefits of using the technology, and a rather neutral perception of potential barriers. In contrast, the participants of the second cluster were characterized as rejecters of video-based AAL technology, as they expressed a clearly negative intention to use the technology, a rather neutral evaluation of potential benefits, but a clear confirmation of the potential barriers to using video-based AAL technology. Summarizing these relationships, the adopters were motivated to use video-based AAL technology predominantly driven by the advantages the technology brings along, whereas the rejecters denied the usage of video-based technology based on a high perception of the technology’s potential disadvantages. From a privacy-perception perspective, it seems that participants manage their privacy regulations based on how much weight they put on the benefits and barriers. Seemingly, AAL-rejecters consider barriers, including concerns regarding privacy, as a disturbance in their manner of “approach .In addition to the differences in the evaluation patterns, the two clusters were characterized by individual differences as well. The adopters showed a more positive technology commitment, while the rejecters were characterized by a higher privacy perception in terms of higher need for privacy. Apparently, for the adopters, the benefits were quite sufficient to satisfy needs such as autonomy, confiding or contemplation , whereasThis study provided detailed insights into a scenario-based but realistic evaluation of different visualization modes being applied for using video-based AAL technologies. In addition to the general insights into the comparably low acceptance and perception of video-based AAL technology, the visualization modes were specifically evaluated in terms of their acceptance and their potential for privacy preservation, as well as other technical and context-dependent characteristics. Here, the “Avatar” mode was revealed to be the most preferred one enabling the highest (perceived) level of privacy preservation.In addition to detailed new findings on the acceptance of video-based AAL technology, some limitations have to be considered for future research regarding methodological and sample-related aspects. Starting with a methodological issue, it has to be noted that the survey was comparably long and complex. In particular, watching all video sequences  for the three selected activities required some patience from the respondents; the time taken could serve as an explanation for the comparably high rate of incomplete data sets (n = 99). However, this procedure enabled an assessment as realistic as possible, avoiding sequence effects through the randomized design. Considering this study’s sample, its distribution regarding age and gender was quite balanced, whereas the educational level was high. This distribution may be due to the online format of the survey. In addition, the acquisition primarily addressed people who are interested in or committed to the topic of the study and corresponding sub-topics. As a result, the sample could also have an increased proportion of more highly educated people. Here, future studies should try to reach and address all levels of education adequately, e.g., by additionally using paper and pencil surveys. Discussing the age distribution in our sample, we realized our aim of reaching a broad sample covering the whole range of adulthood. The median age (49) showed that we reached older users as well as younger users . It would be worth specifically examining the group of oldest and frailest users in more detail, i.e., people with disabilities/chronic illnesses, in order to elaborate on their specific requirements and wishes, as this group might benefit from the video assistance the most.The last aspect refers to the fact that the participants in this study evaluated the usage of video-based AAL technology from their own perspective, imagining the usage within their own home environment. Future studies should investigate other scenarios in order to compare evaluation and acceptance patterns, e.g., perspectives of caring relatives, perspectives of professional caregivers, or the perspective of being an older person in need of care.