Phase and step annotation in surgical videos is a prerequisite for surgical scene understanding and for downstream tasks like intraoperative feedback or assistance. However, most ontologies are applied on small monocentric datasets and lack external validation. To overcome these limitations an ontology for phases and steps of laparoscopic Roux-en-Y gastric bypass (LRYGB) is proposed and validated on a multicentric dataset in terms of inter- and intra-rater reliability (inter-/intra-RR).The proposed LRYGB ontology consists of 12 phase and 46 step definitions that are hierarchically structured. Two board certified surgeons (raters) with > 10 years of clinical experience applied the proposed ontology on two datasets: (1) StraBypass40 consists of 40 LRYGB videos from Nouvel Hôpital Civil, Strasbourg, France and (2) BernBypass70 consists of 70 LRYGB videos from Inselspital, Bern University Hospital, Bern, Switzerland.To assess inter-RR the two raters’ annotations of ten randomly chosen videos from StraBypass40 and BernBypass70 each, were compared. To assess intra-RR ten randomly chosen videos were annotated twice by the same rater and annotations were compared.Inter-RR was calculated using Cohen’s kappa. Additionally, for inter- and intra-RR accuracy, precision, recall, F1-score, and application dependent metrics were applied.The mean ± SD video duration was 108 ± 33 min and 75 ± 21 min in StraBypass40 and BernBypass70, respectively. The proposed ontology shows an inter-RR of 96.8 ± 2.7% for phases and 85.4 ± 6.0% for steps on StraBypass40 and 94.9 ± 5.8% for phases and 76.1 ± 13.9% for steps on BernBypass70. The overall Cohen’s kappa of inter-RR was 95.9 ± 4.3% for phases and 80.8 ± 10.0% for steps. Intra-RR showed an accuracy of 98.4 ± 1.1% for phases and 88.1 ± 8.1% for steps.The proposed ontology shows an excellent inter- and intra-RR and should therefore be implemented routinely in phase and step annotation of LRYGB.The online version contains supplementary material available at 10.1007/s00464-022-09745-2. The aim of Surgical Data Science (SDS) is to analyze data sources acquired during surgical treatment to improve patient safety and clinical outcomes . OpposedSince the introduction of video technology in minimally invasive surgery, video recordings of surgical interventions are easily recorded and therefore readily available. As analysing surgical videos is time consuming, costly, and often lacks objectivity, the full potential of video analysis was often not tapped in the past . In the One of the most analysed surgeries in SDS is laparoscopic cholecystectomy (LCHE). Classical CV tasks in the analysis of surgical videos are phase recognition and tool presence detection. They were developed for LCHE . MoreoveLaparoscopic Roux-en-Y gastric bypass (LRYGB) is one of the most performed bariatric surgeries worldwide , 12. TheSurgical interventions can be hierarchically decomposed into phases , that consist of more fine-grained steps  . In contHowever, most datasets in SDS are small, monocentric, and lack external validation. To overcome these limitations, larger and multicentric datasets are warranted. Furthermore, variability of ontologies and its application on datasets limits generalization across centers . To ensuThis work is the first to propose a LRYGB ontology for phases and steps and to validate it on a multicentric dataset in terms of inter- and intra-rater reliability. The application of this ontology enables workflow analysis across surgeons and centers. Furthermore, it facilitates downstream applications of SDS, as the training of phase and step recognition algorithms using artificial intelligence for automated surgical video analysis.The proposed LRYGB ontology was developed by the surgical staff of the Department of Digestive and Endocrine Surgery at Nouvel Hôpital Civil , France . Based oTwo board certified visceral surgeons (referred to as raters) with over 10 years of clinical expertise applied the proposed ontology to two datasets. (1) The StraBypass40 dataset consists of 40 LRYGB videos recorded at Nouvel Hôpital Civil, University Hospital of Strasbourg, France [Inter-rater reliability (inter-RR) defines the extent of agreement among observers, whereas intra-rater reliability (intra-RR) defines the consistency of observations of a given observer over time. To assess the inter-RR of the proposed LRYGB ontology ten randomly chosen videos of the StraBypass40 and BernBypass70 datasets were annotated according to the step and phase definitions as provided in the Supplementary Material  by both raters using the in-house video annotation tool MOSaiC. The annotations of both raters were compared. To assess the intra-RR of the proposed LRYGB ontology ten randomly chosen videos were annotated a second time by the same rater after a wash out phase of 1 month. The two sets of annotations were compared.Inter- and intra-RR was calculated using accuracy, precision, recall and F1-scores. Accuracy is the proportion of correct predictions among the total number of observations. Precision is the proportion of true positives among all  positives and referred to as the positive predictive value. Recall is the proportion of true positives among all relevant observations  and referred to as sensitivity. F1-score is the harmonic mean of precision and recall and is a measure of accuracy.Furthermore, average transitional delay, noise level and a coefficient of transitional moments were calculated to apply application dependent metrics as proposed in . Every tThe comparison of two sets of annotations is not symmetric. In the validation of computer algorithms, the human annotation always serves as ground truth. Given that this study compares two set of human annotations, each set was treated once as ground truth and metrics were averaged across both comparisons. All metrics were applied for every video separately on phases and steps on a millisecond level and averaged across datasets.For StraBypass40 the mean ± SD video duration was 108 ± 33 min An average LRYGB video consisted of 10 phases and 33 steps.For BernBypass70 the mean ± SD video duration was 75 ± 21 min An average LRYGB video consisted of 8 phases and 27 steps.Average phase and step durations in the StraBypass40 and BernBypass70 datasets are displayed in Fig. Quantitative inter-RR results for StraBypass40, BernBypass70 and overall inter-RR results are shown in Table Across datasets inter-RR and intra-RR metrics show better results for phase compared to step recognition. Furthermore, inter-RR metrics on StraBypass40 show better results than on BernBypass70. The application dependent metrics show a 0.4–1.2% boost compared to the classical metrics.The excellent inter-RR of 95.9% for phases and 80.8% for steps of the proposed ontology demonstrate its easy application and reliable use for the annotation of LRYGB phases and steps by multiple raters in multiple institutions. Moreover, the excellent intra-RR of 98.4% for phases and 88.1% for steps shows that annotations of the same rater are consistent over time.Given these tremendous results, we advocate for the routine use of the proposed ontology in LRYGB. This will standardize video analysis of LRYGB surgery and will allow comparison of surgical workflows across surgeons and centers. The routine use of this ontology facilitates standardized video review for educational purposes, performance assessment, and quality improvement programs. Furthermore, it enables downstream applications as the training of artificial intelligence algorithms to automatically recognize phases and steps, to give intraoperative feedback or assistance.For laparoscopic cholecystectomy, one of the most analyzed surgeries in SDS, a systematic review identified 8 different phase definitions in the literature . MultiplInter-RR and intra-RR metrics are higher for phase compared to step annotation. Comprising 12 phases and 46 steps the proposed ontology is less granular on the phase than on the step level. This leads to lower variability in phase compared to step annotations. Therefore, the ontology performs better in terms of inter- and intra-RR on a phase than on a step level., there is a considerable difference in average video duration (108 vs. 75 min). This is also reflected by the greater average number of phases and steps in StraBypass40 compared to BernBypass70 . In StraBypass40 the creation of the gastric pouch  and the creation of the jejunojejunal anastomosis  takes considerably longer when compared to BernBypass70. The main differences in surgical technique between datasets are the routine division of the omentum , Petersen space  and mesenteric defect closure  in StraBypass40 compared to BernBypass70.As the two datasets are from different institutions and therefore represent different surgical techniques, the aim of this study is not to compare them. However, to understand the performance difference of the proposed ontology on StraBypass40 and BernBypass70 it is crucial to elaborate, how they differ. When comparing StraBypass40 with BernBypass70Inter-RR metrics on StraBypass40 show better results when compared to BernBypass70. This is likely an effect of the difference in average video duration between datasets. Given the same number of phase and step transitions, the longer a video is, the less the metrics are influenced by a single transitional delay.Using application dependent metrics, a 0.4–1.4% boost in accuracy, precision, recall and F1-score can be observed. This is due to the relaxation of transitional moments by extension of the acceptable transitional delay. Setting the transitional delay threshold allows to tailor the metrics application dependent to the desired use case. To estimate the remaining time of an intervention based on a phase and step recognition algorithm a 5 s delay is reasonable. However, for real-time intraoperative decision support 5 s delay are too long and will limit the acceptance of the application. Considering the first use case, in this study the transitional delay threshold was set at 5 s to calculate application dependent metrics as proposed in .Despite being multicentric this study includes only two raters and two institutions. As surgical video annotation is time consuming and needs domain expertise, it is expensive. Therefore, annotation resources must be attributed carefully.The proposed ontology shows an excellent inter- and intra-RR and should therefore be implemented routinely in phase and step annotation of LRYGB videos. This will facilitate education, performance assessment, and quality improvement programs. Moreover, the application of the proposed ontology will enable the development of downstream tasks as automated phase and step recognition, intraoperative feedback, or assistance by use of artificial intelligence.Supplementary file1 (PDF 130 kb)Below is the link to the electronic supplementary material.


Games have been a part of human life since ancient times and are taught to children and adults who want to simultaneously have fun and learn. Nevertheless, in the third decade of this century, technology invites us to consider using video games to learn topics such as entrepreneurship. However, developing a serious game (SG) is difficult because everyone who forms part of the game development team requires adequate learning resources to acquire the necessary information and improve their game development skills.This work aimed to detail the experience gained in developing ATIC , an SG proposed for teaching and learning entrepreneurship.To develop a videogame, first, we established a game development team formed by professors, professionals, and students who have different roles in this project. Scrum was adopted as a project management method. To create concept art for the video game, designers collected ideas from various games, known as “getting references.” In contrast, narratology considers the life of a recent university graduate immersed in real life, considering locations, characteristics, and representative characters from an essential city of EcuadorIn a Unity 3D video game in ATIC, the life of a university student who graduates and ventures into a world full of opportunities, barriers, and risks, where the player needs to make decisions, is simulated. The art of this video game, including sounds and music, is based on the landscape and characteristics of and characters from Cuenca, Ecuador. The game aims to teach entrepreneurs the mechanisms and processes to form their businesses. Thus, we developed the following elements of an SG: (1) world, (2) objects, (3) agents, and (4) events.The narrative, mechanics, and art of video games are relevant. However, project management tools such as leaderboards and appointments are crucial to influencing individuals’ decision to continue to play, or not play, an SG. Developing a serious video game is not an easy task. It was essential to consider many factors, such as the video game audience, needs of learning, context, similarities with the real world, narrative, game mechanics, game art, and game sounds. However, overall, the primary purpose of a serious video game is to transmit knowledge in a fun way and to give adequate and timely feedback to the gamer. Finally, nothing is possible if the members of game development team are not satisfied with the project and not clear about their roles. Entrepreneurship has become a crucial economic activity because it is a pathway to escape poverty and revitalize stagnant economies ,2. HowevAs described by Antonaci et al  and BellEducation to teach the processes inherent to innovation and entrepreneurship is vital in professional training, which have an impact on macroeconomic variables such as economic growth and the development of an economy . SDG 17 This paper aims to detail the experience of constructing ATIC , an SG for entrepreneurship learning that considers the Latin American environment. ATIC is the phonetic equivalent of “atiq,” which means “the one who can do everything” in Quechua—the language spoken by the Inca people and the native language that has spread the most throughout Latin America. ATIC allows the player to gain experience evaluating future scenarios and their implications, a topic related to anticipatory competence. This study aims to solve the research question of what characteristics an SG should have for people to learn entrepreneurship.The most relevant portions of this work include presenting the results of applying a methodology that describes the experience of forming a work team of professors and students; the roles and responsibilities of each team member; and the elements considered in constructing this SG, including the narrative and game mechanics. A simulated virtual environment such as that of a video game provides controlled simulation, making it advantageous for learning entrepreneurship.Methods section presents the methodology used to obtain the results, which are presented in the Results section. Subsequently, we present the Discussion section, in which, among other things, we address the implications of the findings and possible directions for future research.This paper is structured as follows. The next subsection provides state-of-the-art related works, with the application and use of procedures for developing video games in the educational context. The The video game industry is an ever-growing sector, with a projected global market size of US $293.6 billion by 2027 ,29. GameSoftware development is a specialized field  encompasISO/IEC 12207  and ISO/Developing specific SGs, such as educational games, presents unique challenges . Short SAcción Capital, an SG aimed at teaching key concepts about entrepreneurship using the design thinking methodology. Finally, Ahmad et al ) at the University of Azuay in Cuenca, Azuay, Ecuador. The program includes the following macrotopics: (1) the history of video games; (2) gamification; (3) preproduction techniques; (4) game production; (5) storytelling, which includes narrative creation; (6) video game design (first level); and (7) video game art.The game development team is essential ,53. As sIn addition, the following individual courses, as part of the training process, were proposed: (1) narrative creation, (2) digital 3D object development, and (3) Unity 3D video game engine. The first 2 were proposed and to be taught by professors, and the last was proposed by a student.To develop a video game, it is necessary to establish some roles in the preproduction phase. In this case, the following roles were set up: (1) the game designer, who assumes the responsibility of defining the user experience in the game; (2) the game developer: in this case, the students, whose responsibility is to write the video game code; (3) the game producer, who is responsible for the video game marketing; (4) the game administrator, who is responsible for the data administration and project continuity; (5) the game artist: in this case, 1 professor, 1 designer, and the students who have participated in the digital 3D design course; and (6) the game writer, who is responsible for creating the narrative. A total of 57 persons are part of the game development team. The scrum agile method and Milanote were used for project management, with 1-week sprints and effective communication through Milanote’s easy-to-use visual board tool. Milanote also enhanced quality assurance ,59.In developing a video game, the visual aspect is critical for creating an immersive and visually appealing experience. Ambrose et al  discuss To create concept art for the video game, designers collected ideas from various games, known as “getting references.” Three reference styles were obtained from Firewatch , FIFA 20The aggregated graphic line responds to simplified morphologies with rounded edges dominated by vertical lines. It reflects a language related to constant movement and actions. The selected aesthetic responded to dynamic geometries and proportions, adapting to the scale of the characters and scenarios. In addition, some elements were rescaled to build visual hierarchies and guide the observer’s gaze toward more profound, eloquent, and poignant messages. This approach helps create a visually immersive and appealing experience for the player.Musicians have 2 primary roles in video game development: to create sound and musical effects for video games and to apply sound and music creation tools to video game development . Early tAccording to Rone , the musThe background sounds proposed for ATIC include the noises of the city of Cuenca, a space where entrepreneurs will develop their business activities. An omnidirectional microphone was used to capture the contextual sounds, which were then corrected using Adobe After Effects.In narratology, the narrative comprises the story, a series of related events, and the discourse ,68, simiThe narrative designer must consider the game’s initial aspects in an educational context. Students are provided with clarity on the main focus of the SGs, giving them a clear starting point and means to structure their game ,73. The proactive, persistent, risk taker, planner, persuasive, and safe [The narrative writer presents the first-level narrative of an entrepreneurship video game. At the beginning of ATIC, the player must answer 20 questions to set the avatar’s profile, enemies, and challenges. The identified entrepreneurial characteristics in the avatar’s personality test are and safe ,78. The and safe . The narThe work of Brooks and Brooks  discusseAccording to DuBravac , the achTo collect ideas for modifiers, the ATIC working team played various games, including (1) The Lemonade Business ; (2) Gradrag and drop interface to facilitate game development using a set of building blocks , assist objects ,92. ATIC objects . HoweverThus, Unity 3D was considered the best engine for this project. One of the reasons for selecting it, as described by Haas , is thatTechnological media is another component to consider. According to Alonso-Díaz et al , PCs areVideo games can be a powerful tool when used in the correct context, and they have proven to be an effective way to engage users in areas that traditional methods cannot, making things more fun, unique, competitive, and interesting . AlthougAccording to Ecuador’s Organic Law of Personal Data Protection , personaIn the design of a video game, some aspects, such as persuasion and push techniques, influence and change attitudes and behaviors. Here, ethics becomes relevant because it should be considered in the persuasive aspect of the design. Some levels of transparency are required. The users must be informed about the persuasive intent of the technology . AccordiThis is a project of the CIIDTAE laboratory at the University of Azuay. All financial resources were provided by the research department of the University of Azuay. The budget for developing this video game provides for (1) developers’ salaries, (2) designers’ salaries, (3) researchers’ salaries, (4) development licenses, and (5) publication fees for scientific events. Publication fees were sponsored by the University of Pablo de Olavide.Some requirements had to be met to formalize the research project: (1) compliance with ≥1 objectives of Ecuador’s National Development Plan 2019-2021 , (2) comThis section presents the results of the experience of designing and constructing ATIC, considering the video game graphic art, sound and effects, narrative, game mechanics, and resource and budget planning.Methods section, 57 persons are part of the game development team. According to Aleem et al [As described in the em et al , (1) gamIn the concept design stage, the designers made sketches using paper and pencil, inspired by some characteristics of Cuenca city, such as the city’s colonial buildings or well-known personages. As for the selected colors, unsaturated tones are preferred, allowing color palettes with harmonious relationships to be used. In some cases, incorporating complementary colors facilitates the contrast among the navigation interfaces concerning the backgrounds.Different graphic decisions allowed us to establish a contemporary graphic line according to the aesthetics of some video games related to the current scenario. The geometry of the environments abstracts and stylizes the main features of Cuenca, allowing its identification through a quick and simplified reading of some of the most emblematic buildings and incorporating some of the city’s iconic characters.Here, the visual system allows for establishing relevance and identification, thanks to the features that determine the typology of the buildings and the characters. Next, we proceeded to model the wireframes that will allow navigation in the video game, providing easy-access scenarios and visual cleanliness in the selection and access to different game sections. The video game logo manifests itself as an identifier that simplifies, through its geometry, the most significant elements included in an enterprise that are daily recognized and associated with objects belonging to the workplace and businesses.In ATIC, players can select avatars representing different ethnic groups in the region, enhancing their identification and representation. The character sheet identifies each avatar’s physical, psychological, and behavioral characteristics. Unlike the work done by Jauregui Sánchez et al , in whicThe sound effects for ATIC are based on background noises from the historic center of Cuenca, including (1) the murmur of people buying roses on the square near the Old Cathedral, (2) the sound of the tram as it makes its way through the city, (3) birds singing in the central park in front of the Old Cathedral, (4) the noises along the ravine of Tomebamba river, and (5) some music from street musicians. Other sound effects, such as those associated with coins, drinks, payments, scratches, and car beeps, were sourced from the Open Game Art website .Regarding music, the player can select different styles of music, an option similar to that of the Grand Theft Auto game. In the case of ATIC, music was performed by 2 researchers and 2 students, who demonstrated their ability in this artistic field.An SG is designed for a primary purpose beyond pure entertainment, and this influence on players’ cognitive, emotional, and social domains increases learners’ motivation and engagement . In theiConsidering the relevance of the narrative, as described by Khaled and Vasalou , the parOne crucial lesson about designing for participatory transformative experiences that emerged from the work by Tanenbaum  is the iThe gameplay in ATIC is based on a linear narrative with ramifications. The player has several options to choose from, and their choice influences the game’s story, causing the messages and challenges to vary depending on the decision made. However, although the endings could be different, the main story is the same for all players. In addition to providing greater dynamism to the gameplay, the insertion of cinematics and microgames within the main story offers real learning of the conditioning factors of each entrepreneurial scenario in the local environment, generating informative content. It becomes an added value to this simple entertainment.The scenic environment consists of three scenarios: (1) the entrepreneur has to go looking for seed capital and perhaps work for it or do something to obtain it; (2) the entrepreneur hails from a wealthy family and probably does not need to go to a bank to withdraw funds or work to obtain the seed capital; and (3) the entrepreneur starts with a small sum as seed capital and needs a bank loan to obtain the rest of the capital.In this first release of the game, there are three business scenarios: (1) sell fast food as an itinerant seller, walking the streets in search of customers to discover their preferences and the areas of most significant impact so that at the end of the level players have gained experience in terms of the future of the business; (2) start a street food cart, commonly found at massive affluence events such as music festivals, allowing the entrepreneur to gain more experience and prestige; and (3) start a restaurant, in which food industry knowledge is essential. The narrative is presented in The narrative in ATIC considers the following: whereas the narrative framework focuses on the reasons given for the player’s actions within the game’s story, the conditions of the upgrade mechanisms make how well the player plays more critical within the game; by contrast, if the narrative framework is more effective than the upgrade mechanisms, players will be directed toward helping others or taking more responsible action to achieve a goal or task .The development of video games and software applications share similar traits, such as improving user experience, and an affective design is a critical aspect of video game design to evoke emotional responses and interactions . The gamIn ATIC, the player wins the game when he has managed to exceed a set goal, which is made up of a set amount of money, a positive reputation, and a market share achieved. The initial values are established by the system based on the type of entrepreneurial personality identified at the time of creating the avatar. However, these values can be adjusted by the player or the entrepreneurship class teacher who wants to assess the knowledge acquired in the subject.The video game constructed by Jauregui Sánchez et al  is develAs described previously, ATIC was developed in Unity 3D. The reasons for using this platform are as follows: (1) it provides a better user experience, (2) it allows scaling to ubiquitous technologies such as virtual reality, (3) it allows the exporting of the video game to multiple platforms, (4) it allows the collection of crash and exception reports as well as user feedback so that players can better diagnose issues, and (5) it has the support of a prominent developer community .ATIC incorporates the following elements to engage the player and make the learning process more enjoyable and effective: (1) a well-designed storyline relatable to real-life scenarios with clear objectives and challenges to overcome ; (2) busThis project complies with the following objectives of Ecuador’s National Development Plan 2019-2021: (1) to guarantee a dignified life with equal opportunities for all people and (2) to promote productivity and competitiveness for economically sustainable growth in a redistributive and supportive manner. In addition, this project aligns with the following SDGs : (1) quaThe effort in this investigation, which was conducted through the partnership between the University of Pablo de Olavide and the University of Azuay, was relevant to obtain the final product. Furthermore, regarding this point, several options for linking with the community will be presented through chambers of commerce, nonprofit foundations, municipal departments, and entrepreneurship promoters.ATIC could contribute to educators and the entrepreneurship academic curricula by (1) complementing the academic curricula  because Here, we discuss various business simulation games and their effectiveness in teaching business concepts and strategies. A recent review mentions that an ideal business game must consider combining the following analytical tools: (1) Heptalysis; (2) CATWOE; (3) VPEC-T; (4) SCRS; (5) MoSCoW; (6) MOST; (7) failure mode and effects analysis; (8) fault tree analysis; and (9) Ishikawa’s fishbone diagram, which present different perspectives across business areas . TransfoATIC considers not only the urban context but also ethnicities, native language, and gender aspects. ATIC is a platform game and will consider the rural context in the future but currently focuses on social entrepreneurship. On the basis of the notion that platform video games and role-playing video games are a genre of video games that is characterized by having to perform an action on a series of platforms and cliffs, facing challenges while collecting objects to complete the game  and immeSimulation games have shown promising results in teaching business concepts and strategies. Further research could investigate the effectiveness of ATIC and other simulation games in teaching business skills and fostering entrepreneurship.At present, only the alpha version of ATIC is available, and it does not allow multiple players. This feature will be considered in the next project stage. In addition, ATIC will be brought into virtual reality for interaction in metaverse environments in future stages. The work described in this paper is related to the application of this video game in the classrooms of the University of Azuay to identify the entrepreneurial student’s behavior concerning this gamified scenario. The user interaction results will be described in the next project stage.The ATIC video game is the result of research conducted in response to the question of how best to teach entrepreneurship skills in a confined environment, caused by the COVID-19 pandemic in this case. Keeping in mind that the objective of this work is to describe the experience gained in the development of ATIC, the following paragraphs reflect our conclusions.Regarding recent work trends and economic growth, de la Torre et al  suggest Developing a video game is not an easy task. As seen in this work, it is essential to be clear about the game’s objective from the beginning. Next, the selection of the game development team from among those eligible will depend on the skills they have to be able to play the different roles allotted to them in the project. In addition, it is relevant to consider that developing a business simulation video game, although it is software, involves elements that developing business software does not involve, such as narrative creation.The narrative is crucial in a video game, underlying the interaction. Here is where game mechanics becomes essential. The game mechanics design must include the following elements: achievement, appointment, rewards, leaderboards, privacy, social engagement loops , and modifiers. Other elements, such as immediate feedback, points and leaderboards, narratives, missions, quests, competition, and compensation, become part of the mechanics.With regard to the conception of the immersion through microgames provides natural and necessary information for starting a business in Cuenca. In the same way, the use of locations and characters representative of this city allows the player to identify with the environment and enables them to contemplate undertaking a business venture in real life. Furthermore, the reward is reflected in the consumers’ response, the profits obtained, and the challenges overcome that will allow the player to advance in level and meet the proposed objectives.bank account of the entrepreneur in the game.For game continuity to be maintained, events such as waiting for customer requirements that have not been met will appear as part of the appointment mechanism on the platform, enabling daily contact with the game. Another way to maintain continuity is by generating income in the This project aligns with SDGs 4, 8, and 17. First, with regard to SDG 4, through this game, people can try their hand at entrepreneurship several times until they achieve the objective, helping them to rise out of poverty. Second, in alignment with SDG 8, ATIC helps to develop strategic problem-solving as a key competence for sustainability. Third, the partnership between the University of Pablo de Olavide University, Spain, and the University of Azuay, Ecuador, aligns with SDG 17.Finally, the development goal of the game has been achieved. A committed team has produced the narrative, the game mechanics, and the other components, as explained in this paper. Although there are some details that need to be refined, it is expected that the beta version of ATIC will be available in October 2023 to continue the investigation around a general question: is it possible to learn to undertake a business venture in Ecuador through SGs?


The leaky pipeline is a visual metaphor for the under-representation of women in leadership positions in Academic Psychiatry despite their over-representation in medical schools, residency programs, and junior academic positions. The presentation focuses on the key obstacles that pertain to personal and societal attitudes and institutional barriers and proposes empirically tested and pragmatic solutions.None Declared


A signal representation based on the cumulants, however, suffers from a finite convergence radius and cannot represent the ‘localization regime' characterized by a stretched exponential decay that emerges at large gradient strengths. Here, we propose a new representation for the diffusion MR signal. Our method provides not only a robust estimate of the first three cumulants but also a meaningful extrapolation of the entire signal decay.Diffusion MR is sensitive to the microstructural features of a sample. Fine-scale characteristics can be probed by employing strong diffusion gradients while the low  Most commonly, this is accomplished by introducing a pair of diffusion sensitizing gradients   . To overcome this limitation, Özarslan et al. , is described by a stretched exponential function E(b) ∝ e−(bD)1/3, which is the characteristic feature of the localization regime . To represent the signal attenuation, we propose the following expressionWe consider acquisitions featuring diffusion gradients applied either in the same direction (single-direction acquisition), or in different directions uniformly-distributed on the surface of the sphere, where orientational-averaging is to be employed over each r, u, v, w, and p are the parameters of the representation. The asymptotic (large-q) behavior of this function is governed by the desired stretched exponential decay sincewhere We note that the asymptotic behavior (of an expression) becomes increasingly accurate as a variable approaches a limit, usually infinity.q-values, the function is given byAt small where the argument of q behavior is consistent with DKI while the representation exhibits the stretched exponential decay at large-q, characteristic of the localization regime. Thus, our signal representation works at both low and high b-values. The parameters r, u, v, w, and p provide this flexibility for our representation. In the given signal representation, r determines the asymptotic behavior. For a given r, u determines the 2nd cumulant, v determines the 4th cumulant, and w is related to higher order cumulants.This small-r, u, v, w, and p were obtained by solving the nonlinear optimization using the trust-region algorithm implemented in MATLAB .In this study, the representation in Equation (1) is fitted to dMRI signals for two scenarios, namely, to orientationally-averaged signal  is given by:D is the apparent diffusion coefficient and K is the mean kurtosis. Comparing Equations (3) and (4), we have:where andThe experimental data representing gas diffusion with hyperpolarized xenon-129 continuously flowing through different geometries of three physical phantoms were used to validate the proposal: (1) parallel plates separated by a distance of 3 mm  along  gradient directions, respectively . Two acquisition protocols were used: (1) the first protocol comprised six r et al. . Real-var et al. , Gibbs rr et al. , eddy cur et al. . The orib = 0 and 10 non-zero shells at , Δ/δ = 22/11 ms, and (2) two b = 0 and 10 non-zero shells at , Δ/δ = 24/13 ms. In both cases, the data was acquired along 32 gradient directions per shell with a 2 mm isotropic voxel size, TE/TR = 62/5,200 ms and partial Fourier factor of 6/8.Additionally, the data points from Figure 3 in Lee et al.  represenE) ∝ q2/3).q-values, while our representation fixes this problem.The result of fitting the proposed representation to data from cortical gray matter in Lee et al.  is illusr, u, v, w, and p) of the representation for both orientationally-averaged signal and single-direction signals. No significant correlation between the estimated parameter r and the other parameters of the representation is seen in the scatter plots in r map from the brain image is also smooth.q approximation of our proposed representation. A good agreement between the maps from DKI and our proposed method is observed as illustrated in We extracted the apparent diffusion coefficient and kurtosis using DKI and the low-b-values) while only a few studies investigated the localization regime theoretically   and a Wellcome Trust Strategic Award (104943/Z/14/Z), the Swedish Foundation for Strategic Research (RMX18-0056), Linköping University Center for Industrial Information Technology (CENIIT), Sweden's Innovation Agency (VINNOVA) ASSIST, and Analytic Imaging Diagnostic Arena (AIDA). This work was also supported by the British Heart Foundation, UK (SI/14/1/30718), EPSRC (EP/M029778/1), and The Wolfson Foundation. TP acknowledges the Polish National Agency for Academic Exchange for the grant PPN/BEK/2019/1/00421 under the Bekker programme and the Ministry of Science and Higher Education (Poland) under the scholarship for outstanding young scientists (692/STYP/13/2018).EÖ is a shareholder of Spin Nord AB. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.


Chronic kidney disease (CKD) is seen as a diverse disease and a primary contributor to global mortality. Malnutrition arises within chronic illness, which involves protein energy depletion and inadequate levels of essential nutrients. These factors increase the likelihood of death and the overall impact of the disease on affected individuals. Consequently, this study aims to utilize bibliometric and visual analysis to assess the current state of research, the latest advances and emerging patterns in the fields of CKD and malnutrition.Extensive research was conducted using the Scopus database, which is the most authoritative database of research publications and citations, to focus on CKD research between 2003 and 2022, as indicated by title and author keywords. Then, within this vast collection of academic publications, a notable subset of articles was exclusively dedicated to investigating the relationship between CKD and malnutrition. Finally, we performed bibliometric analysis and visualization using VOSviewer 1.6.19 and Microsoft Excel 2013.n = 173; 21.02%), followed by Italy , Sweden , Brazil  and China . The most common terms on the map include those related to the topic of (a) malnutrition in hemodialysis patients and predicting factors; terms associated with the (b) impact of malnutrition on cardiovascular risk and complications in CKD patients; and terms related to the (c) dietary protein intake and malnutrition in CKD.Large global research between 2003 and 2022 resulted in 50,588 documents focused on CKD, as indicated by title and author keywords. In this extensive collection of scientific publications, a staggering portion of 823 articles is devoted exclusively to investigating the link between CKD and malnutrition. Further analysis reveals that this body of work consists of 565 articles (68.65%), 221 reviews (26.85%), and 37 miscellaneous entries (4.50%), which encompass letters and editorials. The USA was found to be the most productive country (This study is the first of its kind to analyze CKD and malnutrition research using data from Scopus for visualization and network mapping. Recent trends indicate an increasing focus on protein-energy wasting/malnutrition in hemodialysis patients and predicting factors, dietary protein intake, and malnutrition in CKD. These topics have gained significant attention and reflect the latest scientific advances. Intervention studies are crucial to examining diet therapy's impact on patients with stages 1 to 5 CKD. We hope this study will offer researchers, dietitians and nephrologists valuable information. Chronic kidney disease (CKD), frequently referred to as kidney failure, is a potentially fatal condition that has emerged as the primary global cause of mortality , 2. The The chronicity criterion for diagnosing CKD requires repeated measurements of low estimated glomerular filtration rate (eGFR) or elevated urine albumin over at least 90 days. CKD stages 3–5 are typically determined by eGFR, while studies combining albuminuria and decreased eGFR report stages 1–5 , 9. The Early detection and treatment might help prevent kidney disease from progressing to the last stage of kidney failure, especially in low- and middle-income countries where CKD represents a heavy burden, as they are the least equipped countries to deal with its consequences. Treatments for CKD can delay progression, stop the onset, reduce complications from GFR decline, lower the risk of cardiovascular disease, increase survival rates, and improve the quality of life .Nutrition and nutritional status are essential for health, survival, quality of life, and many other aspects of daily living. On the other hand, malnutrition is a serious complication of kidney disease due to disease burden and inflammation linked to higher morbidity, death, and expenses. CKD patients are at risk of metabolic and nutritional problems that increase energy expenditure, encourage protein catabolism, reduce appetite and food intake, and result in a condition known as protein energy wasting , 13.The Global Leadership Initiative on Malnutrition (GLIM) proposed a combination of three phenotypic criteria , and reduced muscle mass) and two etiologic criteria  as a consensus, thorough, and workable procedure to diagnose malnutrition in clinical and research settings. When the patient meets at least one combined phenotypic and etiologic criterion, malnutrition is diagnosed . Most paBody weight and BMI are also influenced, as many kidney failure patients suffer from obese sarcopenia that may be related to inflammation . As a reMany pharmacological treatments are often utilized to reduce the occurrence or seriousness of comorbidities in CKD. However, these treatments have substantial side effects and high costs . In conthttp://www.scopus.com. The authors have reviewed the PRISMA 2020 Checklist, and the manuscript was composed and revised in accordance with the PRISMA 2020 Checklist [We performed a web-based bibliometric investigation utilizing the Scopus database provided by Elsevier and can be accessed at hecklist .Scopus, the most reputable scientific research database, provided our data. Scopus provides the most important data sources for bibliometric analysis and lets scholars create thorough search queries utilizing various inquiry methods –28. It aAll information was taken from Scopus on June 2, 2023, to avoid any potential bias brought on by the periodic database updates of Scopus.The relevant terms related to CKD and malnutrition were identified using PubMed Medical Subject Headings (MeSH) and by consulting pertinent publications on this subject –41. ThesThe study focused on terms such as 'Malnutrition' or 'Nutritional Deficienc*' or 'Undernutrition' or 'Malnourishment', which directly addressed the issue of malnutrition itself rather than other associated terminologies such as specific micronutrient deficiencies. Similarly, the study employed keywords such as 'Chronic Renal Insufficienc*' or 'Chronic Kidney Disease*' or 'Chronic Kidney Disease*' or 'Chronic Kidney injury' or 'CKD' or 'Chronic Renal Disease*' or 'Chronic Kidney Insufficienc*' or 'Chronic Renal Injury', which specifically pertained to 'Chronic Kidney Disease' itself rather than other related terms such as specific kidney diseases.The terms were used to search titles, abstracts and author keywords to optimize document retrieval. Asterisks and quotation marks were utilized in the search string to simplify and enhance the accuracy of the search. The search strategy focused exclusively on publications published in peer-reviewed scientific journals, excluding erratum and retraction documents.This study used data from Scopus published between January 1, 2003, and December 31, 2022. The last two decades were chosen because they are assumed to provide a better picture of the pattern of publications and citations received. Additionally, this time range allows for comparison of earlier and more recent periods to identify shifts, growth, or declines in research output and impact. By limiting the study to the years 2003–2022, the researchers were able to capture the most recent trends in the research on malnutrition in CKD. In addition, this time range encompasses a significant and crucial phase of research development in the field. For example, research on malnutrition in patients with chronic kidney disease gained notable attention and development after 2000 despite SGA being first introduced in CKD in 1987 , 43.A random sample of 50 articles was chosen from the retrieved literature. Only articles with even numbers  were included in the sample until the end of the retrieved document list. This selection was made to ensure the results' validity and prevent false-positive results. The top 50 articles cited in the retrieved dataset were also reviewed to further validate the accuracy of the results. It is important to mention that the validation approach used in this study was previously employed by Sweileh et al. –46 and ZTo examine bibliometric indicators, we transferred the collected data to Microsoft Excel for analysis and organization. We obtained significant bibliometric data on research publications on malnutrition and chronic kidney disease, including the annual article count, types of documents, countries/regions involved, institutions, funding agencies, journals along with their impact factors (IF), citation patterns, and the h-index. Our analysis of these findings was carried out using descriptive statistics.The study focuses on analyzing the clustering and co-occurrence of terms found in the titles and abstracts of publications that address the topics of malnutrition and CKD. The research also explores the visualization of these findings through overlay visualization and investigates the collaboration network between multiple countries. VOSviewer software version 1.6.19 was utilized to conduct this analysis , 49. TheExtensive research conducted on a global scale between 2003 and 2022 has resulted in an impressive compilation of 50,588 documents focused on CKD, as indicated by titles and author keywords. Within this vast collection of scholarly publications, a notable subset of 823 articles was dedicated to investigating the relationship between CKD and malnutrition. Further analysis reveals that this body of work consists of 565 articles (68.65%), 221 reviews (26.85%), and 37 miscellaneous entries (4.50%), which encompass letters and editorials.r = 0.946, P < 0.001) was observed between publication productivity in various fields regarding CKD and productivity specifically focused on CKD and malnutrition. Initially, the number of publications experienced a slight increase from 2003 to 2017, with an annual average of fewer than 32.47 articles. However, from 2017, there was a rapid growth in articles specifically addressing CKD and malnutrition, averaging more than 67.2 publications per year. This upward trajectory peaked in 2022, reaching a total of 87 publications.Figure n = 173; 21.02%), the Republic of Italy was found to be the second most productive country with  publications, and Sweden , Brazil , China  and Spain  followed. Figure Researchers from 71 countries signed all included publications related to CKD and malnutrition. Table Karolinska Institutet in Sweden, accounting for 48 publications (5.83%). The David Geffen School of Medicine at UCLA in the USA is closely followed, with 27 publications (3.28%). The University of California, Los Angeles (USA), ranks third in production, with 26 publications (3.16%).According to the number of publications, Table National Institute of Diabetes and Digestive and Kidney Diseases , the National Institutes of Health  and the National Center for Research Resources  were found to be the most active funding agencies in this field in the United States, followed by the National Natural Science Foundation of China (China) with 13 publications (1.58%), the National Center for Advancing Translational Sciences (USA) with 11 publications (1.34%), and Coordenação de Aperfeiçoamento de Pessoal de Nvel Superior (Brazil), Vetenskapsrådet (Sweden) with 9 publications (1.09%).Table Journal of Renal Nutrition , Pediatric Nephrology , American Journal of Kidney Diseases , and Nephrology Dialysis Transplantation .Table According to the analysis of citations, the retrieved articles had an average of 28.24 citations. They achieved an h-index of 75, accumulating a total of 23,245 citations. The range of citations for these articles ranged from 0 to 1356. Of the total, 120 articles received no citations, while 48 garnered 100 or more citations. There was a wide range in the total number of citations for these publications, from 207 to 1356  encompassed terms related to the issue of “malnutrition in hemodialysis patients and predicting factors.” Cluster number 2 (green) comprised terms associated with the “impact of malnutrition on cardiovascular risk and complications in patients with CKD.” Cluster number 3 (blue) included terms related to the “dietary protein intake and malnutrition in CKD see Fig. . Before In Fig. In this study, a bibliometric analysis of malnutrition and CKD studies was performed using the Scopus global database with the help of VOSviewer software to find hotspots and emerging trends. We focus specifically on studies released between 2003 and 2022. A notable subset of 823 articles investigated the relationship between CKD and malnutrition. Several studies have shown a deadly broad area between kidney and cardiovascular disease that requires urgent and focused attention –61. The Several components can clarify the increase in the yield investigated. In general, nutrition has gained more importance in preventing and treating chronic diseases. It has attracted the attention of a multidisciplinary medical team since nutrition management using a multidisciplinary approach improves patients’ nutritional status during medical treatments . AdditioThe United States administration has adequate resources with strong collaboration and partnerships, making it the worldwide leader in food and nutritional research , 65, 66.The “impact of malnutrition on cardiovascular risk and complications in CKD” has emerged as one of the hot topics to focus on in our study. CVD is the most common type of heart disease and the leading cause of death worldwide, accounting for 45–50% of deaths among end-stage dialysis patients . CVD is In addition, patients with CKD experience uremia-specific factors such as anemia, hyperparathyroidism, carnitine deficiency, hyperhomocysteinemia, and low levels of vitamin C, which contribute further to the development of cardiovascular disease. Furthermore, hypertension, dyslipidemia, diabetes, and obesity, commonly known as causes of CKD and CVD in the general population, increase the risk of cardiovascular disease in patients with CKD. The existence and severity of ischemia observed during stress echocardiography were identified as an autonomous indicator of mortality among CKD patients, and this association persisted even after the patients received a kidney transplant .In contrast, it has not been demonstrated that adjusting hyperhomocysteinemia using pyridoxine, vitamin B12, and folic acid supplements effectively lowers cardiovascular risk in kidney patients . Another‘Malnutrition in hemodialysis patients and predicting factors’ as a theme was among the main hot topics in the current study. A significant number of individuals with advanced CKD exhibit noticeable clinical signs of protein energy malnutrition and wasting. These conditions result from decreased food consumption due to anorexia, nausea, and vomiting caused by uremic toxicity, negatively affecting both muscle stem cells and skeletal muscle. Malnutrition is also associated with infections, disturbances in hormones and body metabolism, and increased mortality from cardiovascular diseases in CKD patients . HoweverThe complex mechanisms that lead to muscle protein and fat loss can involve multiple abnormalities that stimulate protein breakdown or decrease protein synthesis . ClinicaMedical nutrition therapy (MNT) was established as a legal term to allow patients to receive personalized nutrition counseling for ‘pre-end stage renal disease’ from qualified dietitians to slow or prevent disease progression. Continued assessment of nutritional status is important and should start as early as possible to minimize uremic toxicity, correct several metabolic abnormalities and symptoms, and prevent waste and malnutrition. This can be achieved by providing adequate calories to prevent catabolism of body tissues during the patient’s hospital stay. Therefore, many studies related to nutrition care highlight inadequate nutritionist and lack of consistency in hospital access to nutrition care and the negative consequences on patients’ hospital stay and cost savings , 78.Another key topic of discussion in the current investigation was “dietary protein intake and malnutrition in CKD”. Several studies in HD patients have established that hypoalbuminemia reflects the presence of malnutrition and is considered a strong independent predictor of cardiovascular mortality. Research has also demonstrated a correlation between indications of malnutrition, especially reduced serum albumin levels, and elevated morbidity and mortality rates , 80. HowThe protein recommendations are designed to offset the reductions caused by dialysis, irregularities in protein metabolism, alterations in albumin turnover, heightened breakdown of amino acids, inflammation, and infection. The recommended protein intake for CKD patients varies depending on the treatment methods, the types of protein, grams of protein per kilogram of standard body weight, and the stage of the disease in infected patients . Thus, nThis study breaks new ground by offering an extensive overview of publications concerning CKD and malnutrition over the last two decades. As a result, researchers might use visual analysis to improve the research areas of malnutrition and development trends in malnutrition and CKD. While relying on a single database could be considered a limitation, the decision to utilize a large and comprehensive database for visualizing and mapping literature remains valid and defensible. It is important to acknowledge the potential for omitting certain articles. Therefore, the findings of this study should be interpreted in light of this potential limitation, which could impact the rankings of countries, institutions, funding agencies, and journals. Our study's objective was not to examine the difference between malnutrition risk and nutritional risk; despite the consensus around defining malnutrition, there is still no distinction between the definitions of malnutrition risk and the general name of the nutritional risk that is characterized by decreased food intake, both in terms of quantity and quality, which can lead to malnutrition over time . ConsequTo date, no comprehensive evaluation of the evidence has been carried out regarding malnutrition and CKD, and our search did not reveal any ongoing bibliometric studies on the topic. The study illuminates the correlation between malnutrition and CKD, highlighting the impact of malnutrition on cardiovascular risk and complications in patients with CKD. People with CKD often suffer from malnutrition. It can exacerbate the disease and cause additional health problems. Since 2017, the number of studies on malnutrition in CKD has increased, indicating a developing interest in this field. The USA, Italy, and Sweden are in the lead in this bibliometric investigation. Malnutrition must be diagnosed and treated promptly in patients with CKD to prevent disease progression. In managing CKD and its complications, nutrition therapy and diet interventions are crucial. The manuscript emphasizes the importance of nutritional therapy as a cost-effective and side-effect-free alternative to pharmaceutical treatments. Furthermore, it indicates a shift in research focus toward investigating malnutrition in hemodialysis patients, identifying factors associated with malnutrition, and examining the influence of malnutrition on cardiovascular risk among patients with CKD. These emerging trends highlight the need for additional research to improve our understanding and management of malnutrition in the context of CKD.Investigations of the nutrition care process of patients with chronic kidney disease from the initial phases include collaborating expert groups to minimize malnutrition and other comorbidities.Studies related to nutrition care highlight the inadequate nutritionist and lack of consistency in access to nutrition care in hospitals and the negative consequences on patients’ hospital stay and cost savings.Experimental studies should be included in the early stages of CKD in future research to obtain the maximum benefits for patients and generalize updated effective guidelines for best practices in healthcare facilities. Continued assessment of nutritional status is important to prevent waste and malnutrition.In the future, nutrition and diet therapy interventions for chronic kidney diseases should be studied. These diets must be tested for their effectiveness and long-term sustainability in prevention and treatment. In addition, nutrition that aligns with drug therapy and invasive therapy in CKD can improve and decrease the side effects of renal replacement therapy.The recent shift in research focus toward malnutrition and dietary protein intake in CKD indicates the need for additional studies on this subject. Future research should investigate the underlying mechanisms and risk factors contributing to malnutrition in patients with CKD on hemodialysis and highlight the importance of alternative measurement methods to assess nutritional status in hospitalized patients and walk-in facilities in medical centers.Interactive collaboration and knowledge sharing can improve research on malnutrition and CKD. Collaborations between researchers from different countries and disciplines can provide a broader perspective and contribute to developing strong evidence-based guidelines to minimize the negative effects on patients with chronic diseases.Research on CKD and malnutrition has many implications.


Robotic-assisted surgery (RAS) is developing an increasing role in surgical practice. Therefore, it is of the utmost importance to introduce this paradigm into surgical training programs. However, the steep learning curve of RAS remains a problem that hinders the development and widespread use of this surgical paradigm. For this reason, it is important to be able to train surgeons in the use of RAS procedures. RAS involves distinctive features that makes its learning different to other minimally invasive surgical procedures. One of these features is that the surgeons operate using a stereoscopic console. Therefore, it is necessary to perform RAS training stereoscopically. This article presents a mixed-reality (MR) tool for the stereoscopic visualization, annotation and collaborative display of RAS surgical procedures. The tool is an MR application because it can display real stereoscopic content and augment it with virtual elements (annotations) properly registered in 3D and tracked over time. This new tool allows the registration of surgical procedures, teachers (experts) and students (trainees), so that the teacher can share a set of videos with their students, annotate them with virtual information and use a shared virtual pointer with the students. The students can visualize the videos within a web environment using their personal mobile phones or a desktop stereo system. The use of the tool has been assessed by a group of 15 surgeons during a robotic-surgery master’s course. The results show that surgeons consider that this tool can be very useful in RAS training. Minimally invasive surgery (MIS) represents a fundamental paradigm shift in the field of surgery that has influenced the techniques used in almost every surgical field  and causA step forward in MIS is robotic-assisted surgery (RAS), also called Robotic Minimally Invasive Surgery (RMIS), or Robotic-Assisted Minimally Invasive Surgery (RAMIS). In this surgical paradigm, the surgeons do not directly handle the surgical tools. Instead, they control a series of robotic arms that are introduced into the patient through small incisions. RAS avoids the surgeon having to stand for a long time, avoids human hand tremor and offers surgeons the possibility of making movements that would be physically impossible if they had to hold the surgical instruments with their own hands, such as twists and full 360° turns. In addition, the surgeon obtains an enhanced perception of the surgical target as stereoscopic cameras are almost always employed in these surgical devices. Of course, the operating consoles have stereoscopic viewers that contribute to increasing the surgeon’s precision and accuracy.Two major drawbacks have emerged with the introduction of this surgical paradigm : (i) theTraining in RAS entails two fundamental aspects: skill and knowledge. Regarding the former, skill is usually trained using simulators , which aA very important question in this regard is that surgeons have a stereoscopic view of the tissues while performing the surgical procedure. This is an important difference with respect to most endoscopic surgical procedures, which are usually mono. Thus, when using surgical videos for teaching RAS, these videos should be viewed by surgeons with a stereoscopic display in their training process, so that RAS trainees can observe the procedure as if they were performing it from the control console of the surgical robot itself. Otherwise, the teaching value of the video will be reduced, since the surgeons will not be able to perceive the depth of the tissues as occurs when they use the surgical robot. To this end, special devices such as Virtual Reality glasses or Head-Mounted Displays (HMD) are necessary to allow the correct visualization of the teaching resources. In addition, raw videos need to be edited to remove/conceal uninteresting sections and highlight the key elements with annotative elements.As a way to help solve this training problem, in this article we present a new tool based on the use of the Mixed Reality (MR) paradigm for the visualization and annotation of stereoscopic videographic material associated with RAS procedures, so that surgeons can use it to explain how they perform a surgical procedure and train future RAS surgeons. The tool is interactive, highly dynamic and collaborative. It works on web technology and low-cost hardware, making it almost universally accessible, and it can reproduce the RAS videos in an MR environment, because the tool allows the surgical videos to be enhanced with virtual annotations, increasing their teaching value.The term Mixed Reality refers to those applications that allow the creation of environments in which real and virtual objects are combined within a single display. When the amount of digital information is small and used to augment real objects, it is often referred to as Augmented Reality (AR), as in –8. HowevFundamentals of Laparoscopic Surgery (FLS) [Unlike the ry (FLS) , which iry (FLS) . Howeveronly to learn how to use the robot console and its tools. Figure Regarding robotic skill training, the use of VR-based simulators to train RAS skills is not uncommon , 12, 13.Unlike MIS, where haptic feedback is essential—VR is generally unable to provide this important perceptual cue–, the loss of tactile feedback in VR-based RAS training is less of a factor, since current robotic surgery systems do not feature haptics . TherefoDry-lab models , such as pelvic dome trainers, are largely used to practice basic skills, such as suturing or dissecting. An interesting proposal in this regard is , in whicIn wet-lab models, surgical techniques are trained on cadaveric or live animals and/or on human cadavers . AlthougRegarding procedure training, the use of MR or VR in robotic surgery is also not new. There are VR-based simulators that include software modules to train specific surgical procedures, besides basic robotic skills. However, the availability of surgical procedures and the fidelity of the simulation is still limited. Therefore, the use of videos to show real surgical procedures is still important. Extending the analysis to other use cases, the spectrum of applications of MR to RAS is large, with many examples . HoweverBedside training is another important element in surgical training. However, its use in RAS as a training asset, albeit important , is limiOf course, the use of videos as a teaching resource in surgery is not new –29, and There are also several academic works describing the use of stereoscopic videos, as we propose. For instance, in  the use There are also systems designed to perform streaming of stereoscopic videos of robotic surgeries , but theAs can be seen, there are several works dealing with the use of stereoscopic RAS videos or with the use of MR in the surgical field. However, there is a research gap when it comes to the use of MR-enhanced stereoscopic videos designed for RAS training. The authors have been unable to find any work that is similar to what we propose. There is also a lack of applications dealing with virtual annotations in stereoscopic RAS videos. Although the use of annotation in the surgical field is also not uncommon –47, exisThe tool presented here is a web-based MR application. The use of web technology is justified by the need to provide universal access to the tool, since medical doctors do not want to waste time installing software suits in order to watch their videos. Web access is almost ubiquitous. Therefore, the development of a web-based tool allows teachers (experts) and students (trainees) to use the application in different locations and/or with different computing devices. The use of MR is justified by the need to enhance the videos with virtual information. They need to be shown in 3D, since RAS systems are always used with stereoscopic cameras and stereoscopic consoles. Thus, the addition of virtual 3D elements is a natural step forward. The problem is that the recorded videos lack depth information, and therefore the introduction of virtual elements which can be properly placed within the 3D space of the video is challenging.RAS teachers and RAS students. Both are expected to be surgeons, but the teachers will be expert surgeons, with vast experience in RAS procedures, who want to share their knowledge to other novel surgeons who do not have experience in RAS procedures—or at least have little or no experience in the type of procedure being taught–. Both teachers and students will access the tool by a standard user-password authentication system, but they will have different interfaces. Both types of users will have access to a list of RAS videos with different options. The students can watch the available videos with all the information, including a stereoscopic view. The teachers can do the same, but they can also edit the information shown in the videos and upload new videos. At any time, a collaborative view is possible, where the teacher controls the playback of the video, while the students watch the stereoscopic visualization. In the next sections each interface is described.Two types of users can be identified in this application: draft mode or in published mode. The students will not be able to view the interventions that are in draft mode; however, they will be able to view the published interventions. In addition, a collaborative display of the videos is possible. In this latter option, the teacher always decides and controls when the surgical procedure is shown (broadcast) to the students. To this end, each surgical procedure has a display button next to it, which creates a virtual room for students to connect and watch the intervention, enhanced with virtual content. When a room is created, a room code is generated. This code is needed for the students to access the MR-enhanced surgical video.Once the teacher has logged in, the tool presents them with the list of surgical procedures that have been stored in the system. New surgical procedures can be added, by simply uploading the stereoscopic video of the surgery. Information about date, surgical specialty, organ and technique can be supplied. Figure Before the teacher shows—using the collaborative display—or allows students to watch—if they choose to watch the videos without the collaborative mode—a surgical video, the teacher needs to have edited it after uploading it. Each surgical procedure in the intervention management interface has an edit button next to it that gives access to a wide set of functionalities to edit the video. This video editor is the authoring part of our application, since it allows the teacher to configure how the video is displayed and how much virtual content will be added to it, in order to create an MR experience.annotations , sections and the video itself, which includes buttons (from left to right) to play/pause the video, to play only the parts of the video marked as sections (and not the whole video), to restart the video, to increase the playback speed and to get a link to download the video. There is also a slider to zoom in/out the video.Figure The annotation section allows the teacher to highlight a particular element of the video with a virtual bounding box and a text. As the videos do not have depth information, the creation of this virtual annotation needs to be processed in order to calculate the proper horizontal disparity at which the virtual information will be shown in the stereoscopic frame. Otherwise, the annotation will not be perceived at the right depth and the students will not perceive the virtual annotation as part of the stereoscopic video, leading to cybersickness. Thus, the visualization of virtual annotations is carried out by using the Mixed-Reality paradigm, by which virtual elements  are correctly blended with the real images of the video. The calculation of this stereoscopic disparity is explained here .The idea behind this annotation system is that the user (teacher) chooses an element—an organ, an anatomical element, a surgical tool, etc.—that they want to highlight in order to provide an explanation. This selection is done through a bounding box. Then the user decides how long they want the element to be highlighted for, and the tool automatically tracks the annotated element throughout the video. This way the user does not need to create key-frames for the annotated element, nor do they need to reposition the virtual annotation as the annotated element moves. This annotation tracking process is explained in  and usesprocess annotation button. This process can be optionally supplied with some parameters, the most important being the update frequency. This update frequency represents how often the tracking system tries to relocate the tracked element. The higher the frequency, the more accurate tracking may be, but the more time it takes to calculate. Values between a tenth of a second and one second are reasonable for this application. The default value is 0.5 s. Figure It is important to emphasize that this annotation processing is performed asynchronously, because annotation tracking and stereo matching can potentially take some time—especially if the annotation lasts for more than a few seconds—and the application needs to continue working while the annotations are being processed. Figure On the other hand, the tool allows the teacher to create sections. Many surgical procedures can last for hours and not all the footage has teaching value, since most of the time the surgeon is performing routine work. Thus, it is important to be able to delimit the sections of the video that are of real interest. These sections can have a name and a description and, of course, a start and an end time. More importantly, the teacher or the student can later decide to play the video showing only the parts that belong to a section, skipping the rest of the video. This allows a surgical procedure lasting several hours to be visualized in a few minutes, since only the important parts will be shown. Figure All the information about the annotations and sections of the video is displayed with colored bars. Sections are portrayed in blue, and annotations are displayed in green when they have been processed, in yellow when they are still being processed and in red when they are yet to be processed. Figure Finally, as discussed at the beginning of this section, there are a series of buttons to play/pause the video, to play it at different speeds , to play only the parts of the video belonging to a section and also a button to zoom in–out in case the stereoscopic video is incorrectly aligned. There is also a button that generates a link, so that the video can be shared with a single URL and downloaded—although an authorized user is needed to be able to use this link. Figure Once the video has been edited by the teacher and enhanced with virtual content, it can be moved from draft status to published status. Only published videos can be shared and broadcast to the students, although published videos can still be further edited. Once a video is published, a display button appears next to it, allowing the teacher to create a virtual room to host the collaborative display of the video to the students.The view from the teacher’s perspective is very similar to the video editor. The only difference is that neither annotations nor sections can be modified. They can only be consulted and watched. The video controls  are the same as in the editor’s view. However, there are now two additions: one button to share a pointer and another button to display the video in VR mode . Finally, students enter the room and start visualizing the video. To visualize it in stereoscopic mode, they need to click on the glasses icon see Fig. . If theySince there are many different smartphones with a plethora of display sizes, the MR viewer that our tool provides allows the adjustment of the horizontal disparity and the zoom of the video so that a comfortable view is achieved see Fig. . Thus, iIn collaborative mode, the teacher will always be in control of the reproduction. Whatever control  they use will be translated into the students’ view. This way all the students share the same view, and the teacher can be sure that what he/she sees is what the students can see.It is important to emphasize that the students and the teacher do not need to share the same physical location. They do not even need to share a common computer network, since the only requirement is that they can establish an IP-based communication. However, it makes much more sense for them to share a common physical room. This way, the teacher can easily combine verbal instructions with visual information.The students enter the application in the same way teachers do, albeit their account does not have upload or edit permissions. Students have two major options, to watch previously uploaded videos (without any collaborative feature) or to join a room class for a collaborative display. In the former, the interface is almost the same as the one previously described for the teacher, but without the editing options. Students can see a list of available videos including a search mechanism by: organs, techniques, etc. Once a video is selected, the student can watch the video with all the related information, including sections and annotations. The reproduction options are similar to the ones previously described, so a student can watch, i.e., a specific section, an annotation or the summary of the video where only sections are played. This visualization interface also includes the stereoscopic visualization mode, so any student can use his/her own mobile device to watch the desired content in an immersive manner. In the latter case they can connect to the shared view where a teacher will control the visualization. Once the teacher casts a surgical procedure and creates a room, students can access it by typing a room code. This room code is always created by the teacher—in fact it is generated automatically by the application at the request of the teacher—and gives access to whatever content the teacher chooses to display.Our RAS training tool needs a computer infrastructure to work. The system is composed of six main modules: a web application, a REST service, an annotation-processing module, a MySQL database, a video storage server and a WebSocket service. Figure The web application provides the interface to the users. This is an HTML5-based application developed with Vue.js, a JavaScript-based framework for web application development. This web application serves as a front-end for the system, where the users perform all the actions. It is composed of several modules that provide the functionalities for login, intervention uploading, editing and visualization. For stereoscopic video visualization the Three.JS library is used. Therefore, WebGL support is needed on the browser’s side. In order to provide a full screen experience, and avoid disturbing UI elements, the Fullscreen API is also used.The REST service was developed with the Python-based Django framework. It exposes a public API, through which the web application and the rest of the modules can communicate with the database by means of HTTP requests. These requests, and the corresponding responses, use JSON format. The REST service also manages the user authentication with expiring JWT tokens. Figure The REST service module communicates with the annotation-processing module whenever an annotation is created and requested to be processed. This module is an OpenCV-based procedure written in Python that calculates all the positions of the annotations asynchronously, so that the rest of the modules of the system can continue working regardless the completion status of this annotation process. The results of this process (the key frames of the highlighted elements with their bounding boxes and annotation texts) are sent to the REST service to be stored in the database.The WebSocket service provides real-time communication between users of the web application. This service manages the different virtual rooms and broadcasts messages from the teacher to the students. This real-time communication allows the teacher to control the video playback on the students’ devices. It also communicates the position of the shared pointer. This module is separated from the web application module and uses Node.js and Socket.IO to manage the communication.surgery, organ, technique, specialty), users and authentication management , video sections (table section) and video annotations (tables mark and keyframe).The database stores all the information about the users and the surgical interventions, so each surgeon can have a customized view of the application. Figure Although the database does store all the information about the videos, it does not store the videos themselves. Instead, it links to the video storage server where all the stereoscopic videos are saved. This server is protected with encrypted access since these videos are sensitive health data of real patients. All of them need to be stereoscopic FullHD videos, in side-by-side format. Thus, the video server stores 3840 × 1080 videos. The acquisition of the footage from the surgical robot is beyond the scope of this work, but our tool is designed to work with any surgical hardware.As aforementioned, the tool is a web-based application. It allows access using computers, tablets and mobile phones. The only requisite is to have a web browser compatible with WebGL . In fact, the expected scenario is that teachers use a desktop computer to enter the application, edit the videos, add virtual information to them and broadcast the MR-based teaching assets, while the students use their mobile phones to watch the training materials created by the teachers. This is possible because of the use of the WebSocket-based communication system by which, any person with an IP-capable device and a compatible web browser can connect to the teacher’s room.This is ideal for the teachers, as they do not need a dedicated computer—with specific software—to run the application. However, the key advantage of this web-based infrastructure is that it allows students to view the training videos edited with this tool on almost any device, including their own mobile phones. This is of the utmost importance, since most professional MR-based medical tools are developed for expensive devices, such as Head-Mounted Displays or stereoscopic displays. Our tool could be used with a simple smartphone. All the students need is a mobile phone with a web-browser and Cardboard VR glasses. These VR glasses can be bought for less than one dollar. Figure Once the tool was ready for deployment, a quantitative subjective evaluation with real users was performed. To this end we deployed all the back-end modules of our application in one of the servers in our university and arranged a test session at the Hospital General de Valencia, where a simple laptop computer was used to display the web application. This hospital is one of the oldest hospitals in Spain, and it is one of the reference centers in robotic surgery in Spain. The experiments took place as part of a master's session in robotic surgery, since this hospital offers a Master’s Degree in robotic surgery. The students on this master’s course are surgeons who want to be trained in robotic surgery. Therefore, it is the perfect place and population to test our application. Fifteen people participated in the experiment, which was completely voluntary. There was no financial compensation for participation and the experiment lasted for one hour.For the experiment, a teacher, who is an expert in RAS, used our tool to show a surgical intervention. Previously, this person had uploaded the video of the surgical procedure and had edited it, so that the most important parts of the surgery were explained to the students. The intervention was a robotic resection of a mediastinal cyst in the thoracic outlet, in the specialty of Thoracic Surgery, performed with a Da Vinci surgical robot.Start of the surgical procedure indicates the beginning of the surgery, after port placement and docking. You can see the mobilization of the lung parenchyma to achieve proper exposure of the lesion at the apex.Opening of the mediastinal pleura once the cyst and the surrounding anatomical structures have been identified, the mediastinal pleura is incised over it to begin the resection.Continuity with the sympathetic chain it can be seen how the cyst seems to depend on the sympathetic chain. This indicates that the only way to perform a complete resection of the cyst would be to section the sympathetic chain at that level.Rib release adhesions from the lower pole of the cyst to the rib are released.Three annotations were also created: these three marks indicate anatomical structures that are particularly important in this surgery.Subclavian artery this is a large caliber artery. Its injury can cause massive bleeding and be a life-threatening emergency.Cyst this is the target lesion.Sympathetic chain At the level of T1 (where the cyst is located), injury to the sympathetic chain, or the stellate ganglion, should be avoided due to its secondary effects such as Horner´s Syndrome .Four sections were created in this video:Figure strongly disagree (1), disagree (2), somewhat disagree (3), neutral (4), somewhat agree (5), agree (6) and strongly agree (7). These questions try to measure if the participants feel that the elements of the tool are useful as training assets. Thus, some questions address the use of annotations, some deal with the sections, some with the pointer, and some are general questions about the usefulness of the tool as a RAS training device. The last question is a 0–10 question to rate overall experience with the tool.After a brief introduction to the tool, the teacher created a virtual room to show the previously MR-enhanced video. Then, we handed out smartphones to the students so they could use them to connect to the room. They tested the application as they followed the teacher’s explanations. Figure Since the evaluation was conducted with students, they only assessed the resulting video and the use the teacher made of the application. Therefore, this evaluation does not assess the authoring part of the application, as this part is only used by the teachers.The results of these questions were processed with SPSS 26 software in order to extract conclusions about the responses of the participants. First, we conducted normality tests, in order to decide if we could use parametric tests or not. Then, we applied one sample parametric and non-parametric tests to compare the results against a reference value. 0.05 was used as a threshold for statistical significance.The first meaningful result is that the evaluation session was conducted without incident; every user was able to use the stereoscopic visualization and all the comments during the session were very positive, highlighting the novelty of the visualization system. There were no noticeable issues related to cybersickness or uncomfortable feelings while using the devices.Regarding the quantitative analysis, Table Although the results are clearly positive, in order to obtain a more detailed picture of the opinions of the participants, we first applied normality tests to the data, so that we could decide if parametric or non-parametric tests should be used to analyze the data. Table neutral (4). All of them also have a median result significantly above somewhat agree (5), and half of them  show a median result that is significantly above agree (6). These questions deal with depth perception (question #2), with the usefulness of 3D visualization to understand the position and orientation of the surgical material in relation to the patient's anatomical elements (question #3), with the integration of the annotations within the video (question #7), with the usefulness of the use of sections (questions #8 and #12) and the pointer (question #9), and the usefulness of the tool as a training asset (question #14). These are very good results that confirm that the surgeons consider this tool to be a very useful one. The questions that show some room for improvement were question #4 (depth perception about annotations), question #5 (usefulness of the annotations), question #6 (annotation tracking), question #10 (depth perception about pointer), question #11 (usefulness of the annotations for training) and question #13 (difference between 2 and 3D). Question #4 and question #6 deal with how to track and show the annotations. It is quite complicated to track internal body elements, since object trackers are not designed with this use case in mind. It is also hard to calculate the exact horizontal disparity of the annotation with endoscopic cameras that are constantly moving. Therefore, some users may notice some discomfort when occlusions or fast movements hinder the calculation of this annotation process. In any case, both display an average of 6.2, which is quite high.In order to check how favorable the opinions of the participants were, we performed a one sample Wilcoxon signed-rank test for questions 1–14. This non-parametric test allows us to know if the answers to each of the questions are significantly above a certain threshold. Three thresholds were used: 4, 5 and 6. Table None of the questions received a value below 6, and some of them received average values close to the maximum possible value (7). Therefore, we can affirm that the evaluation of the tool is satisfactory.Finally, we analyzed question 15. Given that the average value for question #15 is 8.533, it is quite obvious that this value is significantly higher than 5. In any case, we conducted a series of t-tests in order to find out how high a test value could we reach whilst still getting significant results. Table The role of RAS in surgical practice is becoming increasingly important. Therefore, it is also increasingly important to introduce this paradigm into surgical training programs. For this reason, this article presents an MR-based tool for the stereoscopic visualization, virtual annotation and collaborative display of RAS surgical procedures. Although there are some applications that allow the viewing of stereoscopic videos, there is no academic work that allows the annotation and display of RAS videos in the way our tool does. Our application is web-based, and therefore, almost universal. It can work both on local networks and also remotely. It works on computers, tablets and smartphones. More importantly, it can display MR-enhanced content about RAS procedures with both expensive  and inexpensive hardware . It is also collaborative, since a teacher can share a class with several students. Finally, it allows virtual content, in the shape of virtual annotations, to be introduced into an existing stereoscopic video.An experimental test with RAS students was performed with good results. None of the users had problems using the visualization device, and the researchers observed that all of them were pleasantly surprised with the proposed visualization system. The results obtained with the evaluation support this initial impression. Given these results obtained with the evaluation, we can confidently say that the tool is useful and helps train surgeons in RAS. All the questions received very positive answers. The items that were the most positively rated were the usefulness of 3D visualization to understand the position and orientation of the surgical material in relation to the patient's anatomical elements, and the use of sections to reduce the amount of video that needs to be watched. Some of the questions regarding the annotations received slightly worse responses, although none of the responses received an average value lower than 6 . In addition, the users felt that the annotations were well integrated and well visualized within the video, since this question received a value significantly higher than 6. Regarding the limitations of the study, we acknowledge that this is a preliminary evaluation with 15 users. This is not a large number, but it is also true that this is a highly specific application and it is quite difficult to find a large group of people who could be considered potential users of this application. Our target group is small, since this tool is designed for surgeons who are enrolled on RAS courses. Therefore, it is unrealistic to expect to recruit a large group of such people. Nevertheless, it is also true that we performed the test in an uncontrolled non-local network—since the server and the clients of the application were on different networks—with wireless devices, and no communication problems were reported. This increases the validity of the assessment.Future work includes the evaluation of the authoring tool with a usability or effort test, such as the SUS  or the N