During the COVID-19 pandemic, social distancing restricted psycho-oncological care. Therefore, this secondary analysis examines the changes in anxiety, fear of progression, fatigue, and depression in cancer patients after a video-based eHealth intervention. We used a prospective observational design with 155 cancer patients with mixed tumor entities. Data were assessed before and after the intervention and at a three-month follow-up using self-reported questionnaires . The eight videos included psychoeducation, Acceptance and Commitment Therapy elements, and yoga and qigong exercises. The results showed that three months after finishing the video-based intervention, participants showed significantly reduced fear of progression (d = −0.23), depression (d = −0.27), and fatigue (d = −0.24) compared to the baseline. However, there was no change in anxiety (d = −0.09). Findings indicated marginal improvements in mental distress when using video-based intervention for cancer patients for up to three months, but long-term effectiveness must be confirmed using a controlled design. Every second patient with cancer has to deal with the psychological burden of their cancer diagnosis and from the consequences of cancer treatment . One in Additionally, the COVID-19 pandemic increased the psychological burden of cancer patients ,7. A metSince the beginning of the pandemic, psycho-oncological care has met several challenges. First, the necessity of social distancing has led to a reduction in usual psycho-oncological care. In order to meet the high demand for psychological support combined with contact limitations, digital psychological support became more important than ever.Digital interventions are part of eHealth interventions , in whicRegarding the wide range of eHealth applications, studies have shown that web-based applications are suitable for psycho-oncological interventions ,23,24. AOverall, there is a need to examine video-based interventions targeting coping strategies, integrating more tumor entities and follow-up observations. Therefore, our team established a video-based intervention focusing on psychoeducation and physical exercises to improve coping with symptoms of anxiety, depression, and fatigue in a previous study . PsychoeFirst, we hypothesized that the scores for anxiety, fear of progression, depression, and fatigue would improve significantly between the baseline and three months after the intervention.Furthermore, we hypothesized no significant changes in anxiety symptoms, fear of progression, depression, and fatigue symptoms between the end of the intervention and the follow-up period.Secondly, we investigated if participants who maintained video usage and exercised during the follow-up period had better anxiety, fear of progression, depression, and fatigue outcomes over time than those who stopped video usage and practice.The prospective observational study was conducted at the University Hospital of Wuerzburg, Germany, Comprehensive Cancer Center Mainfranken (CCCMF). The Ethics Committee of the University of Würzburg approved the study (ref. 123/20-me). Reporting complies with common guidelines, e.g., .Patients with malignant tumor disease, 18-years or older, and provided informed consent were included in the trial (inclusion criteria). In addition, participants were required to have internet access or at least a DVD-compatible device to watch the videos. There was no preselection concerning their current psychosocial distress. Exclusion criteria were severe physical or mental impairment and inadequate German language abilities.Patients with diverse tumor entities were recruited from institutions within the CCCMF, like the interdisciplinary oncological therapy outpatient clinic (IOT), ambulance of psycho-oncology, and various oncological stations of the University Hospital of Wuerzburg. Written and verbal information about the study was provided before patients gave written consent. The video intervention comprised eight videos and lasted for four weeks. During this period, the waiting control group participants received no videos. The video intervention was offered to the control group afterwards. The participants completed questionnaires at baseline (T1), after the intervention group finished the video intervention, and when the control group ended their video intervention (T2). The last survey was conducted after a follow-up period three months from the end of the video intervention of each group (T3). Since both groups attended the video intervention, we combined both groups for this secondary observational study to examine the changes over time.The sample size calculation of the previous study  determinParticipants had access to the CCCMF website, on which two videos were provided weekly. Participants received weekly information via email when new videos were available. The length of the videos was about twelve to thirty minutes. Patients could watch the videos repeatedly and save them via download. Therefore, the video usage was accessible at the will of participants. In addition, the participants could watch the videos they were interested in at their own pace. Therefore, the flooding of information should have been avoided. Furthermore, self-determined video selection might have reduced dropouts . During A psycho-oncologist performed the video intervention. In line with psychoeducation, each video included knowledge about a central topic and appropriate physical exercise. Therefore, information on signs and symptoms was given, as well as guidance and psycho-oncological tools to improve coping. Besides psychoeducation, some aspects inspired by MBT and ACT were used. Most integrated physical exercises were based on Hatha yoga, which has its roots in mindfulness-based stress reduction from Kabat-Zinn . MoreoveAnxietyIn order to assess anxiety symptoms, we used the Generalized Anxiety Disorder scale (GAD-7), which is a questionnaire with good validity and reliability . GAD-7 cFear of progressionTo assess fear of progression, we used the short form of the fear of progression questionnaire (FoP-Q-SF), which is a valid and reliable instrument for cancer patients ,43. FoP-DepressionThe Patient Health Questionnaire-8 (PHQ-8) was used to measure depressive symptoms . PatientFatigueCancer-related fatigue was assessed with the fatigue scale of the European Organization For Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-FA12). The questionnaire comprises twelve items of fatigue’s physical , cognitive , and emotional aspects , rated on a 4-point Likert scale. The total score is transformed to a scale ranging from 0 to 100, with higher scores representing more severe fatigue symptoms ,47. The Treatment adherenceMoreover, patients were asked about video usage and exercise practice during the follow-up period . Video ut-tests. In addition, standardized effect size  and accompanying 95% confidence intervals (CI) were determined. Questionnaires with missing values were excluded by pairwise deletion. For all statistical analyses, p < 0.05 was considered significant. No adjustment for multiple tests was planned. Therefore, we reported the exact p-values.Analyses were performed using IBM SPSS Statistics Version 27 . To analyze the hypotheses on changes between baseline and follow-up and post-intervention and follow-up, we used paired To explore the relationship between our intervention outcomes and exercise practice or video usage during the follow-up, we calculated an analysis of variance with repeated measures (ANOVA). Data were adjusted for statistical outliers. We explored outliers by calculating box-and-whisker plots for each group and defined outliers as values greater or less than 1.5 × the interquartile range. The number of excluded outliers varied between 0 and 4 for the analyses.n = 17) before the first assessment, our initial sample comprised 155 patients. A further 46 patients were lost until follow-up. Therefore, our analysis refers to patients with data on all assessment points (n = 109). For details on the study flow, see p < 0.001), were under tumor therapy (p = 0.015), or received palliative care (p < 0.001) than the completers.From June 2020 to September 2020, a total of 172 patients were recruited. Due to drop-outs (n = 73) continued using video sequences during the follow-up period. Of all participants, 9% (n = 10) used seven or eight videos, about 24% (n = 26) used four to six videos, and about 34% (n = 37) used one to three videos. One-third (n = 36) had not seen any of the videos during the follow-up period. Concerning the frequency and regularity, of those who continued video usage (n = 73), 4% (n = 3) used the videos several times per week, 26% (n = 19) used the videos once a week, and 67% (n = 49) used the videos rarely. The most popular reasons (n = 70) to continue video use were an improvement in coping with the disease  as well as recovering from sedation . On the other hand, the main reasons for quitting video usage (n = 36) were a lack of need due to general well-being  and a lack of time .Of the 109 participants, 67% (n = 6) conducted daily exercise, 23% (n = 21) performed the exercises several times a week, and 18% (n = 17) practiced once a week, while 38% (n = 35) performed the exercises more rarely. Therefore, 15% (n = 14) conducted none of the exercises during the follow-up period.Ninety-three participants answered the questions about the integrated exercises. Thereof, about 6% (n = 73). Those who failed to use any video sequence during the follow-up period were defined as “non-users” (n = 36). Moreover, participants who executed the exercises daily, several times per week, or once a week were summed up as “practicers” (n = 44). Finally, those who stopped exercise or performed them rarely were termed “non-practicers” (n = 49). Results of the exploratory group comparisons over time (ANOVA) are summarized in In order to examine the relationship between outcomes and video usage or exercise practice, participants who answered yes to continuing the usage of all or several video sequences were determined as “video-users”  was based on self-reporting. Therefore, retrospective bias or answers according to social desirability must be considered.Symptoms of the fear of progression, depression, and fatigue decreased up to three months after the video-based intervention. However, anxiety scores remained unchanged during the study period. Focusing on post-intervention, symptoms of anxiety, depression, and fatigue were stable, while the fear of progression declined. Video usage and exercise practice during the follow-up period did not affect the outcomes. However, patients with higher needs tended to use the videos.Our results support the further development and integration of digital interventions in psycho-oncological health care. However, further studies are required to examine and confirm the lasting effectiveness of video-based interventions. In order to improve outcomes, more interactive interventions, for example, live-guided interventions, could be established as content for further research . Moreove


Predicting where users will look inside head-mounted displays (HMDs) and fetching only the relevant content is an effective approach for streaming bulky 360 videos over bandwidth-constrained networks. Despite previous efforts, anticipating users’ fast and sudden head movements is still difficult because there is a lack of clear understanding of the unique visual attention in 360 videos that dictates the users’ head movement in HMDs. This in turn reduces the effectiveness of streaming systems and degrades the users’ Quality of Experience. To address this issue, we propose to extract salient cues unique in the 360 video content to capture the attentive behavior of HMD users. Empowered by the newly discovered saliency features, we devise a head-movement prediction algorithm to accurately predict users’ head orientations in the near future. A 360 video streaming framework that takes full advantage of the head movement predictor is proposed to enhance the quality of delivered 360 videos. Practical trace-driven results show that the proposed saliency-based 360 video streaming system reduces the stall duration by 65% and the stall count by 46%, while saving 31% more bandwidth than state-of-the-art approaches. Virtual Reality (VR) technology has developed rapidly in recent years. Studies ,2 have sA popular VR application is 360 video streaming . A 360 vIn spite of its promising nature, current 360 video streaming systems are expensive and inefficient in terms of bandwidth because of the need for transporting ultra-high-resolution data ,8,9. Popsalient regions of interest where users tend to view coupling with the user’s head movement history, the accuracy of the head-movement prediction for 360 videos can be improved. A 360 video streaming system employing this double-channel head movement prediction would be able to reduce its spatial fetching error, allowing users to enjoy downloaded content in high quality and improving the Quality of Experience (QoE). The key steps to achieve such a saliency-based 360 video streaming system include (a) a proper saliency detection model for 360 videos, (b) a unified approach integrating salient regions with head movement history to achieve accurate head-movement prediction, and (c) an efficient streaming system that only streams the salient content by exploiting the head movement prediction results.It is well known that understanding human visual attention to 360 videos is the key to improving head-movement prediction ,14. By lDespite this consensus, the vision of improving 360 video streaming with saliency-enhanced head movement prediction has never been realized. All previous works are only able to achieve one or two steps of the aforementioned saliency-based streaming system. Some works have proposed saliency-based head-movement prediction models, but they did not design and evaluate the end-to-end streaming system in a real-world context ,14,15. SIn this paper, we propose a streaming framework to unleash the potential of the salient content in 360 videos. The key is to capture the unique viewing attention of users on 360 videos inside VR HMDs. By supplementing important visual features to the head movement predictor, we aim to ultimately enhance the overall performance of the streaming system. We face several challenges to achieving this goal.First, when training the 360 video saliency detector through deep neural networks, overfitting can happen easily if the training set lacks variety. While increasing the richness of the training data helps, user behavioral data in 360 videos are limited compared to regular 2D videos and are expensive to collect. Hence, we propose to use transfer learning, where the parameters from models previously trained on regular videos are transferred to our model. This allows our model to inherit knowledge about common visual patterns from much larger datasets  in domaiSecond, the predicted saliency may not always indicate the head movement of users. This is because a 360 video can have multiple salient regions evolving through time. It is not straightforward to identify the salient content to be viewed by the user hidden among false-positive noises. To cope with this challenge, we leverage the highly non-linear structure of the Long Short-Term Memory (LSTM) network to build the head-movement predictor by exploiting the temporal–spatial input of both saliency maps and head orientations.Finally, the impacts of head-movement prediction on the performance of the streaming system have not been studied comprehensively. Prior 360 video streaming research either demonstrates the ultimate streaming performance without isolating other design factors such as new video encoding methods and updated adaptation algorithms ,21,22, oWe validate the proposed system with extensive evaluations using viewing logs of HMD users and LTE bandwidth traces ranging from 8 Mbps to 44 Mbps. Results show that under the normal bandwidth condition at 22 Mbps, the proposed 360 video streaming system reduces the stall duration by 65% and the stall count by 46% while saving 31% more bandwidth compared with systems using head movement predictors that do not utilize salient content.A deep convolutional neural network (DCNN) that detects the unique saliency of 360 videos in HMDs.An LSTM-based head-movement predictor leveraging both salient content and users’ head-orientation history.A system that enables saliency-based 360 video streaming and studies the effect of head-movement prediction on the streaming performance.Our contributions can be summarized as follows:Saliency detection is an important topic in the study of human visual attention. Early works focused on constructing saliency models using hand-crafted features ,25. HoweSaliency detection for 360 images/videos has attracted attention recently. Researchers applied a 2D-image-based saliency detector on different parts of a 360 image and linearly combined results to produce the final 360 saliency map ,30. OtheUnlike prior approaches that better handle distorted objects in 360 video saliency detection, we aim to enhance saliency detection by capturing unique visual attentive behaviors of humans in VR HMDs. Our preliminary results have discovered and addressed two types of unique head behaviors related to visual saliency on 360 videos . First, Head movement prediction is an essential component for an efficient 360 video streaming system. Head trajectory history, a series of time-stamped quaternions representing the orientations of the users’ heads, plays a pivotal role in predicting future head movement. The processing and analyzing of head trajectory history in different ways have been used in previous head movement prediction models. Early models reused current head position  or conduTo address this limitation, video content features have been used as another approach to enhancing head-movement prediction. Aladagli et al. predicted head movement based on a pre-trained saliency model by other researchers . Fan et Unlike these works, we employ both factors affecting head movement, i.e., the saliency crafted for dynamic 360 videos and the user’s head movement history, and then explore the interplay between them, to maximize the head movement prediction performance and enhance the 360 video streaming system.Unique challenges are presented by 360 video streaming. The occurrence of lagging and stalling leads to motion sickness  that breIn viewport-adaptive streaming , multiplTile-based streaming ,52,53,54Despite the insights of previous works, they overlook the direct effects of the head movement predictor. The results reported before often showed the overall system performance, which is the result of many factors such as encoding schemes ,11,22, tIn this section, we discuss the overall design of the proposed saliency-based 360 video streaming system. As illustrated in The general interaction among these components is as follows. Saliency maps of 360 videos are generated offline at the server by the saliency predictor. During the session initialization, the saliency maps are transferred over the network to the client before the 360 video data are streamed. At the client, the head predictor utilizes saliency maps and head movement history from the HMD to predict where the user will look in the near future. The scheduler receives the results from the head movement predictor and produces a spatial fetching decision to request the actual 360 video data. Upon receiving the client request, the server responds with the salient 360 video regions for the user to enjoy.Our end-to-end design is a holistic framework including all essential components to realize the idea of saliency-based 360 video streaming. The design of the client highlights the interaction between the head movement predictor and the scheduler, allowing us to investigate the effect of head-movement prediction on system performance. In the following, we will introduce the details of the server and the client, especially the core components.We start by discussing the workflow, introducing how the saliency predictor and other components in the server interact. Then we go into the details of the saliency predictor, the core component of the server.PanoSalNet, the saliency predictor to be introduced in The server has two main tasks, preparing video data and metadata offline and responding to tile requests with video data online. During the offline streaming preparation, the raw 360 video is first temporarily divided into chunks. Each chunk is then spatially cut into tiles of equal size. The tiles are encoded and stored as files in the permanent file storage. The spatial and temporal positions of each tile within the 360 video are recorded so that the server can locate them at a later time. Meanwhile, the server utilizes At the beginning of the streaming session, a Media Presentation Description (MPD) file that contains positioning information about every tile is sent to the client. The positioning information includes the chunk index number and the spatial coordinates of the tile in the chunk. The saliency maps are also sent to the client along with the MPD file. The saliency maps are rescaled to a smaller size to reduce their bandwidth consumption. During the streaming, the Request Handler module waits passively for tile requests from the client. The tile request specifies a list of tiles and the associated positioning information. The server locates the tile files and streams them back to the client. Similar to traditional video streaming servers, the Request Handler is stateless; i.e., it does not store any status associated with any client, making it easy to scale up to a large number of requests.As we discussed, the attentive behaviors of users in 360 video viewing under HMDs differ from those of 2D videos/images and 360 images. Since it is not possible to collect ground truth saliency maps for every video through subjective study, we need a model that can learn unique human attentive behaviors captured in limited public datasets and perform saliency detection on new videos not present in the training. To achieve this objective, we propose PanoSalNet, a DCNN-based panoramic saliency detection model for 360 videos. DCNN has been the primary method in image processing ,27,28 duIn general, a large and diverse dataset is needed for training to avoid SM overfitted DCNN model and to produce a better DCNN performance. Considering the human-centric nature of visual attention studies , it is eThe network architecture of PanoSalNet is illustrated in The predicted saliency is enhanced one more time as the final output by a prior filter , which lN is set to 3, W is a model parameter to be learned, f function calculates the output saliency map based on the input image To expedite the model learning, the frame resolution was downscaled to 512 × 288. Since predicting a saliency map is a regression problem, we use Euclidean distance to measure the difference between ground truth saliency maps and the predicted results. The loss function is defined as follows:After model training, the proposed PanoSalNet is then able to detect the saliency map of an input 360 video.We now discuss the streaming workflow at the client end, i.e., the function of each component and the interaction between them. Then, we introduce the two core components of the client, the scheduler and the head predictor.The streaming client is empowered by the interactions among three components, the scheduler, the head predictor, and the playback buffer. During playback, the scheduler periodically, or at specific events, calls the head predictor to retrieve the predicted user head orientations in the near future. Upon receiving responses from the head predictor, the scheduler sends tile requests to the server. Tiles received from the server are put into the playback buffer before displaying to the user.The client decodes and stitches the received tiles before putting them into the playback buffer. Each buffer item corresponds to a video chunk. The items in the playback buffer are taken out at a steady rate and displayed in the user’s viewport. The length of the buffer is shorter than buffers in regular 2D video streaming systems due to the uncertain nature of the head prediction task . If the When the head predictor receives a request from the scheduler, it gathers input from two data sources and returns back a predicted head orientation map. The two data sources include the gyroscope sensor and the saliency maps initially received from the server. The gyroscope provides head movement data points in real time, which are converted into input head orientation maps. Depending on the type of request from the scheduler, the head predictor could either return the head orientation map corresponding to the current head position or employ the LSTM architecture discussed in The scheduler utilizes the result of the head predictor to request tiles such that the bandwidth is used in an efficient manner while ensuring the QoE. Its job includes forming a tile plan, sending tile requests, managing the buffer content, and handling exceptions. Exceptions occur when the buffer is empty or when the user’s head moves out of the planned tile area.We proceed to introduce the proposed scheduler that highlights the impacts of head movement prediction on 360 video streaming.The purpose of tile planning is to prepare a list of tiles to request. The set of tiles should cover the entire HMD screen while being small enough to not waste valuable bandwidth. To this end, our strategy is to select enough tiles to cover the predicted viewport and the additional margins in all four directions . As most prediction errors are the small misalignment between the predicted and the actual viewports, it is possible to choose an appropriate margin size to cover this mismatch without consuming excessive bandwidth.Let i and j indices denote the row and column positions of a tile in the equirectangular. The underlying idea is that the scheduler first finds a center tile corresponding to the most salient region from the head orientation maps and then adds tiles staying the closest to the center tile to the list of selected tiles until the list reaches the desired size.The In extreme cases, a wide and unexpected head movement could bring the viewport outside the prepared tiled area. Such a case has a detrimental effect on the user QoE as they will see the many blank tiles occupying a large portion of the viewport for a period of time. In this case, stalling the playback is a straightforward approach to prevent users from losing track of the video content. This is similar to the rebuffer in regular video streaming, where playback stalls when there is no content to display.Scheduling Algorithm. The aforementioned behavior of the scheduler is summarized in Algorithm 1.Algorithm 1: Tile scheduling algorithmTo make the decision of stalling the playback, we need to quantify the ratio of the blank screen area inside the viewport. To this end, we first convert the actual viewport into a tile presentation. Let the tiled version of the actual viewport, denoted by In the formula above, To remedy the issue of unexpected head movement, we employ the flush and resend strategy. When the Blank Ratio is greater than a threshold Users see a totally blank screen when the buffer is empty. This happens when bandwidth is insufficient, causing requested tiles not to arrive at the buffer on time. Due to the dynamics of the head movement, the user’s action during this period is not defined. Relying on ongoing requests from previous head predictions is not reliable. In this case, the scheduler should reset the tile request with the current head position. Note that we design this buffer management scheme to more effectively investigate the impacts of head movement prediction on the system performance in bandwidth-limited networks. By using a relatively simple scheme, we aim to isolate other system design factors and focus on head movement prediction.One key obstacle for head movement prediction in previous works is when the user takes a fast head movement, during which the prediction accuracy has been observed to drop significantly . This tyIn order to learn the pattern of head tracking logs of multiple users and to capture the interplay between temporal user behavior and multiple saliency maps from past video frames, a highly nonlinear learning model is required. We propose utilizing a Long Short-Term Memory (LSTM) model, since LSTM is proven to be able to handle a large amount of temporal data and outperform other algorithms .k time steps (prediction window). The input features of the LSTM network include both the 360 video saliency map detected by the proposed PanoSalNet (indicating regions of interest) and the head orientation feature (recording head-movement history).In particular, the proposed LSTM model is a Recurrent Neural Network (RNN) that works on the temporal domain. Since stacking multiple layers of LSTM on top of each other can handle more complex data , we adophead orientation map, a spatial data structure similar to a saliency map. Each pixel in the map corresponds to a video tile, with bright colors signifying proximity to the center of the viewport. Input head orientation map highlights the viewport within a frame that would be viewed under the current head orientation. We generate the head orientation map by first identifying the tile pointed to by the current head orientation vector and set its likelihood to be viewed as 1.0. Using this tile as the center, we then apply a Gaussian kernel to gradually select other tiles of the viewport with a lower likelihood to be viewed around the center tile.To better correlate head orientation with requested/viewed video tiles in 360 video streaming, we follow a similar method in  and reprThe loss function for training is calculated based on the Euclidean distance between the predicted head orientation map and the ground truth head orientation map. Our model parameters are updated with Root Mean Square Propagation (RMSprop) method, a preferred method for RNN as suggested by .In this section, we conduct experiments to investigate the performance of the streaming system. We start by describing our experiment setup. Then, we detail the evaluation results.Here, we describe datasets used to train the saliency predictor and the LSTM-based head movement predictor. Then, we discuss the setup and evaluation of our trace-driven 360 video streaming system.360 Video Saliency Dataset. To train the saliency predictor, we use an existing saliency dataset for 360 video viewing [ viewing . Each sa viewing ,59,60.Head Movement Dataset. To train the head predictor, we use saliency maps output from the saliency predictor and head traces from five videos from [eos from . For eacReal-world Traces. The system receives head-movement traces from real users watching 360 videos in a free viewing context. Traces of 10 users are randomly selected from the 48 users viewing eight videos in 2K resolution [solution . Since tsolution , 2K resosolution . We selesolution  and presOther System Parameters. We set the chunk length to one second such that the system would not commit to one viewport for an extended period during which the user may have already changed the viewing direction. To balance the encoding efficiency and bandwidth efficiency, the number of tiles per chunk has been configured in a range from a half-dozen [(x) as the Expanded Tile Size. It includes tiles covering the viewport and its margin. To explore the performance, the Expanded Tile Size is varied from lf-dozen  to a fewlf-dozen ,52. We tken from . We denoImplementation. The saliency detection was implemented using the Caffe library. The head movement prediction model is implemented with Keras. Both models are trained on Intel Devcloud, a cloud computing platform optimized for artificial intelligence. The utilized Devcloud machine has eight Intel Xeon Scalable 6128 processors, with a GPU equipped with Intel Xeon E-2176 P630 processors. For the streaming system, the system logic is implemented in Python 3.6. The program runs on an Ubuntu OS 18.04.6 LTS installed on a desktop machine with Intel i5-10300H 2.5 Ghz, 16 Gb RAM and a NVIDIA GTX 1660 Ti GPU. The desktop computer was manufactured by Dell Inc., Houston, TX, USA.Baselines and Metrics. We use three baselines to validate our streaming system. Each baseline uses an existing head predictor while the streaming configurations are kept the same. The first baseline [Lnregr) utilizes a linear regression model to process the input saliency maps and head orientation maps. The second baseline [PanoSalNet. The third baseline [Full) does not need input from the head predictor since it always streams all tiles. The prediction window for all models is 1 s.baseline  . The median score of our system is close to Full at 31 Mbps bandwidth. Note that the Full model tends to achieve perfect performance in higher bandwidth conditions where it is free from both rebuffering and head prediction error.We further investigate the distribution of viewport perceived quality scores for individual models at each bandwidth level and report the results in Combining both results from Playback Buffer Size. One of the key challenges in designing a 360-degree streaming system is the length of the playback buffer. It is not possible for 360-degree streaming systems to have more than a few seconds of buffer due to the high uncertainty of users’ viewing behavior. This uncertainty comes from the fact that users are the entities who ultimately decide where and what to watch, despite their tendency to focus on salient content. At any time, users may move their heads away from the prepared region, rendering all the downloaded data to become wasted. To overcome the issue and improve our system, the Expanded Tile Size could be dynamically increased/decreased according to the network condition. Ideally, the buffer content in the far future should cover more areas on the equirectanguler to accommodate any unexpected head movement.Bitrate Adaptation. Bitrate adaptation is a popular technique that switches between different bitrate levels to adapt to the change in bandwidth conditions. With bitrate adaptation, the number of data can be varied depending on the bandwidth availability. This comes with a cost, however, since users may see time-varying video qualities between tiles. Currently, our system uses only one bitrate level since we focus on investigating the impact of the head predictor on the system performance. In the future, incorporating bitrate selection into our system will allow the system to stream 360 videos even in low-bandwidth environments.Tile Planning. The tile planning module selects tiles until they cover the whole projected viewport. The projected viewport is the projection of the user viewport from the spherical surface into the equirectangular frame, which is fixed in our case. In reality, the projected viewport grows when users look up or down near the pole region. Since our tile planning function relies on the fixed projected viewport, the system underestimates the number of requested tiles covering the projected viewport when users look up or down. However, our assumption still holds in the majority of cases since users spend most of the time looking at areas near the equator [ equator ,71,74, wHead Movement Prediction. Recently, self-attentive models such as the Transformer have become a favorable choice for temporal domain prediction tasks such as language interpretation. The main advantage of attentive models lies in their ability to select important but less frequent patterns while disregarding more recurring patterns. This may be beneficial for head movement prediction tasks since head movements are spontaneous, which correlate with few data points in the past movement trajectory. However, Transformer has higher computational requirements and needs a large number of training data. The applicability of the Transformer is still unclear, and investigating its potential for the head movement prediction task is left for future study.This paper demonstrates the feasibility of utilizing salient content uniquely present in 360 videos to ensure accurate head movement prediction and efficient 360 video streaming. Our system leverages unique salient features, capturing humans’ exploratory behavior to assist the prediction of the head movement of users using HMDs. A streaming system seamlessly integrates the head movement prediction into its frameworks to improve overall performance. We conduct trace-driven experiments to confirm the advantages of the proposed approach. Through evaluations, the system is shown to excel for other baselines in multiple respects, such as reducing stalling duration by up to 65% and stall count by 46%. At the same time, the bandwidth saved is 31% more, which helps the systems to be more robust against different bandwidth conditions. Our work is an important step toward bridging the performance gap of head movement prediction and improving the QoE in today’s 360 video streaming systems.


In this article, a graph-theoretic method (taking advantage of constraints among sets associated with the corresponding parity-check matrices) is applied for the construction of a double low-density parity-check (D-LDPC) code  in a joint source-channel coding (JSCC) system. Specifically, we pre-set the girth of the parity-check matrix for the LDPC code pair when jointly designing the two LDPC codes, which are constructed by following the set constraints. The constructed parity-check matrices for channel codes comprise an identity submatrix and an additional submatrix, whose column weights can be pre-set to be any positive integer numbers. Simulation results illustrate that the constructed D-LDPC codes exhibit significant performance improvement and enhanced flexible frame length  compared with the benchmark code pair. A joint source-channel coding (JSCC) system is capable of overcoming the shortcomings of a Shannon separation coding system and enhancing the performance of the digital communication system via joint decoding . In geneAmong these realizations, the D-LDPC code-oriented JSCC system has been intensively investigated since its first appearance in ,10, wherCombinatorial mathematics is an important tool for constructing LDPC codes, which includes the graph-theoretic method, combinatoric designs, finite geometries, and difference sets ,19,20,21In this paper, a construction method of the code pair is developed based on graph theory, which transforms parity-check matrices of the source code and the channel code into inter-set constraint problem and selects the sets which satisfy the constraint. In order to match the parity-check matrix of the source code and the generation matrix of the channel code in the encoding process, the channel code is first constructed during the construction of the code pair via the graph-theoretic method. In contrast to the conventional method, the proposed one can be obtained without constructing the base matrix and performing ‘copy and permute’ operations. Since the proposed channel code takes the systematic form, the generator matrix is easily obtained by transposing the parity-check matrix. In addition, the proposed code pair is constructed with set constraints from the graph-theoretic method, which brings a more adaptive/flexible frame length than the conventional protograph code pair.The D-LDPC system consists of two LDPC codes: one for source code and the other for channel code. As a linear block code, a parity-check matrix can be used to express the LDPC code. A systematic LDPC code can be completely characterized by a generator matrix The parity-check matrix of the JSCC system is represented asvided in .(3)HJ=1The Tanner graph associated with the parity-check matrix ******p) distribution, whose entropy is expressed asp  and the edges (lines or branches) that connect the vertices. In the sequel, we provide multiple definitions and theorems related to graphs.Definition 1. A connected graph, Definition 2. Let Theorem 1. Let Proof. When Theorem 2. According to Theorem 1, the LDPC code pair is constructed for optimizing the performance of the JSCC system. Each column of the parity-check matrix can be represented by a subset . The maximum number of columns in the LDPC source code is determined viawhere the column weights for the source and channel LDPC codes are denoted as Proof. m and W to make it more general, since this statement can be applied to both source code and channel code.) The condition that a larger-length (>4) cycle does not exist is pre-conditioned on the fact that there does not exist any length-4 circles. Therefore, when a larger-length cycle does not exist, the number of columns of the constructed parity-check matrix has to be reduced to satisfy the stricter set constraint in Theorem 1.    □The parity-check matrix of the systematic channel LDPC code consists of an identity matrix According to Theorem 1, the corresponding pseudo-code for the proposed construction algorithm of the LDPC code pair is given in Algorithm 1, described as follows:Step 1: Set up the column weight em 2 and   satisfying Theorem 1 for constructing the column vector of the parity-check matrix for the channel code Step 4: Calculate nship in  and the Step 5: According to Theorem 1, select the desired subsets for constructing the column vector of the parity-check matrix for source code Algorithm 1: Construction of LDPC code pairRequire:        s: girth of the constructed source code and channel code.Ensure:  Initialize: parity-check matrix for channel code 1:Begin2:3:4:5:6:fordo7:   8:if then   9:     ▹ The variable 10:     11:end if   12:   13:for do   14:if then     15:        16:else     17:        18:end if     19:if then     20:        21:else     22:        break;23:end if     24:end for   25:if then   26:Theorem 1.     ▹ Determine whether the subset 27:     28:     29:else   30:     31:end if   32:end for33:Construct the parity-check matrix of the source code 34:EndW is the larger value between the source code column weight We evaluate the performance of the proposed JSCC system over AWGN channels in terms of bit error rate (BER) in this section. The simulation results are obtained by considering binary phase-shift keying modulation and employing joint belief propagation iterative decoding. The benchmark code pairs are generated using the progressive edge-growth algorithm with ‘copy-and-permute’ . In contp is set to pairs in ,16 corre pair in  by yieldIn this article, an algebraic construction method for LDPC code pairs of JSCC systems has been proposed based on the graph-theoretic approach. The constructed code pairs could avoid short-length cycles by following set constraints. The simulation results has shown that the proposed code pairs achieve significant performance improvement compared with the benchmark code pair , ,16. The


Ketamine, an NMDA antagonist, is widely used in clinical settings. Recently, low-dose ketamine has gained attention because of its promising role as a rapid antidepressant. However, the effects of low-dose ketamine on brain function, particularly higher cognitive functions of primate brains, are not fully understood. In this study, we used two macaques as subjects and found that acute low-dose ketamine administration significantly impaired the ability for arbitrary visuomotor mapping (AVM), a form of associative learning (AL) essential for flexible behaviors, including executions of learned stimuli-response contingency or learning of new contingencies. We conducted in-depth analyses and identified intrinsic characteristics of these ketamine-induced functional deficits, including lowered accuracy, prolonged time for planning and movement execution, increased tendency to make errors when visual cues are changed from trial to trial, and stronger impact on combining associative learning and another key higher cognitive function, working memory (WM). Our results shed new light on how associative learning relies on the NMDA-mediated synaptic transmission of the brain and contribute to a better understanding of the potential acute side effects of low-dose ketamine on cognition, which can help facilitate its safe usage in medical practice. This study found that acute low-dose ketamine significantly impaired the ability of arbitrary visual motor mapping, a critical form of associative learning (AL) for flexible behavior, including well-learned stimulus-response contingency or learning of new contingencies. We conducted a thorough analysis and identified intrinsic features of ketamine-induced functional deficits, including decreased accuracy, prolonged planning and motor execution time, increased error tendency when visual cues changed from trial to trial, and stronger impact on combining associative learning and another key higher-order cognitive function, working memory (WM). These findings contribute to a better understanding of the potential acute cognitive side effects of low-dose ketamine and promote its safe use in medical practice.Ketamine is a well-known NMDA receptor (NMDAR) antagonist, widely used in clinical settings for inducing anesthesia, providing pain relief, and sedation . RecentlAssociative learning and working memory are crucial functions that support higher cognition and represent the two pillars of cognitive control. Cognitive control is defined as the ability to acquire and implement the “rules of the game” necessary to achieve a given goal in a specific situation , which iTo fully use AVM, it is often necessary to engage another key cognitive function: working memory. Working memory enables the temporary storage and manipulation of information in the brain . By combTherefore, in this study, we aimed to investigate the effects of low-dose ketamine on the following functional aspects: (1) execution of a learned AVM rule; (2) learning of new AVM rules; and (3) execution of WM-based AVM. To facilitate comparison between different tasks, the same monkeys, ketamine doses, and similar behavioral settings were used. We hope that the results will help delineate the effects of low-dose ketamine on these important higher cognitive functions.The experiment involved two adult male macaques, aged nine and eight years old, with weights of 9 and 10 kg, respectively. All experimental protocols were approved by the Animal Care and Use Committee of the Institute of Automation, Chinese Academy of Sciences. The animals’ physical and psychological well-being was regularly assessed by researchers and veterinarians. Before the experiments, each animal underwent a standard sterile surgical procedure under general anesthesia to implant a headpost  for restDuring the experimental sessions, animals were restrained in a primate chair in an isolated room, and they were delivered juice rewards through a tube attached to the chair. The situation was monitored by the experimenter in a nearby chamber. Visual stimuli were presented on a capacitive touch screen , which was positioned 50 cm from the subjects’ eyes. The touching response of the subjects was registered by the same CTS. In addition, there was a holding rod in front of the primate chair and beside the animals’ right hand a, which There were three tasks during the experiment, including the arbitrary visuomotor mapping (AVM) task, the arbitrary visuomotor mapping with novel cue (AVM-NC) task, and the arbitrary visuomotor mapping with working memory (AVM+WM) task b. All taIn both the AVM and AVM+WM tasks, the same visual cues and mapping rules were applied for all trials. Specifically, letter O represented the lower button, and letter X represented the upper button. The animals achieved an accuracy of around 90% through pretraining. In the AVM-NC task, for each daily experimental session, two novel cues that the animals have never seen before were presented, and the corresponding mapping rule needed to be learned on the go. The experimental sessions were completed once the animals achieved 90% accuracy in 50 consecutive trials.In a given daily session, the subjects were randomly assigned to receive either a low dose (0.2 mg/kg) or a high dose (0.8 mg/kg) of ketamine, or saline control, administered intramuscularly in the left arm. Prior research has shown that this dosage of ketamine produces working memory deficits with no anesthetic effects in rhesus monkeys . The AVMThe experimental procedure was conducted using NIMH Monkeylogic software , which aBehavior performance was assessed during three phases: preinjection phase (15 min), early-postinjection phase (0–30 min after injection), and late-postinjection phase (30–60 min after injection) for AVM and AVM+WM tasks. Accuracy was analyzed using the number of complete trials, including both correct and incorrect trials, as the denominator. Incomplete trials, where the monkeys did not respond to the trial start cue, released the holding rod too soon, or did not touch the response buttons, were excluded from the accuracy analysis. The percentage of correctly responded trials was calculated in 5-min bins, resulting in 15 time bins for each daily session. For the AVM-NC task, a moving window consisting of 50 trials was used to calculate the percentage of correctly responded trials for each animal since there was no preinjection phase.We divided the reaction time (RT) into two components: planning time and movement time. Planning time was defined as the duration between the appearance of response buttons and the release of the holding rod by the animals, while movement time was the period from the release of the holding rod to touching the response buttons. Incomplete trials were excluded from the reaction time analysis.To investigate the impact of ketamine (0.8 mg/kg) on the relationship between reaction time and accuracy across all tasks, we analyzed all trial data from each session during both the early and late-postinjection phases. We first sorted the trials based on reaction time  and compared the mean accuracy of the first quarter of trials with the shortest reaction time to the last quarter of trials with the longest reaction time in each task. Additionally, we examined the correlation between reaction time and accuracy for each task separately, based on the analysis of planning and movement time. We divided reaction time into 0.05 s windows and calculated the mean accuracy for each window, then used MATLAB for linear regression analysis to obtain the correlation coefficient.To assess the impact of ketamine on task engagement in monkeys, we calculated the trial-complete index, which reflects the extent of the animals’ involvement in the task. This index was defined as percentage trial done, which is the ratio of completed trials to all trials . This metric was calculated separately for each daily session during the early-postinjection phase and compared across three dosages  separately in each task.To explore the factors contributing to heightened error rates in the ketamine condition, we classified trials into two categories: those featuring the same visual cue as the preceding correct trial (the “same cue” condition) and those featuring a different visual cue (the “different cue” condition). We then calculated the percentage of incorrect trials in each situation by dividing the number of incorrect trials by the total number of completed trials, during both the early-postinjection and late-postinjection phases, for all tasks in each session. We compared the results across three dosage conditions . We expanded our analysis to encompass the two and three preceding trials, to eliminate perseveration . To facilitate comprehension, we note the same cue and different cue situations as AA and AB, respectively. The first letter denotes the cue for the previous trial, while the second letter represents the cue for the current/analyzed trial. If the previous two/three trials were considered, we can similarly define the same cue situation (AAA/AAAA) and the different cue situations (AAB/AAAB). In all these analyses, we only included the cases in which the previous trials were responded correctly.In this study, we analyzed data obtained from two monkeys. To ensure clarity and concision in our presentation, we have combined the results from both animals in the main text. However, for readers interested in more detailed information about each individual subject, we have included their respective data in the extended figures.post hoc analyses, with Kruskal–Wallis test (K-W) of variance used for non-normally distributed data, and p-values adjusted for multiple comparisons. We used unpaired two-tailed t tests to compare the drug effects on planning time, movement time, and mean accuracy between fast and slow trials. For non-normally distributed data, we used the Mann–Whitney test. In addition to MATLAB, we also performed statistical analyses using GraphPad Prism 9.0.0, as appropriate. Results are presented as mean ± SEM unless otherwise specified in all figures.In order to accurately assess the effect of the drug, we compared the animals’ behavioral performance on the AVM and AVM+WM tasks after ketamine injection to their preinjection baseline within the same recording session. We also compared their performance across three drug conditions  during the early-postinjection period on the AVM, AVM-NC, and AVM+WM tasks. Statistical significance was assessed using ANOVA and p = 0.0054). However, the impairment was temporary, and the accuracy recovered during the late-postinjection period, becoming significantly higher than the early-postinjection period (p = 0.0222) and similar to the preinjection period (p = 0.8351). We conducted a cross-condition comparison to account for the effect of long experimental sessions on performance. We found that the saline, 0.2 mg/kg ketamine, and 0.8 mg/kg ketamine conditions significantly differed during the early-postinjection period . Specifically, the accuracy of 0.8 mg/kg ketamine was significantly lower than saline (p = 0.001) and 0.2 mg/kg ketamine (p = 0.0035), while the accuracy of the saline condition was similar to the 0.2 mg/kg ketamine condition (p > 0.9999). The individual performance of each subject is presented in Extended Data p = 0.76; Monkey Z, p = 0.98). Therefore, our results demonstrate that 0.8 mg/kg ketamine significantly impairs already learned AVM rules.To assess the impact of ketamine on the AVM task, we first examined the effects of ketamine on already learned rules. Our findings, shown in 10.1523/ENEURO.0015-23.2023.f2-1Extended Data Figure 2-1a, Percentage of correct trials in the AVM task for saline (gray), ketamine 0.2 mg/kg (green), and ketamine 0.8 mg/kg (blue) sessions. The lower right corner box plot shows the average percentage of correct trials during the early-postinjection period across three conditions. b, Accuracy in 50 consecutive trials in the AVM-NC task for saline, ketamine 0.2 mg/kg, and ketamine 0.8 mg/kg sessions. The lower right corner plot shows the learning cost time across three conditions. c, Percentage of correct trials in the AVM+WM task for saline, ketamine 0.2 mg/kg, and ketamine 0.8 mg/kg sessions. The lower right corner box plot shows the accuracy changed from the preinjection period to early-postinjection period across three conditions. d, Comparison of the effects of ketamine on AVM and AVM+WM tasks for Monkey A and Monkey Z. The effect of 0.8 mg/kg ketamine was quantified as the percent performance drop, calculated by (Apre − Aearly-post)/Apre %, where Apre and Aearly-post are the accuracy of the preinjection period and early-postinjection period, respectively. The bars represent the declined rate of accuracy. All error bars represent SEM. Statistical significance is denoted by *p < 0.05, **p < 0.01, ***p < 0.001. Download Figure 2-1, TIF file.Behavioral performance per subject. p < 0.0001). However, the 0.8 mg/kg ketamine condition took significantly longer  to reach 90% accuracy than the saline  and 0.2 mg/kg ketamine  conditions. The individual performance of each subject is presented in Extended Data In this study, we aimed to evaluate the impact of ketamine on abstract rule learning by assessing the ability to learn mapping rules to novel visual cues in the AVM-NC task. As shown in p = 0.0005). Although the accuracy slightly recovered during the late-postinjection period, it remained significantly lower than the preinjection period (p = 0.0463) and similar to the early-postinjection period (p = 0.1314). A cross-condition comparison showed that the saline, 0.2 mg/kg ketamine, and 0.8 mg/kg ketamine conditions significantly differed during the early-postinjection period  = 13.29, p < 0.0001). Specifically, the accuracy of 0.8 mg/kg ketamine was significantly lower than saline (p < 0.0001) and 0.2 mg/kg ketamine (p = 0.0014), while the accuracy of the saline condition was similar to the 0.2 mg/kg ketamine condition (p = 0.9176). The individual performance of each subject is presented in Extended Data p = 0.85; Monkey Z, p = 0.13). These results indicated that the AVM+WM based on the same mapping rule was also significantly impaired by 0.8 mg/kg ketamine.The purpose of this study was to assess the impact of ketamine on AVM+WM task accuracy. Our results, shown in pre − Aearly-post)/Apre %, where Apre and Aearly-post are the accuracy of the preinjection phase and early-postinjection phase, respectively. Our findings, presented in t test, t = 3.935, df = 17, two-tailed, p = 0.0011). The individual performance of each subject is presented in Extended Data We investigated the differential effects of ketamine on the performance of the AVM and AVM+WM tasks by exploring the extent of impairment caused by 0.8 mg/kg ketamine. The effect of ketamine was quantified using the percent performance drop formula , AVM-NC task , and AVM+WM task . Additionally, we observed that 0.8 mg/kg ketamine induced significantly more error trials than the 0.2 mg/kg ketamine condition in the AVM task  and AVM-NC task . No significant differences were observed between the saline and 0.2 mg/kg ketamine conditions in each task. In the same cue situation, as shown in To investigate the specificity of ketamine-induced errors, we conducted a study in which we divided the trials into two categories based on whether the visual cues were the same or different from the previous trial. We compared the percentage of incorrect trials in these two situations in each task. As illustrated in 10.1523/ENEURO.0015-23.2023.f3-1Extended Data Figure 3-1a, Average wrong percentage of trials in the different cue situation after injection for saline (gray), 0.2 mg/kg ketamine (green), and 0.8 mg/kg ketamine trials (blue). b, Average wrong percentage of trials in the same cue situation after injection for saline (gray), 0.2 mg/kg ketamine (green), and 0.8 mg/kg ketamine trials (blue). All error bars are SEM. *p < 0.05, **p < 0.01, ***p < 0.001. Download Figure 3-1, TIF file.The bad performance in two situations for per subject. The bad performance of trials in different situations per subject. By dividing trials into different cue situations and same cue situation adjusted by the cues was changed or unchanged between two adjacent trials during 60 min after injection. 10.1523/ENEURO.0015-23.2023.f3-2Extended Data Figure 3-2Figure 3-2, TIF file.The figure depicts the percentage of incorrect trials across three trial-forms  for each task under the 0.8 mg/kg ketamine condition. In the AAB and AAAB trial-forms, the cue from the previous trials that had been responded correctly was denoted as A, while the cue from the analyzed trial was denoted as B. The error bars correspond to the SEM. Download To account for the impact of perseveration on the study outcomes, we analyzed the percentage of incorrect trials across three trial-forms  of each task, while administering 0.8 mg/kg ketamine. Our findings suggest that there were no significant differences between the three task forms when ketamine was given at 0.8 mg/kg , 0.2 mg/kg ketamine (green) and 0.8 mg/kg ketamine (blue) conditions for Monkey A and Monkey Z. The higher right corner box plot was the average planning time or movement time in the early-postinjection period over saline (gray), 0.2 mg/kg ketamine (green) and 0.8 mg/kg ketamine (blue) conditions. All error bars are SEM. *10.1523/ENEURO.0015-23.2023.t4-1Extended Data Table 4-1Table 4-1, PDF file.Statistics for the plan-0.8/Tplan-saline and Tmove-0.8/Tmove-saline, where Tplan-0.8 (Tmove-0.8) and Tplan-saline (Tmove-0.8) denotes the mean planning (movement) time under the 0.8 mg/kg ketamine and saline conditions, respectively. Our analysis revealed a general trend that planning time was more affected than movement time, although this difference only reached statistical significance in the AVM-NC task  and Tplan-saline (Tmove-0.8) denotes the mean planning (movement) time under the 0.8 mg/kg ketamine and saline conditions, respectively. All error bars are SEM. Statistical significance is denoted by *p < 0.05, **p < 0.01. Download Figure 5-1, TIF file.Comparison of planning time and movement time during the early-postinjection period of each task per subject. To assess the effects of 0.8 mg/kg ketamine on planning time and movement time, we calculated the prolonged index for all each task. The prolonged index is defined as the ratio of TF = 1.450, p = 0.2516) and movement time , as illustrated in To examine the significance of the effect in prolonging planning and movement times among the AVM, AVM-NC and AVM+WM tasks, we analyzed the prolonged part of planning time and movement time with 0.8 mg/kg ketamine and performed statistical analysis across the three tasks. Specifically, we calculated the average planning time and movement time of each session in the early-postinjection and obtained the prolonged change relative to saline conditions by subtracting the values with 0.8 mg/kg ketamine by the mean values of all saline conditions. Our results showed that there was no significant difference across the three tasks in planning time  time relative to saline at a dose of 0.8 mg/kg during the early-postinjection phase of the task. Download 10.1523/ENEURO.0015-23.2023.f6-2Extended Data Figure 6-2a, The mean planning time of both monkeys between early (blue) and late (orange) sessions. b, c, The performance of Monkey A and Monkey Z. Download Figure 6-2, TIF file.The planning time of early and late saline sessions in the AVM-NC task. The mean interval time between early and late sessions was 14.3 d. p = 0.033). Similar trends were observed in Monkey A  and Monkey Z . However, we observed no significant changes in accuracy, movement time, and percentage trial done after approximately two weeks of intermittent ketamine administration . In addition, we observed no significant differences in the accuracy, planning time, movement time, and trial done percentage in the AVM and AVM+WM tasks. Thus, in our results, ketamine exhibited no detectable chronical effects except for the planning time in the AVM-NC task.Additionally, to investigate the chronic effect of low-dose ketamine as used in our experiment, we analyzed the performance of three early and three late sessions under saline condition in the AVM-NC task . We took the mean values of each session, and used estimation statistics  to examip < 0.0001), AVM+WM , and AVM-NC  to that of the quarter of trials with the longest total reaction time . Our results indicated a significant difference in accuracy between fast and slow reaction time trials in in each task: AVM  following an injection of 0.8 mg/kg ketamine during a 60-min period. Specifically, trials were divided into two groups within each daily session: the fast response group  and the slow response group . The figure shows a comparison of the accuracy between the fast response group (Fast) and the slow response group (Slow). All error bars correspond to the SEM. Significance levels are denoted as follows: *p < 0.001), −0.6016 for AVM-NC task (p < 0.001), and −0.4257 for AVM+WM task (p < 0.001). Taken together, these findings suggest a negative correlation between accuracy and reaction time in the tasks we investigated following the administration of 0.8 mg/kg ketamine. The individual performance of each subject is presented in Extended Data Subsequently, we conducted further analysis to investigate the correlation between accuracy and reaction time of trials after 0.8 mg/kg ketamine administration. Based on previous results  and AVM+WM task  = 2.13, p = 0.1336). However, the trial-complete index was significantly reduced with 0.8 mg/kg ketamine in the AVM-NC task compared with saline  and 0.2 mg/kg ketamine . This figure shows the percentage of complete trials in early-postinjection phase under saline (Nacl), 0.2 mg/kg ketamine (Ket 0.2), and 0.8 mg/kg ketamine (Ket 0.8) in each task. The error bars indicate the SEM. Statistical significance is denoted by *In this study, we conducted an in-depth analysis of the effects of low-dose ketamine, specifically subanesthetic doses, on AVM in monkeys. Our results demonstrate profound ketamine-induced behavioral deficits, including: (1) decreased accuracy in three tasks, including the AVM, AVM+WM, and AVM-NC tasks, with a stronger impact in the AVM+WM task than the AVM task; (2) prolonged reaction times, particularly in planning time rather than movement time; (3) increased reaction time during daily sessions in trials with lower accuracy; (4) induced more errors when the visual cue changes; and (5) reduced the ratio of completed trials to all trials in AVM-NC task.Our study shows that ketamine impaired animals’ performance in AVM, AVM+WM, and AVM-NC tasks at the dose of 0.8 mg/kg but not 0.2 mg/kg, with the latter condition exhibiting similar performance compared with the saline condition. The does dependency is consistent with previous findings that ketamine induces acute impairment of spatial WM tasks above 0.25 mg/kg .In our experiments, the AVM task was trained on the animals with the same visual cues for over a half year before this experiment. It indicates that ketamine impairs the retrieval of well-learned abstract rules in the AVM task, which is in line with a previous study showing that anesthetic ketamine impaired the retrieval of information from post-training memory components . The ketOne possible explanation for the negative impact on learning/performance is through NMDA receptor blockade. However, ketamine may also exert its effects through other mechanisms. For example, ketamine can activate the GABAergic system  and incrIn contrast to previous studies that used eye gaze tasks , we usedUnder the condition of administering 0.8 mg/kg of ketamine, there was a decrease in the prolonged change in planning and movement time for the AVM+WM task compared with the AVM and AVM-NC tasks . One posAdditionally, we observed a negative correlation between reaction time and accuracy after ketamine injection, indicating that the effects of ketamine vary on a trial-by-trial basis, and that there may be a common mechanism underlying this variation, affecting both the quality (as reflected by reduced accuracy) and the speed (as reflected by prolonged reaction time). Our findings suggest that the prolonged reaction time, which is easy to measure in a clinical setting, can serve as an effective indicator of the ketamine effects on higher cognition.As a study investigating the effects of ketamine on behavior, our results provide important insights for further understanding the underlying neural mechanisms. We examined the nonspecific effects of ketamine on motivation by calculating the level of engagement of the subjects, as reflected by the ratio of completed trials to the total number of trials. To exclude such nonspecific effects, we reported the accuracy and reaction time results in the main text based on the data of completed trials only. Our findings suggest that ketamine tends to lower the level of engagement of animals in the AVM-NC task with a statistically significant difference and in the AVM task with a downward trend, while the level of engagement in the AVM+WM task was not affected by ketamine . One posPrevious research has shown that ketamine (0.3 mg/kg) caused attention deficits in humans, which can be detected using electroencephalogram and horizontal electrooculogram . SimilarAn interesting result obtained in our study is that ketamine significantly induced more incorrect responses when the visual cue is changed from the previous trial in each task. This result is consistent with prior findings that PFC injury or lesion can impair behavioral performance in an AVM+WM task, particularly when visual stimuli change from the previous trials . HoweverTo investigate the effect of ketamine on perseveration, a phenomenon similar to performance after PFC injury  and chroAccording to these results, we can infer that low-dose ketamine mainly impacts the brain’s ability to flexibly coordinate decision-making and actions to achieve the current goal , as well


Mycobacterium marinum is a ubiquitous water‐borne non‐tuberculous mycobacterial (NTM) pathogen. In humans, M. marinum infections are acquired through direct inoculation of skin wounds and are almost exclusively localized to skin and soft tissues. Pulmonary infection with M. marinum is extremely rare, and to our knowledge, invasive endobronchial disease has not been reported. Here, we present a case of a 71‐year‐old immunocompetent male surfer with invasive endotracheal M. marinum granulomatous disease. The patient was successfully cured with a regimen of azithromycin 250 mg daily, ethambutol 900 mg (15 mg/kg) daily and rifampicin 600 mg daily for 12 months following culture conversion. This case highlights several important concepts: Firstly, M. marinum infection, including invasive endobronchial infection, should be considered a rare cause of NTM pulmonary disease. Secondly, endotracheal infection can be successfully eradicated with this selected therapeutic regimen. Finally, the absence of M. marinum skin or soft‐tissue infection in this patient, raises the possibility that human disease might also be acquired via inhalation of M. marinum contaminated water in rare circumstances. Mycobacterium marinum infections are acquired through direct inoculation of skin wounds and are almost exclusively localized to skin and soft tissues. Pulmonary infection with M. marinum is extremely rare, and to our knowledge, invasive endobronchial disease has not been reported previously. Here, we present a case of a 71‐year‐old apparently immunocompetent male surfer with invasive endotracheal M. marinum granulomatous disease. The patient was successfully cured with a 12‐month regimen of azithromycin 250 mg daily, ethambutol 900 mg daily and rifampicin 600 mg daily.In humans,  Mycobacterium marinum is a water‐borne non‐tuberculous mycobacteria (NTM) ubiquitously distributed in aquatic environments.M. marinum causes skin and soft tissue disease, resulting in chronic granulomatous ulcers, nodules and nodular lymphangitis. Infection occurs via direct inoculation, when contaminated water is exposed to skin wounds or abrasions. While contemporary chlorination practices have reduced M. marinum outbreaks in public swimming pools,M. marinum continues to be an important cause of human skin infection in individuals who engage in natural outdoor marine activities or cleaning and maintenance of fish tanks.M. marinum infections are nearly always localized to skin and soft tissues, in part due to the portal of inoculation and the growth characteristics of the organism; which demonstrates optimal growth at 30°C and suboptimal replication at 37°C., M. marinum infection results in locally invasive disease into deeper soft‐tissue structures, including tenosynovitis, osteomyelitis, arthritis, and bursitis.M. marinum is extremely rare; with only five cases in apparently ‘immunocompetent’ individuals reported in the literature to date.M. marinum infection has never been reported to our knowledge.In humans, M. marinum had cultured from an expectorated sputum sample.A 71‐year‐old Caucasian male never a smoker was referred to our service for investigation and management of suspected NTM pulmonary disease. The patient had a 24‐month history of productive cough with tenacious purulent sputum, mild left upper lobe nodular infiltrates, cylindrical bronchiectasis and an endoluminal soft tissue irregularity of the distal trachea on thoracic Computed Tomography Figure . Acid‐fa1 1.57 L, 55.1% predicted), allergic rhinosinusitis, varicella zoster virus reactivation, gastroesophageal reflux disease and keratoacanthomas. Medications were inhaled fluticasone propionate/salmeterol xinafoate (200/6 mcg) 2 puffs daily, pantoprazole 40 mg daily, intranasal mometasone 50 mcg spray daily. He was prescribed 1–2 short courses of oral prednisone (25 mg for 5–7 days) most winters in recent years for poorly controlled asthma.Medical history included childhood‐onset allergic asthma with moderately‐severe fixed expiratory airflow limitation  and rifampicin 600 mg daily, for a total of 12 months after bronchoscopic culture conversion. S. marcescens and B. gladioli coinfections were also treated with a 14‐day course of trimethoprim‐sulfamethoxazole at treatment outset. At 3‐month follow‐up, there was a complete resolution of clinical symptoms and the patient returned to surfing in the ocean. Clinical improvement was also associated with complete resolution of bronchial secretions with smear and culture conversion from bronchoscopy washings. After 12‐months of treatment, white‐light bronchoscopy demonstrated near‐complete resolution in macroscopic endotracheal abnormalities; with resolution of the erythematous mucosal protuberance affecting the lateral wall of the distal trachea  pulmonary disease.S. marcescens and B. gladioli coinfection in this clinical presentation; the histological finding of abundant mycobacteria associated with caseating granulomas suggests M. marinum as the causative pathogen.It should be recognized that although this patient did not have a detectable immunodeficiency syndrome; asthma, inhaled corticosteroids use, intermittent oral steroids and gastroesophageal reflux disease are known host risk factors non‐tuberculous mycobacterial infection.M. marinum demonstrates optimal growth at 30°C and more limited replication at 37°C, we cultured the pathogen on liquid culture media at 36°C after 17 days, demonstrating the ability of the isolate to replicate at temperature ranges present at the distal tracheal mucosal interface.M. marinum infection has been underestimated as incubation at the optimal growth temperature of 30°C is not usually utilized for respiratory samples .Although M. marinum infection to include invasive endobronchial granulomatous disease which is treatable using a three‐drug combination antibiotic regimen.This case expands the spectrum of human All authors contributed to drafting and editing of the manuscript.None declared.The authors declare that appropriate written informed consent was obtained for the publication of this manuscript and accompanying images.


Incorrect lineup identification decisions have devastating consequences on the wrongfully accused and for society when guilty perpetrators remain at large. This paper heeds the call by the National Academy of Sciences to utilize technology to increase correct identifications and minimize incorrect identifications. We found that the simultaneous interactive lineup procedure, which allows witnesses to actively explore the lineup members’ faces, increases discriminability compared to traditionally used video and simultaneous photo lineups. Interactive lineups were beneficial regardless of perpetrator encoding viewpoint . In the real world, eyewitnesses may view a perpetrator from multiple perspectives. The interactive procedure allows witnesses to adjust face viewpoints to match their memory of the crime, which may be beneficial for memory retrieval. Accurate witness identification is a cornerstone of police inquiries and national security investigations. However, witnesses can make errors. We experimentally tested whether an interactive lineup, a recently introduced procedure that enables witnesses to dynamically view and explore faces from different angles, improves the rate at which witnesses identify guilty over innocent suspects compared to procedures traditionally used by law enforcement. Participants encoded 12 target faces, either from the front or in profile view, and then attempted to identify the targets from 12 lineups, half of which were target present and the other half target absent. Participants were randomly assigned to a lineup condition: simultaneous interactive, simultaneous photo, or sequential video. In the front-encoding and profile-encoding conditions, Receiver Operating Characteristics analysis indicated that discriminability was higher in interactive compared to both photo and video lineups, demonstrating the benefit of actively exploring the lineup members’ faces. Signal-detection modeling suggested interactive lineups increase discriminability because they afford the witness the opportunity to view more diagnostic features such that the nondiagnostic features play a proportionally lesser role. These findings suggest that eyewitness errors can be reduced using interactive lineups because they create retrieval conditions that enable witnesses to actively explore faces and more effectively sample features. Eyewitness identification is a cornerstone of police inquiries and national security investigations. However, mistaken identification has been implicated in about 70% of U.S. wrongful convictions . FailureNevertheless, the technology used to conduct identifications has not fundamentally changed over the past century. The static 2D photo lineup is the most widely used procedure worldwide . A lineuA large body of research dedicated to improving discriminability has accumulated. In laboratory studies, participant witnesses are shown a mock crime and tested under different lineup conditions. Many studies have compared sequential to simultaneous lineups, wherein, respectively, witnesses view the members one at a time versus all together. In research comparing simultaneous to sequential lineups, some studies have found discriminability is higher in simultaneous lineups  versus noninteractors , G = 2.06, P = 0.98; profile encoding: interactors  versus noninteractors , G = 0.98, P = 0.83]. An analysis of all participant data (including noninteractors) is in SI Appendix. SI Appendixprovides ID decision frequencies by confidence level without and with interactors , and analyses of decision time by experimental condition (Table S1). None of the differences in discriminability can be explained by decision time differences across the lineup conditions.Thirty-one participants in the interactive condition did not interact with any of the faces in the twelve lineups as instructed. We did not anticipate this and therefore did not preregister that we would remove participants who failed to follow instructions. However, by not interacting, the interactive lineup becomes equivalent to the photo lineup, making it difficult to meaningfully compare procedures. Further, discrimination accuracy did not differ for interactors compared to noninteractors, suggesting the participant groups were similarly motivated to make an accurate identification  – 13 parameters [3 lineup conditions × 4  and σtarget]. To test if any differences in d were statistically significant, we fit a series of reduced models , constraining d to be the same across two conditions and compared the fit of the reduced to the full model (full model explained the data well (χ2(14) = 12.39, P = 0.575). Discriminability was larger in interactive (d = 1.32) than photo (d = 1.12) lineups, but this difference was not statistically significant (χ2(1) = 3.58, P = 0.058). Discriminability was significantly larger in interactive compared to video (d = 1.00) lineups (χ2(1) = 7.48, P = 0.006). There was no significant difference in discriminability between photo and video lineups (χ2(1) = 1.02, P = 0.313). For profile encoding, the full model explained the data well (χ2(14) = 12.32, P = 0.581). Discriminability was significantly larger in interactive (d = 1.04) than in photo  = 19.02, P < 0.001) and video  = 16.86, P < 0.001) lineups, but did not differ between photo and video lineups (χ2(1) = 0.00, P = 1.000).Discriminability . This could explain why lineups that have correlated memory signals (owing to shared features across faces) yield higher discriminability than showups   .target  and σb (the correlation) to differ across the lineup conditions. Again, the model-estimated σtarget was constrained to be the same across the lineup conditions, and the confidence criteria could vary. We found that for both front- and profile-encoding, the correlation is smallest and d is largest in the interactive condition, consistent with the distributional account (SI Appendixfor model fits).To investigate the foregoing, we fit a model allowing μOverall, the signal-detection modeling validates the ROC analysis results, though for front-encoding, increased discriminability for the interactive compared to the photo lineup did not reach statistical significance in the modeling. The discriminability improvement observed for interactive lineups is most parsimoniously explained by the distributional account.SI Appendixfor results with noninteractors included). This suggests active exploration and encoding specificity boost discriminability. Further, the modeling results suggest interactive lineups move the memory strength distributions for innocent and guilty lineup members further apart, ostensibly because they increase the number of diagnostic features available.We compared the simultaneous interactive to simultaneous photo and sequential video lineups, two widely used police procedures worldwide. Encoding angle (front or profile) was systematically controlled. For front-encoding, discriminability was significantly higher in simultaneous interactive compared to sequential video lineups and simultaneous photo lineups; but, this latter difference was statistically significant only in the ROC analysis and not the modeling. For profile-encoding, discriminability was higher in simultaneous interactive compared to simultaneous photo and sequential video lineups, which did not differ from each other (see P = 0.058). The front-encoding findings suggest that active over passive exploration enhances feature sampling  , and thehttps://osf.io/rkafh/).The methods and analyses were OSF preregistered (www.mturk.com) and compensated £2.70 for participating. Totally, 75 individuals were excluded based on a failed attention check (n = 71) and technical problems during the experiment (n = 4). The final sample (N = 475) was 52% female and 47% male (1% preferred not to say), and 73% Caucasian, 14% Black, 7% Asian, and 6% other.In all, 550 participants were recruited using Amazon Mechanical Turk .Participants were randomly assigned to one of the six between-subject conditions (front-interactive A 2 (encoding: front or profile view) × 3  × 2 (target: present or absent) mixed design was used. Encoding and procedure were varied between subjects, while target was varied within subjects. Each participant learned twelve faces at encoding (front view or profile view) and was tested on twelve lineups, of which, six were target-present and six were target-absent. The outcome variables were identification accuracy and decision confidence.We used ROC analyses because this method has been successfully applied in lineup research  and can 28Ethical approval was granted from the Science, Technology, Engineering and Mathematics Ethical Review Committee at the University of Birmingham.Materials were adapted from Colloff et al. .n = 8) and female (n = 4), and Caucasian (n = 6) and South Asian (n = 6). Different photographs of the target were used at encoding and retrieval to maintain reliably, generalizability, and to test face recognition rather than image recognition , each showing a face from shoulders up. The targets were male  and a six-person target-absent lineup  for each target face was constructed for each lineup procedure . Fillers were selected to match the targets’ physical appearances . To minimize position effects, two lineups with randomised face orders were generated for each target. Colloff et al.  ensured Participants performed the experiment using a computer and were randomly assigned to a condition. First, consent and demographic information was collected. Next, participants were sequentially presented with twelve faces for 5 s each and were instructed to pay attention as they would be asked questions about the faces later. Participants then watched a 1-min cartoon about a cat and a mouse as a distractor task. Afterward, participants were presented with the instructions and told that they should attempt to identify the targets that they previously saw. Participants were informed that the target may not be present in the lineup, and they should reject the lineup if need be by answering “not present”. Participants in the interactive condition were instructed to use their computer mouse to click on one face and to drag it to see it from different angles. They were informed that all faces would move together and that it is crucial to engage with the faces as their mouse movements will be recorded. They could only proceed to the lineup decision once they had moved their mouse. Participants in the photo condition were presented with six frontal-pose faces simultaneously, and they could proceed after 30 s. Participants in the video condition were presented with six 20-s videos that were shown sequentially, and they could only proceed once all videos had been displayed.Subsequently, the twelve lineups were presented in a random order. After each lineup, participants made a decision and provided their using an 11-point Likert-type rating scale, ranging from 0% “guessing” to 100% “completely certain.”Finally, participants answered an attention check question about what happened in the distractor task video and reported whether they had experienced any technical difficulties. Participants were thanked and debriefed.Appendix 01 (PDF)Click here for additional data file.