Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> Synchronization between locations is an important factor for enabling remote shared experiences. Still, experimental data on what is the acceptable synchronization level is scarce. This paper discusses the synchronization requirements for watching online videos together - a popular set of services that recreate the shared experience of watching TV together by offering tools to communicate while watching. It studies the noticeability and annoyance of synchronization differences of the video being watched, as well as the impact on users' feelings of togetherness, both for voice chat and text chat. Results of an experiment with 36 participants show that when using voice chat, users notice synchronization differences sooner, are more annoyed and feel more together than when using text chat. However, users with high text chat activity notice synchronization differences similar to participants using voice chat. <s> BIB001 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> Ambient Intelligence is a new paradigm in which environments are sensitive and responsive to the presence of people. This is having an increasing importance in multimedia applications, which frequently rely on sensors to provide useful information to the user. In this context, multimedia applications must adapt and personalize both content and interfaces in order to reach acceptable levels of context-specific quality of service for the user, and enable the content to be available anywhere and at any time. The next step is to make content available to everybody in order to overcome the existing access barriers to content for users with specific needs, or else to adapt to different platforms, hence making content fully usable and accessible. Appropriate access to video content, for instance, is not always possible due to the technical limitations of traditional video packaging, transmission and presentation. This restricts the flexibility of subtitles and audio-descriptions to be adapted to different devices, contexts and users. New Web standards built around HTML5 enable more featured applications with better adaptation and personalization facilities, and thus would seem more suitable for accessible AmI environments. This work presents a video subtitling system that enables the customization, adaptation and synchronization of subtitles across different devices and multiple screens. The benefits of HTML5 applications for building the solution are analyzed along with their current platform support. Moreover, examples of the use of the application in three different cases are presented. Finally, the user experience of the solution is evaluated. <s> BIB002 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> The growing success of tablets and smartphones has shifted the focus of the interactive TV industry to the introduction of second screen applications. One example is second screen companion apps that offer extra information about a television program, often synchronized with what happens on screen. In this paper, we investigate a second screen companion app, from the perspective of the viewers and producers of such apps. Based on observations and interviews with viewers and producers, and actual usage data of a companion app from Google Analytics, we present several insights and recommendations for how to design companion apps related to ease of use, timing, social interaction, attention and added value. <s> BIB003 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> Due to recent development of TVs in the direction of highly interactive multimedia platforms, interactive TV (iTV) applications gain popularity. In terms of control possibilities a variety of input modalities have become available, though effects on performance and user experience of different age groups when controlling different iTV applications remain unclear. We present an empirical investigation comparing three input modalities (tablet, freehand gestures, remote) for controlling two iTV applications (Photo Browser, Nutrition Tracker) used by older and younger adults. Results show that all three independent variables had significant influence on performance, while we did not find influence of age or application on user experience. Overall tablet input based on a mirrored TV screen showed the best performance and was preferred by both age groups. Older adults were overall slower and showed a particularly large performance gap with the remote in comparison to younger adults. <s> BIB004 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> The term `object-based media' has become commonly used to describe the representation of media content by a set of individual assets together with metadata describing their relationships and associations. At the point of consumption these objects can be assembled to create an overall user experience. The precise combination of objects can be flexible, and responsive to user, environmental and platform specific factors. An object-based approach like this can serve end users more effectively; by optimising the experience to best suit their access requirements, the characteristics of their playback platform or personal preferences. This paper presents an overview of the key characteristics and potential benefits of Object-Based Broadcasting and a case study of an object-based audio documentary, which can change duration in response to implicit or explicit input from a listener. Further examples will be given to illustrate the capabilities of a future object-based broadcasting system. <s> BIB005 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> HTML5 is driving a strong trend towards interoperable Web-based applications, enabling a wider range of devices to run this kind of applications. The key challenge of next-generation media applications is to federate cooperative devices to provide multi-device experiences overcoming current second screen solutions within the connected TV industry. There is a gap on the experience of users, since they perceive devices as isolated pieces of applications when they would prefer to have a single experience through multiple devices at the same time. This paper proposes a unified methodology and a common specification over Web Components for the adaptation of a single application, seamlessly running different instances on one or more devices simultaneously, according to the multi-device context of the user and the specific features of the devices. The solution presented in this paper extends current Web standards towards an interoperable architecture and offers broadcasters and media application developers the possibility to easily design applications that will automatically provide a unique consistent experience across the connected devices. The architectural design is targeted to be included in the roadmap of the standards. <s> BIB006 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> In recent years, mobile devices have become a part of our daily lives--much like television sets had over the second half of the 20th century. Increasingly, people are using mobile devices while watching television. We set out to understand this behavior on a minute-by-minute quantified level as well as users' motivations and purposes of device use while watching television. We conducted a novel mixed-methods study inside seven households with fourteen instrumented phone and tablet devices, capturing all app launches and app use durations, correlated with the moment in the television program when they occurred. Surprisingly, we found little difference between the volume of device use during programs and commercials. Our two main findings are that 1) participants often joined family members in the TV room to physically be together; when they lack interest in the program, they spend the majority of the show on a secondary device and watch TV only during key moments. 2) Virtually none of participants' app and web use during TV consumption was directly related to the running show. With our study, we set the stage for larger-scale investigations into the details of mobile interactions while watching television. Our novel method will aid future work of the community as a means of fully understanding multi-device use alongside television consumption. <s> BIB007 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> This paper describes the development and implementation of an open communication standard between Internet-connected TVs and companion screens, over the home network. Content providers know that the better Internet connectivity and prevalence of personal mobile devices is encouraging our audiences to seek more interactive experiences across multiple screens. In order to deliver a coherent integrated user experience, with content on all screens presented according to a common timeline, the application on the companion device needs to discover what is being shown on the TV and what the timeline position is. DVB-CSS provides a standardised way to enable this synchronisation between the TV and any personal device on the home network. We describe its development, use cases and early prototype implementation. <s> BIB008 </s> Cross-device media: a review of second screening and multi-device television <s> The infrastructure of the multi-screen home <s> Media synchronization is getting renewed attention with ecosystems of connected devices enabling novel media consumption paradigms. Social TV, hybrid TV, and companion screens are examples that are enabling people to consume multiple media streams at multiple devices together. These novel use cases require media synchronization, as unfortunately there are substantial delay differences between the various delivery routes for television and streaming media. Broadcasters have started using proprietary solutions for over-the-top media synchronization, such as media fingerprinting or media watermarking technologies. Given the commercial interest in media synchronization and the disadvantages of proprietary technologies, consumer-equipment manufacturers, broadcasters, and telecom and cable operators have started developing a new wave of international standards for media synchronization. This article provides an overview of recently published standards from the most relevant bodies: IETF, ETSI, MPEG, DVB, HbbTV, and W3C. <s> BIB009
The notion of the connected home is making the vision of multi-device content more realisable. Given that recent reports are indicating that internet-enabled TVs are now the norm (according to a 2015 survey, internet-enabled TV penetration is [50%  ), that internet-enabled mobile devices are ubiquitous, and that most of us run these systems on a home network, the infrastructure is mostly in place. Therefore, recent work has looked into how we may integrate such additional infrastructure into normal practice for users with lightweight, extensions of their current usage. There are several key considerations when devising cross-device infrastructure and systems for multi-screen experiences. First of all, there is the effort required for the user to pair the device with the television. This depends largely on whether one needs to download specialist apps, own certain hardware, or need some prior technical knowledge to initiate the multi-device experience. Secondly, there is the content's mutability to the device. Such factors are dictated by internet connection, the method of distribution, and indeed the user's chosen device (i.e. screen size and form factor). Finally, there is latency-the extent to which content temporally synchronises across devices is vitally important for both for collocated devices BIB003 and remote shared viewing BIB001 . For instance, high synchrony is essentially for subtitles BIB002 and directors commentaries  . We now discuss these three factors in the context of existing technologies to provide cross-device media and remark on its potential future. Cross network standards, such as the BBC's Universal Control API  , are not yet a norm. However, the BBC's API allows for the control of set-top boxes and similar devices on a given network, and for all devices on a network to communicate. Such standards not only allow for interfacing for numerous Internet-of-Things (IoT) devices but also have strong implications for the design of custom interfaces. Recent work by from the BBC, by Vinayagamoorthy et al. BIB008 , proposes an open communication standard between Internet-connected TVs and companion screens, over the home network. This work provides a standardised, frame-accurate, method to enable synchronisation between the TV and any personal device on the home network. Such endeavours, along with work such as that by Zorrilla et al. BIB006 , which also looks at a method for synchronous cross-device experiences, may allow for seamless simultaneous multi-screen video experiencessuch as alternate angles of live footage-with no perception of lag in the near future. Another avenue of exploration in this area is audio watermarking (also known as audio fingerprinting), which has been used by a number of applications to synchronise, in time, the device and the television (e.g. in BIB007 ). Audio watermarking overcomes the complications of client-server architecture by simply embedding imperceivable (over 20 k) high-frequency audio cues in a piece of video material. These markers serve as triggers to cue content on the device, resulting in reasonable synchrony between screens. This method, however, has major shortcomingsmost notable inconsistencies between devices, poor handling of user interactions, and its proprietary nature BIB009 . It has therefore not been widely adopted. Parallel to this, however, much work has looked at standards for multi-stream, multi-device media synchronisation (see BIB009 for more detail). Due to international standards, such as hbbTV (Hybrid Broadcast Broadband TV), BIB004 networked synchrony is improving vastly for collocated and remote shared viewing. This noted, technologies which allow for frame-accurate synchrony between a device and television are very much still at the research stage-for example, the aforementioned the open communication standard proposed by BBC Research and Development BIB008 . However, as such technologies become more widely adopted by broadcasters and developers, it is likely that such networked experiences offer a strong future to cross-device experiences. The future envisioned concept of media in the home is Object-Based Broadcasting (OBB) (see Fig. 4 ). OBB is a responsive internet-driven approach to broadcasting in which separate elements of a programme are sent with associated metadata and assembled on the user's end to describe their assembly in accordance with the user's needs BIB005 . Such technologies would allow for the full customisation of content to the extent where each person has their own experience, curated to them.