PRIORITY CLAIM

This application claims the benefit of the filing date of Australian Patent Application Serial No. AU 2019902055, filed Jun. 12, 2019, and claims the benefit of the filing date of Australian Patent Application Serial No. AU 2018904297, filed Nov. 13, 2018, the contents and disclosure of each of which is hereby incorporated herein in its entirety by this reference.

TECHNICAL FIELD

The present disclosure relates generally to digital user interfaces, for example, for mobile devices. Some embodiments relate to technology adapted to provide a user interface via presentation of two-dimensional content via three-dimensional display objects rendered in a navigable virtual space.

BACKGROUND

Any discussion of the background art throughout the specification should in no way be considered as an admission that such art is widely known or forms part of common general knowledge in the field.

The ubiquity of high-speed data networks has, in recent years, resulted in a dramatic increase in the amount of digital video content being generated and consumed. The demand for video content, particularly over mobile devices (e.g., tablets and phones), has resulted in new and efficient techniques for delivering high-quality digital video content to viewers. For example, dedicated content streaming services are accessible by mobile devices for all manner of content types, including live sport broadcasts, movies, TV shows and webcasts.

In many instances, it is desirable to present video accompanying digital content in association with the content being streamed to the mobile device user. For example, during a live sports stream, it may be desirable to show player statistics at contextually relevant times. Such information may also include paid advertising that is presented for a period of time.

However, current techniques for displaying such video accompanying digital content typically involves presenting the information overlaying the streamed content, which necessarily obstructs some portion of the content.

In a mobile context, where screen real estate is already at a premium, obstructing the already compacted video image can be particularly problematic and is known to adversely affect the overall user experience. Indeed, users may choose not to use a streaming service that is heavy on video accompanying digital content, even though such content may ultimately be beneficial to the user.

It would be advantageous if there was provided a technique for presenting video accompanying digital content on a mobile device that did not obstruct the associated video stream. It would also be advantageous if such a technique allowed for streaming service providers to deliver any desired amount or type of video accompanying digital content so as to maximize the user experience and/or revenue associated with providing the streaming service.

BRIEF SUMMARY

In accordance with a first aspect of the present disclosure, there is provided a computer implemented method for providing video accompanying digital content on a display of a mobile device, the method comprising: displaying a video stream on the mobile device display in a two-dimensional display format; responsive to determining an instruction (optionally a user instruction) for the video accompanying digital content, causing the display of the mobile device to transition from the two-dimensional display format to a three-dimensional display format, the transition step comprising: rendering the video stream and video accompanying digital content as separate objects within a three-dimensional virtual space such that each of the video accompanying digital content objects are displayed in a different spatial location to the video stream object thereby ensuring that the video stream displayed by the corresponding object is unobstructed by the video accompanying digital content.

In an embodiment, when in the three-dimensional display format, the three-dimensional virtual space can be navigated by a user of the mobile device to view a desired one or more of the rendered objects, the navigation affected by movement of the mobile device in real space.

In an embodiment, movement of the mobile device is determined based on an output of a gyroscope embedded in the mobile device.

In an embodiment, when in the three-dimensional display format, the user views the three-dimensional virtual space through a virtual camera located in the center of the virtual space such that movement of the mobile device simulates looking around the virtual space.

In an embodiment, the orientation of the virtual camera is adjusted based on the output of the gyroscope.

In an embodiment, the output of the gyroscope is representative of a pitch, yaw and roll of the mobile device.

In an embodiment, the rendered objects are spatially anchored within the three-dimensional virtual space.

In an embodiment, when in the two-dimensional display format, the video stream is mapped and displayed in a three-dimensional image plane that is sized and aligned to the aspect ratio and local orientation of the display of the mobile device.

In an embodiment, the step of transitioning comprises transforming the three-dimensional image plane.

In an embodiment, the image plane is attached to the mobile device's local camera and wherein the step of transitioning further comprises changing a coordinate space from a local camera coordinate system to a virtual space coordinate system while retaining the transform of the image plane relative to the local camera.

In an embodiment, when in the three-dimensional format, the image plane interpolates over time from original transform to anchored virtual space transform.

In an embodiment, responsive to determining an instruction for returning to the two-dimensional format, the method further comprises hiding the three-dimensional virtual space and re-snapping the image plane to the display of the mobile device for full screen viewing.

In an embodiment, the step of returning to the two-dimensional format further comprises changing the coordinate space from the virtual space coordinate system to the local camera coordinate system while retaining the transform of the image plane relative to the local camera.

In an embodiment, when in two-dimensional format, the image plane interpolates over time from original transform to new local camera aligned transform.

In an embodiment, the instruction is a predefined touch screen gesture.

In an embodiment, the video stream is a live video stream.

In an embodiment, the video accompanying digital content is dynamic content associated with the live video stream.

In an embodiment, the dynamic content is loaded, synchronized with the video stream content and kept in background processes while in the two-dimensional display format.

In an embodiment, the video accompanying digital content is user interactive content that is contextually relevant to the video stream and wherein the method further comprises allowing the user to interact with the interactive content through predefined user gestures.

In an embodiment, the video accompanying digital content comprises feedback that is presented in an associated object at moments of interest determined using artificial intelligence.

In an embodiment, the method further comprises rendering a user interface object in the virtual space that is operable to allow a user to input predefined phrases and/or emoticons that are displayed to other users in a live feed.

In an embodiment, at least one of the video accompanying digital content objects is located in the virtual space adjacent to the video stream object.

In accordance with a second aspect there is provided a computer system for providing video accompanying digital content on a mobile device, the system comprising: a video server configured to stream video to the mobile device over a mobile communications network; a content server configured to communicate the video accompanying digital content to mobile device over the mobile communications network; wherein the mobile device is operable to display the video stream in a two-dimensional display format; and responsive to determining an instruction for the video accompanying digital content, cause the display of the mobile device to transition from the two-dimensional display format to a three-dimensional display format, the transition step comprising: rendering the video stream and video accompanying digital content as separate objects within a three-dimensional virtual space such that each of the video accompanying digital content objects are displayed in a different spatial location to the video stream object thereby ensuring that the video stream displayed by the corresponding object is unobstructed by the video accompanying digital content.

One embodiment provides a computer implemented method for providing a navigable video-centric user interface on a mobile device having a display screen, the method including:rendering of a video content display object on the display screen, wherein the video content display object is configured to present two-dimensional video data retrieved from a video data source, wherein the rendering includes rendering the video content display object as a three-dimensional object in a three-dimensional virtual space, such that the video content display object includes a two-dimensional display pane on which the two dimensional video data is presented,rendering in the three-dimensional virtual space one or more further content display objects, the one or more further content display objects being configured to present two-dimensional content retrieved from respective two-dimensional content data sources; andenabling field-of-view navigation of the three-dimensional virtual space via user inputs delivered to the mobile device, thereby to allow a user of the mobile device to selectively view desired portions of the video content display object and one or more further content display objects.

One embodiment provides a computer implemented method for providing data to a navigable video-centric user interface on a mobile device having a display screen, wherein the mobile device is configured to perform a method including:rendering a three-dimensional virtual space;rendering in the three-dimensional virtual space a plurality of content display objects, wherein each content display object includes two-dimensional display pane configured for presenting two-dimensional content;data mapping each of the content display objects with a source of two-dimensional rendering content, such mobile device is operable to retrieve from respective designated locations two dimensional content for display by the content display objects, such that the two-dimensional content is transformed and presented in the three-dimensional virtual space;enabling field-of-view navigation of the three-dimensional virtual space via user inputs delivered to the mobile device, thereby to allow a user of the mobile device to selectively view desired portions of the video content display object and one or more further content display objects.

One embodiment provides a method for displaying content at a mobile device having a display screen, the method including:configuring the mobile device to render a three-dimensional virtual space;being responsive to an input to transition in a visually seamless manner between:a first mode, which includes rendering a content display object to occupy substantially all of the display screen, wherein the content display object is configured to display two-dimensional content; anda second mode, which includes rendering a three-dimensional content display object in the three-dimensional virtual space, wherein the three-dimensional display object displays the same two-dimensional content;such that by transitioning from the first mode to the second mode, the user is able to continue seamlessly viewing the two-dimensional content, and navigate the three-dimensional virtual space thereby to in parallel view one or more further content display objects respectively configured to display dynamic content retrieved from remote content sources.

Reference throughout this specification to “one embodiment,” “some embodiments” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment,” “in some embodiments” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment, but may. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

As used herein, unless otherwise specified the use of the ordinal adjectives “first,” “second,” “third,” etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements/features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements/features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.

As used herein, the term “exemplary” is used in the sense of providing examples, as opposed to indicating quality. That is, an “exemplary embodiment” is an embodiment provided as an example, as opposed to necessarily being an embodiment of exemplary quality.

The description below refers to “systems” and “modules.” The term “module” refers to a software component that is logically separable (a computer program), or a hardware component. The module of the embodiment refers to not only a module in the computer program but also a module in a hardware configuration. The discussion of the embodiment also serves as the discussion of computer programs for causing the modules to function (including a program that causes a computer to execute each step, a program that causes the computer to function as means, and a program that causes the computer to implement each function), and as the discussion of a system and a method. For convenience of explanation, the phrases “stores information,” “causes information to be stored,” and other phrases equivalent thereto are used. If the embodiment is a computer program, these phrases are intended to express “causes a memory device to store information” or “controls a memory device to cause the memory device to store information.” The modules may correspond to the functions in a one-to-one correspondence. In a software implementation, one module may form one program or multiple modules may form one program. One module may form multiple programs. Multiple modules may be executed by a single computer. A single module may be executed by multiple computers in a distributed environment or a parallel environment. One module may include another module. In the discussion that follows, the term “connection” refers to not only a physical connection but also a logical connection (such as an exchange of data, instructions, and data reference relationship). The term “predetermined” means that something is decided in advance of a process of interest. The term “predetermined” is thus intended to refer to something that is decided in advance of a process of interest in the embodiment. Even after a process in the embodiment has started, the term “predetermined” refers to something that is decided in advance of a process of interest depending on a condition or a status of the embodiment at the present point of time or depending on a condition or status heretofore continuing down to the present point of time. If “predetermined values” are plural, the predetermined values may be different from each other, or two or more of the predetermined values (including all the values) may be equal to each other. A statement that “if A, B is to be performed” is intended to mean “that it is determined whether something is A, and that if something is determined as A, an action B is to be carried out.” The statement becomes meaningless if the determination as to whether something is A is not performed.

The term “system” refers to an arrangement where multiple computers, hardware configurations, and devices are interconnected via a communication network (including a one-to-one communication connection). The term “system,” and the term “device,” also refer to an arrangement that includes a single computer, a hardware configuration, and a device. The system does not include a social system that is a social “arrangement” formulated by humans.

At each process performed by a module, or at one of the processes performed by a module, information as a process target is read from a memory device, the information is then processed, and the process results are written onto the memory device. A description related to the reading of the information from the memory device prior to the process and the writing of the processed information onto the memory device subsequent to the process may be omitted as appropriate. The memory devices may include a hard disk, a random-access memory (RAM), an external storage medium, a memory device connected via a communication network, and a ledger within a CPU (Central Processing Unit).

