BACKGROUND

Video conferencing technology can be used to facilitate audiovisual communication between users that are remote from one another. For example, a first device may capture audio and video of a first user and transmit the audio and video data to a second device for presentation to a remotely positioned second user to facilitate near-real time communication between the first user and the second user. Similarly, the second device may capture audio and video of the second user and transmit the audio and video data to the first device.

Video conference is widely used in diverse contexts and provides remote communication in a manner that preserves nonverbal signals and cues (e.g., facial expressions) that are lost in communication techniques that lack a visual component (e.g., phone communications). However, existing video conferencing technology is associated with a number of shortcomings.

For example, existing systems for facilitating remote visual communication between users often utilize high bandwidth in order to provide only average or poor video or image frame quality. Although many video conferencing techniques attempt to compress image frames to reduce bandwidth usage, such techniques often give rise to compression artifacts that can negatively affect user experiences. For example, a greater compression ratio may lead to a greater number of artifacts present in the compressed image frames.

Furthermore, users participating in video conferencing often desire that they will be perceived by other participants under preferred conditions. For example, a user participating in a video conference may desire to be captured by a camera from a particular perspective (e.g., a head-on perspective with the camera horizontally aligned with the user's eyes) so that other participants perceive the user according to the particular perspective. Other examples of preferred conditions may be related to camera distancing from the captured user, lighting conditions, user attire, user personal hygiene and/or grooming conditions, and/or others.

In many instances, preferred conditions for video conferencing are difficult or impractical to maintain. For example, users often participate in video conferencing from diverse locations, in diverse contexts, and with diverse devices. For instance, a user may receive a video call (e.g., from a supervisor) while the user is not appropriately dressed or groomed for such a call, or when the user is not in an ideal location for such a call. Furthermore, in some instances, a user receives a video call while the user is in a private location, and the user may desire to refrain from broadcasting details of the private location to other users in the video call.

In some instances, a user operating a mobile electronic device (e.g., a smartphone) may be able to achieve an ideal, head-on camera perspective by holding their device from a position extended away from their body, but such positioning may be tiresome to maintain for long periods of time and may present obstacles to locomotion by and/or about the user. Furthermore, a user attempting to maintain such a position may cause inadvertent shaking or movement of the camera, which may cause motion blur and/or other undesired effects.

In view of the difficulties associated with maintaining such a head-on camera perspective, users often revert to holding a mobile electronic device from a relaxed position with their hand closer to their body and at an elevation below their head. Such positioning causes the camera of the device to capture the user's face from a perspective that looks upward toward the user's face. Such an upward perspective, however, is often regarded as unflattering, as it can provide undesirable views of the nostrils of the user and of accentuated neck features of the user (e.g., as the user orients their head downward toward the camera and device).

Furthermore, as noted above, users often participate in video conferencing from various locations and/or while traveling between locations. Accordingly, users often encounter limited or varying connectivity to wireless networks while participating in video conferencing, which can negatively affect the quality of the transmitted image frames or can entirely preclude participation in video conferencing (e.g., reverting to audio only).

Any difficulties associated with video conferencing, such as those described hereinabove, may be exacerbated when more than two users are participating in video conferencing.

Still furthermore, some video conferencing platforms are configured to display image frames received from other participants in an arrayed manner, such that users can simultaneously view more than one other participating user. Some platforms further present the image frames in a stylized arrangement, such as to emulate audience seating with representations of the other participants placed among the audience. However, because different devices may capture participating users from different perspectives and depths, a simultaneous presentation of multiple other participating users may give rise to discrepancies between adjacently arranged presentations of other participating. For example, adjacently “seated” presentations of other participating users may appear as though the other participants are at different depths, even though one would expect other participants that are “seated” next to one another to appear at about the same depth.

Thus, for at least the foregoing reasons, there is an ongoing need and desire for improved systems and methods for facilitating remote visual communication.

The subject matter claimed herein is not limited to embodiments that solve any disadvantages or that operate only in environments such as those described above. Rather, this background is only provided to illustrate one exemplary technology area where some embodiments described herein may be practiced.

BRIEF SUMMARY

Disclosed embodiments are directed to systems, methods and devices that facilitate low-bandwidth remote visual communication.

Some embodiments include a system for facilitating low-bandwidth remote visual communication based on real-time imaging. The system includes one or more processors and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the system to perform various acts.

The system is configured to receive input from a user to initiate a process of generating a holodouble of the user. The holodouble includes a photorealistic three-dimensional (3D) simulated representation of the user. The system is also configured to obtain image data of the user within an environment that is captured by a camera from multiple perspectives of the user relative to the camera, and to deconstruct the image data to obtain a set of sparse data that identifies one or more attributes associated with the image data the user.

The system is also configured, in some instances, to use a holodouble training model to generate and train the holodouble of the user based on the set of sparse data and obtained image data. The holodouble is, in some instances, enabled to be rendered from a plurality of different perspectives associated with the user, based on new image data of the user, including the multiple perspectives of the user relative to the camera, when the image data was obtained for generating and training the holodouble, as well as from one or more different perspectives of the user relative to one more cameras that capture the new image data of the user in subsequent image processing.

The holodouble training model includes a machine learned or machine learning model that has been trained on image data of humans and corresponding sparse data of attributes associated with the image data of the humans to generate and render simulations of the humans from different perspectives.

The system is also configured, in some instances, to render a representation of the holodouble to the user concurrently while capturing the new image data of the user, receive input from the user comprising approval of the holodouble, and complete training of the holodouble by saving the holodouble for subsequent use. The subsequent use includes one or more remote visual communication sessions in which the user is visually simulated at one or more second devices as the holodouble corresponding to and concurrently with images that are captured of the user during the one or more remote visual communication sessions at a first device.

Embodiments also include corresponding systems that select, utilize, modify and/or display corresponding holodouble representations of a user during remote visual communication sessions based on real-time imaging obtained of the user and to simulate the user during the remote visual communication sessions (without sending the real-time imaging) to the remote destination device(s) that render the holodouble representations of the user.

The holodouble is rendered, in some instances, on a display device with at least one of a context or perspective of the user that is different than a corresponding context or perspective of the user that is represented in the images captured of the user by the camera during the visual communication session.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

Additional features and advantages will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the teachings herein. Features and advantages of the invention may be realized and obtained by means of the instruments and combinations particularly pointed out in the appended claims. Features of the present invention will become more fully apparent from the following description and appended claims or may be learned by the practice of the invention as set forth hereinafter.

