BACKGROUND OF THE DISCLOSURE

Field of the Disclosure

The present disclosure relates to an in-car cloud VR technology, and more particularly, to an in-car cloud VR device and method capable of adaptively responding to traveling conditions and providing a realistic video to a user in a traveling vehicle.

Related Art

Recently, virtual reality (VR) devices have appeared, but are not widely spread like smartphones due to problems such as high price, low resolution that hinders a sense of immersion, and a lack of VR content. In particular, physical discomfort such as dizziness that may occur due to a discrepancy between a user's head movement and a visible VR video may be a problem that needs to be resolved. To solve this problem, motion-to-photon (MTP) redundancy needs to be reduced to be within 10 to 20 ms. To achieve this, there are technical difficulties in a wireless environment (for example, WiFi, mobile network, or the like).

Korean Patent No. 10-0820132 (2008 Apr. 1) relates to a method and system for image intra-prediction mode estimation, transfer, and organization, and discloses a technology capable of improving efficiency of a coding process by providing a method and system for estimating a pixel prediction mode used in a video encoding or decoding process, transferring a pixel prediction mode between a video encoder and decoder, and ordering an intra-prediction mode of a pixel.

RELATED ART DOCUMENT

[Patent Document]

(Patent Document 0001) Korean Patent No. 10-0820132 (2008 Apr. 1)

SUMMARY OF THE DISCLOSURE

The present disclosure provides an in-car cloud VR device and method capable of adaptively responding to traveling conditions and providing a realistic video to a user in a traveling vehicle.

The present disclosure also provides an in-car cloud VR device and method capable of overcoming a limitation of a physical space of the existing VR device and providing an effective VR/AR videos even under vehicle traveling conditions.

The present disclosure also provides an in-car cloud VR device and method capable of processing a video in real time through hybrid interworking between a cloud server and a VR device.

In an aspect, an in-car cloud VR device is provided. The in-car cloud VR device includes: a GIS information acquisition unit that detects a location of a vehicle through a GPS sensor installed in a traveling vehicle and acquires the GIS information near the vehicle; an thing information acquisition unit that collects videos around the vehicle through at least one camera sensor installed in the vehicle and acquires surrounding thing information around the vehicle; a movement information acquisition unit that acquires movement information of a user in the vehicle through at least one gyro sensor; and an FOV video determination unit that determines a first FOV video based on the GIS information, updates the first FOV video based on the surrounding thing information, and determines a second FOV video within the first FOV video based on the movement information of the user.

The GIS information acquisition unit may match the GPS signal acquired from the GPS sensor with GIS coordinates defined on a digital map and then acquire the GIS information.

The thing information acquisition unit may divide the video into a plurality of frames, identify at least one dynamic object by image analysis for each frame, and acquire object information on the at least one dynamic object from a previously constructed object DB as the surrounding thing information.

The movement information acquisition unit may measure head movement associated with the user's head through the at least one gyro sensor, and generate head coordinate data on the head movement to determine a user's gaze direction.

The FOV video determination unit may determine, as the first FOV video, a stereoscopic video of a 360° field of view centered on a location of the vehicle based on the GIS information, and synthesize virtual graphic corresponding to the surrounding thing information with the stereoscopic video to perform the update.

The FOV video determination unit may determine the second FOV video according to the gaze direction corresponding to the user's head movement, and perform Foveated Rendering on the first FOV video when the gaze direction is maintained for a predetermined time or longer to increase a resolution of an actual field of view expressed by the second FOV video.

The FOV video determination unit may include an FOV determination module that receives information on the movement of vehicle and the user to determine the field of view (FOV) according to the location of the vehicle and the user's gaze direction, a gaze video capture module that captures a gaze video for the FOV, and a video encoding module that encodes the gaze video to generate stereoscopic video expressed to the user.

In another aspect, an in-car cloud VR method is provided. The in-car cloud VR method includes: detecting a location of a vehicle through a GPS sensor installed in a traveling vehicle and acquiring the GIS information near the vehicle; collecting videos around the vehicle through at least one camera sensor installed in the vehicle and acquiring surrounding object information around the vehicle; acquiring movement information of a user in the vehicle through at least one gyro sensor; and determining a first FOV video based on the GIS information, updating the first FOV video based on the surrounding object information, and determining a second FOV video within the first FOV video based on the movement information of the user.

