BACKGROUND

Embodiments of the inventive concept described herein relate to a method and a program for providing augmented reality image using depth data.

Augmented reality (AR) refers to a technology for superimposing virtual objects such as texts and graphics in a real world and displaying the virtual objects as an image. The AR technology was in R&D and test application steps until the mid-2000s. However, the AR technology has recently entered the commercialization step as the technological environment is in place. In particular, with the advent of smartphones and the development of Internet technology, the AR technology has started to draw attention.

The most common way to implement AR is to make the user feel as if a virtual reality and an actual reality are mixed, by photographing the real world using the camera of a smartphone and overlaying and outputting the pre-generated computer graphics on the photographed image. In this method, most AR applications are implemented in smartphones because a user may easily obtain the image of reality using the camera of the smartphone, and computer graphics may be easily implemented through the computing function of the smartphone. Besides, with the recent advent of wearable devices in the form of glasses, the interest in the AR technology is increasing.

SUMMARY

The AR image needs to be augmented at the correct location in the reality space to provide a user with a sense of reality. However, in the case where the spatial placement of an object (e.g., a marker) with which the AR image is combined is changed as a user moves, when the image frame at a specific time point is missing, the AR image may be placed at an inappropriate location, and may be unnaturally displayed in the reality space and the sense of reality decreases as the AR image shakes. Embodiments of the inventive concept provide a method and a program for providing augmented reality image using depth data, which provides a realistic and unshakable AR image even when a user moves or a frame is missing, by placing the virtual image data, which is a two-dimensional image at an appropriate location in the reality space using depth data.

Embodiments of the inventive concept provide a method and a program for providing augmented reality image using depth data that may generate an AR image without additional masking data by assigning a region to be transparent to each pixel of a virtual reality (VR) image based on added depth data.

Moreover, embodiments of the inventive concept provide a method and a program for providing augmented reality image using depth data that may transform VR content and may utilize the transformed VR content as AR content, without separately producing an AR image augmented and expressed in a reality space separately from the VR image.

The technical problems to be solved by the present inventive concept are not limited to the aforementioned problems, and any other technical problems not mentioned herein will be clearly understood from the following description by those skilled in the art to which the inventive concept pertains.

According to an exemplary embodiment, a method in which a client provides an augmented reality (AR) image includes receiving, by the client, virtual image data from a server, wherein the virtual image data includes color data and depth data, determining, by the client, a location, at which each pixel is to be displayed in a reality space, based on the depth data, and displaying the virtual image data in the reality space based on the determined location.

Furthermore, in another embodiment, the depth data is included in a separate channel distinguished from a color data channel for each pixel, and the depth data and the color data are synchronized and transmitted.

Moreover, in another embodiment, the virtual image data is a two-dimensional image stored for each pixel by obtaining depth data at each point during capture or generation.

Also, in another embodiment, the determining of the location includes determining, by the client, a specific depth as a transparency adjustment criterion and determining whether to perform transparency processing, by distinguishing a depth range based on the transparency adjustment criterion. The transparency adjustment criterion is to set a boundary line of content to be displayed on a screen.

In addition, in another embodiment, the depth range is a plurality of regions divided based on a plurality of depths by setting the plurality of depths depending on the transparency adjustment criterion.

Furthermore, in another embodiment, the virtual image data further includes acquisition location data and image direction data. The determining of the location includes comparing current location data obtained by the client with the acquisition location data and comparing playback direction data obtained by the client with the image direction data and adjusting a location of a pixel in the virtual image data based on the comparison result.

Moreover, in another embodiment, the determining of the location further includes adjusting, by the client, a color or a chroma of each pixel in the virtual image data based on a light irradiation direction of a reality space.

Furthermore, in another embodiment, the method further include correcting, by the client, reality image data based on an output delay time, when the client is a device for outputting combined image data obtained by combining the virtual image data and reality image data obtained by a camera. The output delay time is a time required until the captured reality image data is output on a screen after the reality image data is captured.

According to an exemplary embodiment, a program for providing an AR image using depth data that is stored in a medium to execute the above-described method for providing an AR image using depth data in combination with a computer that is a piece of hardware.

According to an exemplary embodiment, a device providing an AR image includes a virtual image data receiving unit receiving virtual image data from a server, a control unit determining a location, at which each pixel is to be displayed in a reality space, based on the depth data, and an image output unit displaying the virtual image data in a reality space based on the determined location. The virtual image data includes color data and depth data.

According to an exemplary embodiment, a method in which a server generates virtual image data for implementing AR in a client includes obtaining, by the server, color data for each pixel of the virtual image data provided to the client at a specific time point, obtaining and storing depth data for each pixel of the virtual image data, and synchronizing, by the server, the color data and the depth data to transmit the synchronized color data and the synchronized depth data to the client. The depth data is used during correction based on first virtual image data at a first time point when second virtual image data at a second time point is not received. The second time point is a time point at which a virtual image data transmission period elapses from the first time point.

Furthermore, in another embodiment, the virtual image data further includes acquisition location data and image direction data. The client compares current location data with the acquisition location data, compares playback direction data with the image direction data, and adjusts a location of a pixel in the virtual image data based on the comparison result, and the current location data and the playback direction data are obtained by the client in real time or at a unit time.

