A high resolution aerial 3D display using a directional backlight
CC Attribution - NoDerivatives 2.0 UK: England & Wales:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Kakeya, Hideki
Ishizuka, Shuta
This paper describes a high resolution aerial 3D display using a time-division multiplexing directional backlight. In this system an aerial real image is generated with a pair of large convex lenses. The directional backlight is controlled based on the detected face position so that binocular stereoscopy may be maintained for a moving observer. By use of the directional backlight, the proposed system attains autostereoscopy without any moving parts. A wide viewing zone is realized by placing a large aperture convex lens between the backlight and the LCD panel. With the advantage of time-division multiplexing, a high resolution 3D image is presented to the viewer. 
 So I'm Hidekake from the University of Tsukuba, and this is a joint work with my student, Shute Ishizuka. So the title of my talk is High Resolution Area 3D Display Using a Directional Backlight. So the contents is like this. First, I'm going to talk about the background of this research. So I'm working with medical doctors, surgeons, and of course 3D. Medicine is one of the promising areas for application of 3D. And the highest priority for medical doctors for 3D or the imaging is resolution. So high resolution is the highest priority. But another, well, merit of 3D for medical purpose is direct manipulation style. What is direct manipulation is, you know, for example, when we think of surgery simulator, here's part of the brain or any part of the body. And for surgery simulation or rehearsal, the operator can touch and manipulate, as if he's touching with his hand directly. So this is direct manipulation. So this kind of style is very good for training or rehearsal of surgery. So to do that, one option is 3D aerial display. So this is the work. Oops. Can I play the video? Okay. Thank you. So this is our work about 16 years ago. So the image is floating here. Oh, sorry. And I presented this system in SeaGraph 2000. It's about 16 years ago. And how it works is like this. So here's a pair of lenses. And with the lenses, the real image of this screen is generated in here. So when the viewer looks at the image, it's floating. So the eye's focus is fixed here. So it's very easy to recognize something is in here. And only with that configuration, just a plane is floating. So to show depth, we have to do a certain amount of parallax, binomial parallax, to do that. So actually we can do that without forcing the viewer to wear glasses. Because of the lenses, the light rays impinging on the right eye once gathers here. And the light impinging on the left eye once gathers here. So if we place a filter here, for example, in the system we constructed 16 years ago, we showed images with two different polarization on the screen. And we set a pair of polarization filters with two different polarization. Then the image, two different images can be shown to the left eye and the right eye. Of course, when the viewer's position changes, these points also change. So we have to track the position of the viewer. And by moving the filter in accordance with the viewer's position, then we can keep on showing two different images to the right eye and the left eye of the viewer. And to show the right image to the right eye and the left eye, for example, to show a triangle here and make it stable, we calculate the optical rays like this. So to show point A in the air, we prolong the line segment between the eye and point A and calculate the refraction of the lens. And we show the pixel of A here on the screen. And we do the same thing for the left eye here. And we do that for all the pixels. And of course, this calculation changes when the viewer's position changes. So we can keep on tracking the viewer's position and we can keep on calculating to show the stable image in the air. And of course, the lens includes aberration or distortion. So on the screen, we show the image distorted in the reverse direction like this. Then from the viewer's position, the right cubic can be observed. Of course, this way of distortion changes when the viewer's position changes. So when the viewer moves sideways, then we show the image like this on the screen. And from the viewer's position, the right image can be observed like this. So the problem of this system, actually, I made the first system 16 years ago, but it was not commercialized. So the main problem is the physical motion of the filter. For demonstration, it's okay, but for practical use, it's not very good to present a system with a physical motion. And there have been some solutions to do away with the physical motion filters. One is to use electronic filters. And actually, I also made a kind of prototype system. But with only one layer of electronic filter, we can move in the horizontal direction or in the vertical direction, but we can't move in the depth direction. If we place multiple layers of filters, then the viewer can move forward and backward. But if we place multiple shutters, then the image becomes very dark. So it's not practical. So to solve this problem, we use directional backlight in place of mobile filters or electronic filters. So this is the Notosruss-Sorg display using directional backlight. Actually, we presented the system based on these optics last year in this conference. So in this system, these are the lenses. And we keep the distance between the backlight and the lens the same as the focal length of the elemental lenses. Then the light is collimated so we can realize directional backlight to the right eye and to the left eye. So here to show the image only to the right eye, here we show the right eye image. And we just extend the line from the eye to the center of the elemental lenses and we make these parts bright for this person and make these parts bright for this person. And then both of the viewers can see the image here only with the right eye and not from the left eye. And for the left eye, we make these parts bright. So this is also connecting the eye and the center of the lens. And then only from the left eye, the left eye image can be observed. So by alternating between these two modes, we can achieve time division multiplexed auto-Australia display. But not only with this configuration, the image observed is like this. Because we are using lens array, the seam of the lens is quite distinct and annoying. And also we are enlarging the backlight pixels. So there are lots of artifacts because the pixel structure is magnified with the lenses. So to make it practical, we use lens array like this. So the height of the elemental lens is very short and the phase of each row is different. So the bright part of the lens is different in each row. And with this lens and also we combine a vertical diffuser, then the bright part and dark part are averaged. And the image shown becomes quite uniform. And because this is a vertical diffuser, it does not disturb the directionality of light to the right and the left eye. Because basically the right and the left are parallel to the ground. So this is the image observed. So it's quite good and practical. And last year we demonstrated the system, but at that time the image was quite dark. So not very good. But we replaced the backlight with the brighter one and now it's much better. And here we are using it for liver surgery simulator. As I said, we are working with surgeons of University of Tsukuba hospitals. And this is the cancer and blood vessels are running. So it's similar and it's already used. So actually this is a scene of training of medical students in University of Tsukuba hospital. So back to the presentation's main story. We combined these two ideas to make a high resolution floating area display without moving parts. So this is the actual system we made. So this part is the aerial display we made 16 years ago. This part is the directional backlight which we used in the last year's presentation. So here and here the light to the left and the right eye gathers. And we realized directional backlight to gather light here and here. So when these parts, of course, when the viewer changes, the depth of these gathering points have to be shifted. We can achieve that by changing the interval of the bright parts. So these points can be moved backwards and forward by changing the interval of the bright parts. So the viewer can move forward and backward. So this is the actual system we made from the front. So this is a large lens to generate a real image in the air. And this is the backlight part. So this is the backlight. So lenses and the diffusers and so on. So this is the specification of the prototype system. So the scale is like this. Not very large, but we're using a 27, 24 inch LCD panel. So this is the observed image. So we took these pictures with the Fujifilm camera, 3D camera. And so this is the left eye view and the right eye view. And we attached a position sensor to the camera. With that configuration, we can realize motion paths also. So again, the video, would you click the slide please? Oops. Oh, it's not working. No. Okay, so actually the video is not working, but no. Actually the video is the video of this scene. Okay. And discussion. Actually, I just talked about the kind of realistic situation, but in actual optics, the lens includes aberration. So to show a large image, we actually because of the aberration, the backlight for the left eye and the right eye overlaps like this. To avoid this phenomenon, one effective way is a very simple way, is to narrow the aperture of the elemental lens to decrease the blur. And another point is that if we use the lens with large focal length here, then the aberration becomes smaller. So we can show a very large image without crosstalk. It's just a simulation based. We haven't made this system. But theoretically, we can enlarge the area of a little crosstalk. So conclusion. Our full HD area, 3D image is printed without moving parts by using a direction backlight. And to show a large image, we have crosstalk, as I said, small elemental lenses, and use of convex lens with larger focal length is effective. Thank you.
 So I'm Hidekake from the University of Tsukuba,  and this is a joint work with my student, Shute Ishizuka.  So the title of my talk is High Resolution Area 3D Display Using a Directional Backlight.  So the contents is like this. First, I'm going to talk about the background of this research.  So I'm working with medical doctors, surgeons, and of course 3D.  Medicine is one of the promising areas for application of 3D.  And the highest priority for medical doctors for 3D or the imaging is resolution.  So high resolution is the highest priority.  But another, well, merit of 3D for medical purpose is direct manipulation style.  What is direct manipulation is, you know, for example, when we think of surgery simulator,  here's part of the brain or any part of the body.  And for surgery simulation or rehearsal, the operator can touch and manipulate,  as if he's touching with his hand directly. So this is direct manipulation.  So this kind of style is very good for training or rehearsal of surgery.  So to do that, one option is 3D aerial display.  So this is the work. Oops.  Can I play the video?  Okay. Thank you.  So this is our work about 16 years ago.  So the image is floating here. Oh, sorry.  And I presented this system in SeaGraph 2000.  It's about 16 years ago. And how it works is like this.  So here's a pair of lenses.  And with the lenses, the real image of this screen is generated in here.  So when the viewer looks at the image, it's floating.  So the eye's focus is fixed here.  So it's very easy to recognize something is in here.  And only with that configuration, just a plane is floating.  So to show depth, we have to do a certain amount of parallax, binomial parallax, to do that.  So actually we can do that without forcing the viewer to wear glasses.  Because of the lenses, the light rays impinging on the right eye once gathers here.  And the light impinging on the left eye once gathers here.  So if we place a filter here, for example, in the system we constructed 16 years ago,  we showed images with two different polarization on the screen.  And we set a pair of polarization filters with two different polarization.  Then the image, two different images can be shown to the left eye and the right eye.  Of course, when the viewer's position changes, these points also change.  So we have to track the position of the viewer.  And by moving the filter in accordance with the viewer's position,  then we can keep on showing two different images to the right eye and the left eye of the viewer.  And to show the right image to the right eye and the left eye,  for example, to show a triangle here and make it stable,  we calculate the optical rays like this.  So to show point A in the air, we prolong the line segment between the eye and point A  and calculate the refraction of the lens.  And we show the pixel of A here on the screen.  And we do the same thing for the left eye here.  And we do that for all the pixels.  And of course, this calculation changes when the viewer's position changes.  So we can keep on tracking the viewer's position and we can keep on calculating  to show the stable image in the air.  And of course, the lens includes aberration or distortion.  So on the screen, we show the image distorted in the reverse direction like this.  Then from the viewer's position, the right cubic can be observed.  Of course, this way of distortion changes when the viewer's position changes.  So when the viewer moves sideways, then we show the image like this on the screen.  And from the viewer's position, the right image can be observed like this.  So the problem of this system, actually, I made the first system 16 years ago,  but it was not commercialized.  So the main problem is the physical motion of the filter.  For demonstration, it's okay, but for practical use,  it's not very good to present a system with a physical motion.  And there have been some solutions to do away with the physical motion filters.  One is to use electronic filters.  And actually, I also made a kind of prototype system.  But with only one layer of electronic filter,  we can move in the horizontal direction or in the vertical direction,  but we can't move in the depth direction.  If we place multiple layers of filters, then the viewer can move forward and backward.  But if we place multiple shutters, then the image becomes very dark.  So it's not practical.  So to solve this problem, we use directional backlight in place of mobile filters or electronic filters.  So this is the Notosruss-Sorg display using directional backlight.  Actually, we presented the system based on these optics last year in this conference.  So in this system, these are the lenses.  And we keep the distance between the backlight and the lens the same as the focal length of the elemental lenses.  Then the light is collimated so we can realize directional backlight to the right eye and to the left eye.  So here to show the image only to the right eye, here we show the right eye image.  And we just extend the line from the eye to the center of the elemental lenses  and we make these parts bright for this person and make these parts bright for this person.  And then both of the viewers can see the image here only with the right eye and not from the left eye.  And for the left eye, we make these parts bright.  So this is also connecting the eye and the center of the lens.  And then only from the left eye, the left eye image can be observed.  So by alternating between these two modes, we can achieve time division multiplexed auto-Australia display.  But not only with this configuration, the image observed is like this.  Because we are using lens array, the seam of the lens is quite distinct and annoying.  And also we are enlarging the backlight pixels.  So there are lots of artifacts because the pixel structure is magnified with the lenses.  So to make it practical, we use lens array like this.  So the height of the elemental lens is very short and the phase of each row is different.  So the bright part of the lens is different in each row.  And with this lens and also we combine a vertical diffuser, then the bright part and dark part are averaged.  And the image shown becomes quite uniform.  And because this is a vertical diffuser, it does not disturb the directionality of light to the right and the left eye.  Because basically the right and the left are parallel to the ground.  So this is the image observed. So it's quite good and practical.  And last year we demonstrated the system, but at that time the image was quite dark.  So not very good.  But we replaced the backlight with the brighter one and now it's much better.  And here we are using it for liver surgery simulator.  As I said, we are working with surgeons of University of Tsukuba hospitals.  And this is the cancer and blood vessels are running.  So it's similar and it's already used.  So actually this is a scene of training of medical students in University of Tsukuba hospital.  So back to the presentation's main story.  We combined these two ideas to make a high resolution floating area display without moving parts.  So this is the actual system we made.  So this part is the aerial display we made 16 years ago.  This part is the directional backlight which we used in the last year's presentation.  So here and here the light to the left and the right eye gathers.  And we realized directional backlight to gather light here and here.  So when these parts, of course, when the viewer changes, the depth of these gathering points have to be shifted.  We can achieve that by changing the interval of the bright parts.  So these points can be moved backwards and forward by changing the interval of the bright parts.  So the viewer can move forward and backward.  So this is the actual system we made from the front.  So this is a large lens to generate a real image in the air.  And this is the backlight part.  So this is the backlight.  So lenses and the diffusers and so on.  So this is the specification of the prototype system.  So the scale is like this.  Not very large, but we're using a 27, 24 inch LCD panel.  So this is the observed image.  So we took these pictures with the Fujifilm camera, 3D camera.  And so this is the left eye view and the right eye view.  And we attached a position sensor to the camera.  With that configuration, we can realize motion paths also.  So again, the video, would you click the slide please?  Oops.  Oh, it's not working.  No.  Okay, so actually the video is not working, but no.  Actually the video is the video of this scene.  Okay.  And discussion.  Actually, I just talked about the kind of realistic situation,  but in actual optics, the lens includes aberration.  So to show a large image, we actually because of the aberration,  the backlight for the left eye and the right eye overlaps like this.  To avoid this phenomenon, one effective way is a very simple way,  is to narrow the aperture of the elemental lens to decrease the blur.  And another point is that if we use the lens with large focal length here,  then the aberration becomes smaller.  So we can show a very large image without crosstalk.  It's just a simulation based.  We haven't made this system.  But theoretically, we can enlarge the area of a little crosstalk.  So conclusion.  Our full HD area, 3D image is printed without moving parts by using a direction backlight.  And to show a large image, we have crosstalk, as I said, small elemental lenses,  and use of convex lens with larger focal length is effective.  Thank you.
SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_0_671.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_672_2675.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_2676_2735.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_2736_2867.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_2868_3431.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_3432_7847.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_7848_10643.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_10644_10955.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_10956_13607.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_13608_13691.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_13692_13751.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_13752_17087.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_17088_17519.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_17520_20027.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_20028_20663.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_20664_21191.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_21192_21323.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_21324_21395.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_21396_22439.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_22440_22511.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_22512_22631.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_22632_22679.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_22680_23279.png SD_A_2016__A_high_resolution_aerial_3D_display_using_a_directional_backlight_gZjJgsUwMWs_23280_24335.png