Interpolating vertical parallax for an autostereoscopic 3D projector array
CC Attribution - NoDerivatives 2.0 UK: England & Wales:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Jones, Andrew V.
Nagano, Koki
Liu, Jing
Busch, Jay
Yu, Xueming
Bolas, Mark T.
Debevec, Paul
CONTEXT: We present a technique for achieving tracked vertical parallax for multiple users for a variety of autostereoscopic projector array setups including front- and rear- projection, and curved display surfaces. This “hybrid parallax” approach allows for immediate horizontal parallax as viewers move left and right, and tracked parallax as they move up and down, allowing cues such as 3D perspective and eye contact to be conveyed faithfully. OBJECTIVE: Projector arrays are well suited for 3D displays because of their ability to generate dense and steerable arrangements of pixels. We have developed a new autostereoscopic display utilizing a single dense row of 69 pico projectors. The projectors are focused on a 30x30cm vertically anisotropic screen that scatters the light from each lens into a vertical stripe while preserving horizontal angular variation. Each viewer’s eye observes the combined effect of image stripes from multiple projectors which combine to form a seamless 3D image. As every viewer sees a different 3D image, it is possible to customize each view with a different vertical perspective. Given a sparse set of tracked viewer positions, the challenge is to create a continuous estimate of viewer height and distance for all potential viewing angles to provide consistent vertical perspective to both tracked and untracked viewers. METHOD: Rendering to a dense projector display requires multiple-center of projection imagery, as adjacent projector pixels diverge to different viewer positions. If you assume constant viewer height and distance for each projector, viewers may see significant cross-talk and geometric distortion particularly when multiple viewers are in close proximity. We solve this problem with a custom GPU vertex shader projection that dynamically interpolates multiple viewer heights and distances within each projector frame. Thus, each projector’s image is rendered in a distorted manner representing multiple centers of projection, and might show an object from above on the left and from below on the right. RESULTS: We use a low-cost RGB depth sensor to simultaneously track multiple viewer head positions in 3D and interactively update the imagery sent to the array. Even though each user sees slices of multiple projectors, the perceived 3D image is consistent and smooth from any vantage point with reduced cross-talk. This rendering framework also frees us to explore different projector configurations including front and rear- mounted projector arrays and non-flat screens. Our rendering algorithm does not add significant overhead enabling realistic dynamic scenes. Our display produces full color autostereoscopic 3D imagery, with zero horizontal latency, and a wide 110o field of view which can accommodate numerous viewers. NOVELTY: While user tracking has long been used for single-user glasses displays, and single-user autosteroscopic display [Perlin et al. 2000] in order to update both horizontal and vertical parallax, our system is the first autostereoscopic projector array to incorporate tracking for vertical parallax. Our method could be adapted to other projector arrays [Rodriguez et al. 2007, Kawakita et al 2012, Kovacs and Zilly 2012, Yoshida et al 2011]. Furthermore, our display is reproducible with off-the-shelf projectors, screen materials, graphics cards, and video splitters.
 So, thank you for the introduction. What I'm going to talk about is how we can generate both horizontal and vertical parallax using a dense autostereoscopic projector array. Now, the brute-forced way of generating vertical parallax would be to have a full 2D array of video projectors, sort of like this, where each projector basically serves as an angular varying pixel. And if you have enough projectors and they're far enough away so you don't see any gaps, you would have full horizontal and vertical parallax. So the image would change as you move horizontally and vertically. Unfortunately, this is rather impractical because right now these require thousands and thousands of video projectors. So it would cost a lot of money and it would take up a lot of space, I think, as the previous talk referenced. But as projectors are becoming smaller and cheaper, it is possible to build a one-dimensional array of projectors for just providing horizontal parallax only. Now you now need to add some sort of diffusing screen to basically fill in the gaps. You can use a vertically diffusing screen to scatter the light from the projectors vertically. So each projector lens now gets scattered into a vertical plane. And then you can add a slight horizontal diffusion to fill in the horizontal gaps. And these horizontal gaps correspond to the spaces remaining between the projectors. And this will give you a seamless 3D image that will change as you move horizontally left and right. This is nice because in general people tend to move in the horizontal plane. People aren't jumping up and down all the time, but people are walking around your display. Similarly, you could have multiple people sitting side by side. However, people do tend to be different heights. Some people are taller, some people are shorter. And you'd want people who are shorter to be able to look up at your 3D scene and see the underside of the teapot. People who are taller would be able to look down and see the top of the teapot. However, with a horizontal parallax only display, you don't get any of this vertical parallax effects. In fact, the image sort of eerily follows you and just sort of shows force shortening as you move vertically. So our solution is to try and have a hybrid technique where we use autostereoscopic horizontal parallax. However, we combine it with user tracking of multiple users in order to dynamically update the vertical parallax for each viewer's position. This is possible because each viewer is, in effect, receiving their own customized view going with these angularly varying pixels. Now the challenge here is that we only have these sparse tracked viewers. We want to interpolate a continuous estimate of the viewer height across all directions. Secondly, we have to deal with the fact that tracking systems tend to have a lot of errors. They have a lot of latency and they also are fairly inaccurate. So our system needs to be able to handle all of this in sort of a way that reduces cross talk. So the idea of having a projector array actually goes back to the 1930s where Ive's built a projector array coming out of the area of lenticular printing where they had an array of slide projectors projecting onto a two-layer lenticular screen. It had one set of cylindrical lenses that would focus the light down onto a diffuse screen and then a second set of cylindrical lenses that redistribute those rays out to different directions. So in this case, each eye is having angularly different rays coming from different directions. Now the drawback of this is that you need to have very precise alignment of the projectors and the lenses. And secondly, at some point, the lenses will start to self-acclude. So this lens will include this adjacent lens. So this sort of limits the overall field of view that you can achieve. You can actually also achieve an autostereoscopic projector array with a simpler technique, as I alluded to earlier, where you just use a single screen with a wide vertical and isotropic scattering. Now the advantage of this is you don't need to have precise alignment of the projectors in the screen. But what's happening here is that the screen is basically preserving the original angular distribution of light from the projectors. So light going through the screen basically continues in the same direction as just diffused vertically. Now the one thing I also want to point out here is that there's no longer a sort of one-to-one correspondence between projectors and viewers. So for a particular viewer, they're actually seeing slices of pixels coming from multiple projectors. So here you have this projector illuminating the eye from the right and this projector illuminating light from the left. Now the similar effect has been observed in spinning isotropic displays such as, oh, before I get to that, there are actually commercial systems, or existing systems, I've used these isotropic, vertical and isotropic screens such as the holographic system, there's the Quartico-Anon or Simpon 2012, and the Ushida tabletop system from 2011. Now going back to those spinning isotropic screens, these have a single video projector but the isotropic screen moves reflecting light to different viewing positions. Now most of these systems have all assumed constant height and distance across all the viewers. Now I grew back in 2009, developed a teleconferencing system using a spinning mirror and here we actually modeled the fact that different viewers can be different heights in order to achieve accurate eye gaze. Now in order to do this we needed to sort of assume that there was a constant height at least across each projector frame. So we then used a concave mirror to refocus the light from the particular projector down to a sort of more localized viewing area. In this work I'm going to explain how to do the same thing but without using special optics and just sort of adjusting the rendering algorithm to localize the vertical parallax. This would be a top view. So actually this is an unfolded optical system. So really this projector would probably be above and we were reflecting off a spinning mirror and then that would go out to the lens. So this is assuming that there's a concave mirror that would reflect the lights out. This wouldn't work in a transmissive case because you need to have the screen basically magnifying the lenses. So here's a video of our system. We have a set of video projectors reflecting off an anisotropic screen. It has a wide 110 degree field of view. The content that you're seeing here on the screen is actually 3D geometry of a face scan in our light stage device or being rendered in real time. So as I mentioned there's 72 video projectors. We're using the Texas Instruments Pico projectors which are LCD projectors. We chose them basically as far as I know there were the smallest projectors that were commercially available so we can stack them really close together with only 1.6 degrees between projectors. This gives us a very high angular resolution and also the very wide field of view. Now the screen material has two components. There's a lenticular lens but this isn't a lenticular lens as people usually use it. Usually lenticular lenses are aligned so it will cylinders run vertically but in this case we're putting them horizontal so it just takes light coming in and scatters it into a vertical plane. The second component is a holographic diffuser which is not being used as a hologram but it's just a light shaping diffuser so that it provides a slight angular blur to fill in all those gaps between the projectors. Without any diffusion you actually see a set of stripes and each of these stripes corresponds to one lens of a different projector. With a little bit of horizontal diffusion then you can become a seamless image and ideally the amount of diffusion should be approximately the angular spacing between projectors. So there's a video just showing with and without the holographic diffuser. The other interesting effect is that adding the holographic diffuser actually slightly increases the perceived spatial resolution of the display because you're actually blurring in pixels that would be seen by the person on your right or on the left into your view. This is something that will be interesting to explore further in the future. We also have done both a front and rear projection system. In this case it's a transmissive screen using two degrees of horizontal diffusion plus the lenticular screen. You can also model it as a front projection system. In this case we painted the back of the lenticular screen black and just using the front surface reflection off the plastic. We only need to have one sheet of diffusion here because the light is actually transmitting through the holographic diffuser twice. Now one of the fun things here is that we were generating all our images on a single computer. We used four ATI affinity cards each with six video outputs for a total of 24. And then here's what the desktop looks like, we're using the display fusion commercial software to basically handle this massive desktop. Now to go from 24 video outputs to 72 we used the Matrox triple head to go video splitters that take one video input and split it into three outputs so we can get 72 displays all hooked up to a single computer. Next step is to calibrate the system. We model the calibration as a simple 2D rectification that lines all the projector images up to the display and we simply project a AR marker which we can automatically detect in a fixed camera. Now how do we actually render these images? And as I mentioned earlier we have to render these multiple center of projection images and these are because each eye sees images or pixels from multiple projectors and similarly each projector now has to generate different rays that are rendered for multiple viewers. So better way to look at this is what do these images actually have to look like? And this is what we render. And the first thing to notice is that this is a left sort of center and right projector is that you sort of have this unwrapped view of the face. So part of this image is rendered from the side of the head and another part of the image is sort of rendered more from the front because these pixels are going to be seen by different viewers. The next thing to remember is that different viewers could be different heights. So if we have a viewer on the left is now sort of at a straight on view they're going to see the front of the cube whereas a viewer from the in the middle might be higher up so they're going to see more of the top of the view and we want to be able to seamlessly transition between these two perspectives. So first I'm going to review how would you generate these multiple center of projection images for a single view of height? And one way you could do it is you would render a very large number of horizontal and vertical just traditional perspective views and then you could re-sample those images to create these multiple center of projection images. That's fairly inefficient because you're rendering a much larger number of views than you actually need sort of extra pixels and secondly you're adding an extra re-sampling step. So what we do is we take 3D geometry basically take our 3D vertices and we come up with a custom vertex projection algorithm implemented as a vertex shader that maps these vertices to the screen based on which viewer will see this particular vertex. So we start out with a particular vertex and then we trace array from a given projector out to find the nearest viewer. And the set of viewers for a constant height could be viewed as just a single circle of viewers all at the same height. So we find a viewer that will see this particular ray then we go up to the whatever height they are at and we trace back to the projector through the screen and this gives us the point where this particular vertex should actually be displayed. Now if we have multiple viewers at different heights we can take the nearest two viewers to a particular ray and represent them as two different circles at different heights and distances. We take the same ray and intersect it with both circles so we get two intersection points. Now neither of the tracked viewers would necessarily lie on this ray so we have to interpolate between the two nearest viewers. And we do this by looking at the angle between the intersection point and the original, the tracked viewer positions and then the computer weighted average based on these two angles. And then once we have that average point we can then re-project back onto the screen as before. Now in practice we use a, an average is computed using a Gaussian interpolation so as you move further away from a particular viewer there's a sort of a Gaussian falloff. We try and make the width of this Gaussian equivalent to the width of a human's shoulders so as long as someone's not actually looking over your shoulder there won't be any cross talk between views. Secondly we also incorporate a default height and distance so as you move further away from a particular tracked viewer position or if someone's untracked then they will still see a reasonable perspective. The third thing is that you can also modulate these weights by the confidence of your tracking system so if you are very confident in a particular viewer then you can use the tracked position and as someone maybe arrives or leaves and their confidence is lower then it will just transition back to the default height and distance. So this shows what the interpolated height would be. We do, for our system we track using a Microsoft connect which can track about two to six viewers. This provides both height and depth and here at the bottom is the interpolated height so these two regions here correspond to the interpolated height of these two viewers which smoothly moves as they move around. So this is what the interpolated vertical parallax looks like. On the left this is our proposed algorithm which as you move back and forth smoothly interpolates between different heights. Here on the right this is a technique where you assume that for each projector frame all the heights are constant and this results in image tearing basically where you're seeing images that were intended for the person on your left or the person on your right and they might be at different heights. There's two different algorithms here. One shows a abrupt fall off or transition from one height to the other and the other shows the smooth Gaussian. Most of the results I'm going to show later on are all using the smooth Gaussian. Using per vertex view blending also reduces crosstalk. So in this case we have a untracked viewer here that's staying stationary and we have this actively moving tracked viewer and as he moves close to this untracked viewer ideally this view remains fixed. Now if you use per vertex constant height across each projector frame there's going to be significant crosstalk whenever this tracked viewer moves close to this untracked viewer. So another thing we explore in the paper is different mirror shapes. So I mentioned earlier that using a flat screen you use a holographic diffuser to fill in the gaps. Now if you actually curve the shape of the screen using a convex mirror you can actually increase the spacing between the stripes. And what's happening here is that you're actually magnifying the position of the projector so that they are closer to the screen and have smaller gaps behind the screen. And this only really works for a front projection case. Now one issue is that this magnification is only really true for the center of the screen and as you move towards the edge of the screen different rays will no longer converge on a single point. So we came up with an algorithm that would basically improve this. You can start off with just averaging those intersection points and use that to project onto the screen as I described earlier and get a screen point. But then you can then refine the projector position by using the local neighborhood around this projected position and find a new intersected virtual projector position. And then we simulated a whole bunch of different mirror shapes even beyond the ones that we could physically build and there's more details about that in the paper. But these are the virtual positions and these are the images that you'd have to send to all these unbuilt displays. Here's the vertical parallax using a curved screen and you see there's still a consistent perspective. Our system runs in real time so we can have dynamic scenes. This is a dynamic scene with structured light geometry. And finally because we were all inspired by Star Wars and trying to recreate Princess Lair we stuck Princess Lair on the display. So this is LEGO Princess Lair until I can actually get a scan of Carrie Fisher. This is the best I can do. So future work, as projectors become smaller and cheaper we should be able to build a brighter much larger display. Perhaps one that actually goes four or three hundred and sixty degrees around. And now we're treating the tracking as pretty much a black box but as there are future improvements in tracking technology we believe that this should be easily incorporated into our system. And finally I mean one of the biggest hassles is just Windows is really not designed to spot this number of projectors and it's not auto stereoscopic. So if there's better ways of incorporating 3D functionality into the OS that would make the system a lot more practical. So in conclusion we've presented a hybrid parallax technique, it provides instantaneous horizontal parallax with tracked vertical parallax and I think this is a good compromise given how people tend not to move as much vertically but you can still get accurate perspective from all positions. It handles multiple users and it's robust to tracking errors. So that I will take any questions.
 So, thank you for the introduction.  What I'm going to talk about is how we can generate both horizontal and vertical parallax  using a dense autostereoscopic projector array.  Now, the brute-forced way of generating vertical parallax would be to have a full 2D array of  video projectors, sort of like this, where each projector basically serves as an angular  varying pixel.  And if you have enough projectors and they're far enough away so you don't see any gaps,  you would have full horizontal and vertical parallax.  So the image would change as you move horizontally and vertically.  Unfortunately, this is rather impractical because right now these require thousands  and thousands of video projectors.  So it would cost a lot of money and it would take up a lot of space, I think, as the previous  talk referenced.  But as projectors are becoming smaller and cheaper, it is possible to build a one-dimensional  array of projectors for just providing horizontal parallax only.  Now you now need to add some sort of diffusing screen to basically fill in the gaps.  You can use a vertically diffusing screen to scatter the light from the projectors  vertically.  So each projector lens now gets scattered into a vertical plane.  And then you can add a slight horizontal diffusion to fill in the horizontal gaps.  And these horizontal gaps correspond to the spaces remaining between the projectors.  And this will give you a seamless 3D image that will change as you move horizontally  left and right.  This is nice because in general people tend to move in the horizontal plane.  People aren't jumping up and down all the time, but people are walking around your display.  Similarly, you could have multiple people sitting side by side.  However, people do tend to be different heights.  Some people are taller, some people are shorter.  And you'd want people who are shorter to be able to look up at your 3D scene and see  the underside of the teapot.  People who are taller would be able to look down and see the top of the teapot.  However, with a horizontal parallax only display, you don't get any of this vertical parallax  effects.  In fact, the image sort of eerily follows you and just sort of shows force shortening  as you move vertically.  So our solution is to try and have a hybrid technique where we use autostereoscopic horizontal  parallax.  However, we combine it with user tracking of multiple users in order to dynamically  update the vertical parallax for each viewer's position.  This is possible because each viewer is, in effect, receiving their own customized view  going with these angularly varying pixels.  Now the challenge here is that we only have these sparse tracked viewers.  We want to interpolate a continuous estimate of the viewer height across all directions.  Secondly, we have to deal with the fact that tracking systems tend to have a lot of errors.  They have a lot of latency and they also are fairly inaccurate.  So our system needs to be able to handle all of this in sort of a way that reduces cross  talk.  So the idea of having a projector array actually goes back to the 1930s where Ive's built  a projector array coming out of the area of lenticular printing where they had an array  of slide projectors projecting onto a two-layer lenticular screen.  It had one set of cylindrical lenses that would focus the light down onto a diffuse  screen and then a second set of cylindrical lenses that redistribute those rays out to  different directions.  So in this case, each eye is having angularly different rays coming from different directions.  Now the drawback of this is that you need to have very precise alignment of the projectors  and the lenses.  And secondly, at some point, the lenses will start to self-acclude.  So this lens will include this adjacent lens.  So this sort of limits the overall field of view that you can achieve.  You can actually also achieve an autostereoscopic projector array with a simpler technique,  as I alluded to earlier, where you just use a single screen with a wide vertical and isotropic  scattering.  Now the advantage of this is you don't need to have precise alignment of the projectors  in the screen.  But what's happening here is that the screen is basically preserving the original angular  distribution of light from the projectors.  So light going through the screen basically continues in the same direction as just diffused  vertically.  Now the one thing I also want to point out here is that there's no longer a sort of  one-to-one correspondence between projectors and viewers.  So for a particular viewer, they're actually seeing slices of pixels coming from multiple  projectors.  So here you have this projector illuminating the eye from the right and this projector  illuminating light from the left.  Now the similar effect has been observed in spinning isotropic displays such as, oh,  before I get to that, there are actually commercial systems, or existing systems, I've used these  isotropic, vertical and isotropic screens such as the holographic system, there's the  Quartico-Anon or Simpon 2012, and the Ushida tabletop system from 2011.  Now going back to those spinning isotropic screens, these have a single video projector  but the isotropic screen moves reflecting light to different viewing positions.  Now most of these systems have all assumed constant height and distance across all the  viewers.  Now I grew back in 2009, developed a teleconferencing system using a spinning mirror and here we  actually modeled the fact that different viewers can be different heights in order to achieve  accurate eye gaze.  Now in order to do this we needed to sort of assume that there was a constant height  at least across each projector frame.  So we then used a concave mirror to refocus the light from the particular projector down  to a sort of more localized viewing area.  In this work I'm going to explain how to do the same thing but without using special optics  and just sort of adjusting the rendering algorithm to localize the vertical parallax.  This would be a top view.  So actually this is an unfolded optical system.  So really this projector would probably be above and we were reflecting off a spinning  mirror and then that would go out to the lens.  So this is assuming that there's a concave mirror that would reflect the lights out.  This wouldn't work in a transmissive case because you need to have the screen basically  magnifying the lenses.  So here's a video of our system.  We have a set of video projectors reflecting off an anisotropic screen.  It has a wide 110 degree field of view.  The content that you're seeing here on the screen is actually 3D geometry of a face  scan in our light stage device or being rendered in real time.  So as I mentioned there's 72 video projectors.  We're using the Texas Instruments Pico projectors which are LCD projectors.  We chose them basically as far as I know there were the smallest projectors that were commercially  available so we can stack them really close together with only 1.6 degrees between projectors.  This gives us a very high angular resolution and also the very wide field of view.  Now the screen material has two components.  There's a lenticular lens but this isn't a lenticular lens as people usually use it.  Usually lenticular lenses are aligned so it will cylinders run vertically but in this  case we're putting them horizontal so it just takes light coming in and scatters it into  a vertical plane.  The second component is a holographic diffuser which is not being used as a hologram but  it's just a light shaping diffuser so that it provides a slight angular blur to fill  in all those gaps between the projectors.  Without any diffusion you actually see a set of stripes and each of these stripes corresponds  to one lens of a different projector.  With a little bit of horizontal diffusion then you can become a seamless image and ideally  the amount of diffusion should be approximately the angular spacing between projectors.  So there's a video just showing with and without the holographic diffuser.  The other interesting effect is that adding the holographic diffuser actually slightly  increases the perceived spatial resolution of the display because you're actually blurring  in pixels that would be seen by the person on your right or on the left into your view.  This is something that will be interesting to explore further in the future.  We also have done both a front and rear projection system.  In this case it's a transmissive screen using two degrees of horizontal diffusion plus the  lenticular screen.  You can also model it as a front projection system.  In this case we painted the back of the lenticular screen black and just using the front surface  reflection off the plastic.  We only need to have one sheet of diffusion here because the light is actually transmitting  through the holographic diffuser twice.  Now one of the fun things here is that we were generating all our images on a single  computer.  We used four ATI affinity cards each with six video outputs for a total of 24.  And then here's what the desktop looks like, we're using the display fusion commercial  software to basically handle this massive desktop.  Now to go from 24 video outputs to 72 we used the Matrox triple head to go video splitters  that take one video input and split it into three outputs so we can get 72 displays all  hooked up to a single computer.  Next step is to calibrate the system.  We model the calibration as a simple 2D rectification that lines all the projector images up to the  display and we simply project a AR marker which we can automatically detect in a fixed  camera.  Now how do we actually render these images?  And as I mentioned earlier we have to render these multiple center of projection images  and these are because each eye sees images or pixels from multiple projectors and similarly  each projector now has to generate different rays that are rendered for multiple viewers.  So better way to look at this is what do these images actually have to look like?  And this is what we render.  And the first thing to notice is that this is a left sort of center and right projector  is that you sort of have this unwrapped view of the face.  So part of this image is rendered from the side of the head and another part of the image  is sort of rendered more from the front because these pixels are going to be seen by different  viewers.  The next thing to remember is that different viewers could be different heights.  So if we have a viewer on the left is now sort of at a straight on view they're going  to see the front of the cube whereas a viewer from the in the middle might be higher up  so they're going to see more of the top of the view and we want to be able to seamlessly  transition between these two perspectives.  So first I'm going to review how would you generate these multiple center of projection  images for a single view of height?  And one way you could do it is you would render a very large number of horizontal and vertical  just traditional perspective views and then you could re-sample those images to create  these multiple center of projection images.  That's fairly inefficient because you're rendering a much larger number of views than you actually  need sort of extra pixels and secondly you're adding an extra re-sampling step.  So what we do is we take 3D geometry basically take our 3D vertices and we come up with a  custom vertex projection algorithm implemented as a vertex shader that maps these vertices  to the screen based on which viewer will see this particular vertex.  So we start out with a particular vertex and then we trace array from a given projector  out to find the nearest viewer.  And the set of viewers for a constant height could be viewed as just a single circle of  viewers all at the same height.  So we find a viewer that will see this particular ray then we go up to the whatever height they  are at and we trace back to the projector through the screen and this gives us the point  where this particular vertex should actually be displayed.  Now if we have multiple viewers at different heights we can take the nearest two viewers  to a particular ray and represent them as two different circles at different heights  and distances.  We take the same ray and intersect it with both circles so we get two intersection points.  Now neither of the tracked viewers would necessarily lie on this ray so we have to interpolate  between the two nearest viewers.  And we do this by looking at the angle between the intersection point and the original, the  tracked viewer positions and then the computer weighted average based on these two angles.  And then once we have that average point we can then re-project back onto the screen as  before.  Now in practice we use a, an average is computed using a Gaussian interpolation so as you move  further away from a particular viewer there's a sort of a Gaussian falloff.  We try and make the width of this Gaussian equivalent to the width of a human's shoulders  so as long as someone's not actually looking over your shoulder there won't be any cross  talk between views.  Secondly we also incorporate a default height and distance so as you move further away from  a particular tracked viewer position or if someone's untracked then they will still see  a reasonable perspective.  The third thing is that you can also modulate these weights by the confidence of your tracking  system so if you are very confident in a particular viewer then you can use the tracked position  and as someone maybe arrives or leaves and their confidence is lower then it will just  transition back to the default height and distance.  So this shows what the interpolated height would be.  We do, for our system we track using a Microsoft connect which can track about two to six viewers.  This provides both height and depth and here at the bottom is the interpolated height so  these two regions here correspond to the interpolated height of these two viewers which smoothly  moves as they move around.  So this is what the interpolated vertical parallax looks like.  On the left this is our proposed algorithm which as you move back and forth smoothly  interpolates between different heights.  Here on the right this is a technique where you assume that for each projector frame all  the heights are constant and this results in image tearing basically where you're seeing  images that were intended for the person on your left or the person on your right and  they might be at different heights.  There's two different algorithms here.  One shows a abrupt fall off or transition from one height to the other and the other  shows the smooth Gaussian.  Most of the results I'm going to show later on are all using the smooth Gaussian.  Using per vertex view blending also reduces crosstalk.  So in this case we have a untracked viewer here that's staying stationary and we have  this actively moving tracked viewer and as he moves close to this untracked viewer ideally  this view remains fixed.  Now if you use per vertex constant height across each projector frame there's going  to be significant crosstalk whenever this tracked viewer moves close to this untracked  viewer.  So another thing we explore in the paper is different mirror shapes.  So I mentioned earlier that using a flat screen you use a holographic diffuser to fill in  the gaps.  Now if you actually curve the shape of the screen using a convex mirror you can actually  increase the spacing between the stripes.  And what's happening here is that you're actually magnifying the position of the projector  so that they are closer to the screen and have smaller gaps behind the screen.  And this only really works for a front projection case.  Now one issue is that this magnification is only really true for the center of the screen  and as you move towards the edge of the screen different rays will no longer converge on  a single point.  So we came up with an algorithm that would basically improve this.  You can start off with just averaging those intersection points and use that to project  onto the screen as I described earlier and get a screen point.  But then you can then refine the projector position by using the local neighborhood around  this projected position and find a new intersected virtual projector position.  And then we simulated a whole bunch of different mirror shapes even beyond the ones that we  could physically build and there's more details about that in the paper.  But these are the virtual positions and these are the images that you'd have to send to  all these unbuilt displays.  Here's the vertical parallax using a curved screen and you see there's still a consistent  perspective.  Our system runs in real time so we can have dynamic scenes.  This is a dynamic scene with structured light geometry.  And finally because we were all inspired by Star Wars and trying to recreate Princess  Lair we stuck Princess Lair on the display.  So this is LEGO Princess Lair until I can actually get a scan of Carrie Fisher.  This is the best I can do.  So future work, as projectors become smaller and cheaper we should be able to build a brighter  much larger display.  Perhaps one that actually goes four or three hundred and sixty degrees around.  And now we're treating the tracking as pretty much a black box but as there are future improvements  in tracking technology we believe that this should be easily incorporated into our system.  And finally I mean one of the biggest hassles is just Windows is really not designed to  spot this number of projectors and it's not auto stereoscopic.  So if there's better ways of incorporating 3D functionality into the OS that would make  the system a lot more practical.  So in conclusion we've presented a hybrid parallax technique, it provides instantaneous  horizontal parallax with tracked vertical parallax and I think this is a good compromise given  how people tend not to move as much vertically but you can still get accurate perspective  from all positions.  It handles multiple users and it's robust to tracking errors.  So that I will take any questions.
SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__0_887.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__888_1139.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__1140_1763.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__1764_1919.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__1920_10811.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__10812_11051.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__11052_11183.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__11184_13955.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__13956_14207.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__14208_23003.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__23004_23231.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__23232_25163.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__25164_26939.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__26940_28247.png SD_A_2014__Interpolating_vertical_parallax_for_an_autostereoscopic_3D_projector_array__9011-5__Wyed__28248_30278.png