A lot of people still do not consider Pokemon Go “real” AR. At launch, the AR features were extremely limited and in most cases the camera would simply plaster an animated Pokemon over the top of your real world environment which resulted in the creature seemingly standing inside of objects and walking across things it shouldn’t be able to walk across. In short, it lacked real depth and environmental awareness. Recently Niantic acquired a London-based technology company called Matrix Mill. By using machine learning and neural networks, they’ve more or less solved the issue. Watch the results in the video below: “At Niantic, we frequently talk about how in order to augment reality, you need to be able to understand it,” says the Pokemon Go developer in a blog post. “The Matrix Mill team has come up with novel ideas that push the boundaries of what machines can process, thinking around occlusions, and seeing the world closer to the way human eyes can. As a result of this hard work, AR experiences can feel more natural to the eye, which is a goal we have squarely in our sights.” This sort of stuff is possible today using expensive cameras and advanced equipment, but everyday smartphones with single cameras on them like the Google Pixel 2 or the latest line of iPhones couldn’t achieve this effect. Now, with Matrix Mill’s help, Niantic can deliver much more convincing AR. By interpreting and then re-sending the data back to devices, the depth effect in AR works seamlessly and characters like Pikachu and Eevee can be occluded behind objects in the world or even people walking by in real-time. We don’t really know the timeline for when this sort of functionality will officially roll out to users, but it’s exciting nevertheless, particularly as the company preps for the eventual launch of their next AR game, Harry Potter: Wizards Unite. Let us know what you think of this occlusion demo down in the comments below!<doc-sep>Niantic, the developer behind Pokémon Go and the upcoming Harry Potter AR game, announced they’ll be opening up access to their latest work in AR, dubbed the ‘Niantic Real World Platform’, to third-party developers soon. To boot, the company also introduced a few key AR technologies that will have you salivating over the possibilities of actually chasing down pocket monsters on your commute to that next Pokéstop. Niantic says their Real World Platform blends machine learning and computer vision to tackle the classic challenge of building a useful and realistic AR experience on mobile devices—something that can sense small details, understand surroundings, and model them in an interactive 3D space that a smartphone can digest. One area of research Niantic has been work on is proper occlusion, or making sure digital imagery fits into the physical world correctly, and allowing it to be obscured naturally by objects in the environment. The company published a quick video on their blog, showing off their latest work in the area of AR occlusion. What better test subject than Pikachu? Creating correct occlusion in AR requires that the computer, in this case a smartphone, contextually understands the world around it. Slowing down the video some, it becomes a little more clear however that the company still has a ways to go, as the occlusion masks oftentimes overcompensate, or misjudge the alignment of objects as Pikachu scampers about. While a proof-of-concept, it’s definitely a tantalizing look at the near future of smartphone AR, and a clear departure from what we saw at Pokémon Go’s launch back in Summer 2016. The Niantic Real World Platform is also focusing on cross-platform AR for shared, multiplayer experiences. The biggest obstacle, the company says, is invariably latency. To this effect, the company says they’ve developed “proprietary, low-latency AR networking techniques” to overcome this problem, which allowed them to realize a unified, cross-platform solution with a single code base. To demonstrate, Niantic built a multiplayer smartphone AR shooter, dubbed ‘Neon’, which shows six users playing at once. We can attribute some of this to the company’s recent acquisitions; Niantic recently acquired Escher Reality, a studio touted for its cross-platform, multi-user AR platform, and the computer vision and machine learning company Matrix Mill—two decisive moves forward after the company’s $200 million Series B finance round. “It’s through the coordination of these teams that we’ve been able to establish what the Niantic Real World Platform looks like today, and what it will be in the future,” Niantic says in a statement. As for third-party developers looking to get in on Niantic’s platform, the company says they’ll be picking a “handful” of devs to begin working with their tools later this year. To receive more information, sign up here.<doc-sep>
Pokémon Go creator Niantic announced today that it’s planning on letting third-party developers use its augmented reality platform to develop apps. To showcase how that technology, which Niantic calls its Real World Platform, is improving all the time, the company made a demo video to showcase a technique it calls occlusion. Basically, Niantic used machine learning techniques it picked up from an acquisition of London-based startup Matrix Mill to create a neural network that could, in real time, obscure virtual images behind real-world objects. This will let creatures like Pikachu hide behind flower pots and park benches, and it will even obscure the virtual pocket monster behind dynamic parts of the scene, like moving people. Here’s what you might normally see through your smartphone camera’s viewfinder with a standard AR application today: And here’s what Niantic can do with its new occlusion technique: This is just an experimental proof-of-concept, Niantic CEO John Hanke told reporters at a meeting at its San Francisco headquarters yesterday. However, it’s a strong indication of the type of next-generation AR techniques the company is pioneering for games like Pokémon Go and the upcoming Harry Potter: Wizards Unite. We don’t know when this kind of sophisticated AR will arrive in actual products, but it’s safe to say Niantic wants its Real World Platform to be as robust as possible, so developers decide to build apps on top of it. Hanke and other Niantic executives stressed that they see the platform much like Amazon does its AWS cloud computing platform, so we can expect it to be a big focus for the company’s most ambitious AR advancements going forward.