• Technical Conference:  15 – 19 September 2019
  • Science & Industry Showcase:  17 – 18 September 2019

On the Creation of Realistic Virtual Experiences

by Nala Rogers


Imagine mapping the world with an array of pinhole cameras. That's essentially how our eyes make sense of things, sampling a new set of angles each time we move our heads, said Mark Bolas, a professor of interactive media at the University of Southern California, who is currently on leave to work on the Hololens team at Microsoft. In the Wednesday morning visionary lecture "Bending Light to Bend Reality," Bolas explained how a computer can use the same approach to build 3D models—then tweak those models to manipulate a person's reality.


To create a realistic virtual experience, one needs both a detailed model of space, and an understanding of how the human brain works, said Bolas. For example, many developers focus only on the center of the field of view, believing that the periphery doesn't matter. But while the center may be the focus of conscious attention, the edges are crucial for making someone feel immersed, said Bolas.

"[The periphery] is wired to a different place in our brains," he said. "It's wired to the one that makes us scared; it's wired to the one that makes us jump. Not the one that makes us think."


Understanding human sensory systems also allows researchers to warp models and create illusions. For example, by showing someone a scene that doesn't quite match their body movements, you can trick them into thinking they are hiking a long meandering path, when in fact their path is a tight figure-eight. Or you can also show someone a row of virtual boxes, and make them grab first one, then another—while in reality, their hand closes over the same box each time.


We already live in a mixed reality world, constantly divided between physical reality and that of our smartphones, noted Bolas. And it's no wonder we love smartphones. Human brains are constantly building models of the world, and those models are as much about what we can do as they are about physical space.


"The mobile phone is just one big, giant, yummy bundle of things to do," said Bolas, holding his smartphone up between himself and his audience. But, he added, peering at the small screen, "this is a heck of a way to view the world."


Someday, the virtual reality techniques Bolas described could lead to more natural, intuitive way of interfacing with technology.


Posted: 20 Sep 2018 by Nala Rogers

Sponsored by: