The New Realities: Virtual, Augmented and Mixed

January 4, 2017

Kevin (What Technology Wants) Kelly concludes an epic article in Wired magazine about artificial reality with the pronouncement that, after a long gestation period, “It’s real.”

Mark Zuckerberg, quoted in a Vanity Fair piece on the virtual reality headset, Oculus Rift, said, “This is ready. This is the future.”

And Rony Abovitz, the rather secretive Founder and CEO of Magic Leap, a mixed reality company that has raised $1.4 billion without releasing a single product, has mused, “We are building the Internet of presence and experience.”

What are all these smart people talking about? Well, Wired’s cover proclaims this is nothing less than the “quest to create a new kind of reality”. And whoever wins this grand endeavor may very well determine the future of computing, as well as the demise of the desktop, laptop and, perhaps, the mobile phone, itself. It could become the screen that rules them all.

But before we go there, it is worth taking a step back and looking at the various forms of reality that are being artificially created and the implications that each of the approaches may have. And, as important, what impact these new devices and “realities” could have on how we (and our kids) interact with both the online world and the “real” world.

First, let’s assume that we all know what reality is. OK, that’s a pretty big assumption seeing that philosophers since Plato and his cave have been arguing about what is real, imagined, illusory or, even, part of an alternate reality, as in a parallel universe. But for the purposes of this piece, let’s assume that reality refers to things, places and people as they actually exist. Or as many teens refer to it in abbreviated, text speak: IRL (in real life).

So the new reality that many in the tech space are talking about is broadly known as artificial reality, which covers a continuum of experiences from virtual (VR) to augmented (AR) and even mixed reality (MR). And the shape and complexity of the hardware and software that is needed to achieve the desired effect will be determined by where on the artificial reality continuum the developer is trying to land.

Take virtual reality. This is usually a completely immersive, artificial world that uses sensory cues to fabricate a “real” experience. It requires a headset that takes over the user’s visual field and can be enhanced by head phones to provide a sound track to whatever world they are inhabiting. Examples of VR include Google Cardboard, Samsung’s Gear VR, HTC’s Vive and, of course, Oculus Rift. These and others coming onto the market differ between being completely mobile (Cardboard) to totally tethered (Rift) with an equally wide range of pricing.

Having recently tested both the Rift and Cardboard, I can attest to the remarkable realness of the experience. Your brain (and the rest of your body) is tricked into thinking and feeling like it’s on a roller coaster or flying with a flock of seagulls or falling from the top of a skyscraper. I ducked and dodged and, according to colleagues, gave out whoops and whelps as the coaster I was riding plunged or as the birds I was flying with soared over a virtual mountain.

The experience is so real, VR headset manufacturers caution to only use them while sitting down less you crash or hurl yourself out of the way of a bullet or oncoming train or whatever else the software throws at you. The Rift was so realistic it took me a while to re-adjust to real life. While entertainment and gaming are obvious uses of VR, it could also be used for education, social networking (hence Facebook’s purchase of Oculus) and collaboration. It could just be the killer app of the 2020’s.

Then there is augmented reality, which uses the real world and overlays data and digital images, sound and video on to it. Classic examples include the ill-fated Google Glass or the heads-up displays on car windshields. Facebook’s recent plans for AR glasses could not only integrate data of, say, a football stadium you are about to enter, but also all your friends’ photos and videos ever taken there, a list of who else is in the crowd and ways to message them in real time.

Rather than entering a completely virtual world, augmented reality promises a data rich interaction with the real world and all the real people - friends included - who inhabit our world.

There are numerous AR glasses either on the market or will be soon. Probably the best known of these is Meta. Check out the Ted talk given by Meron Gibetz giving the most tantalizing demo yet of where this technology is heading. And the reviews from top tech watchers and developers are instructive in their emotional reaction to working and sculpting with virtual 3D objects in a real world environment.

And then there is the category known as mixed reality. As the name suggests, it borrows aspects of both virtual and augmented reality technologies and overlays the mixed experience on the real world. The best known or, at least, most hyped example of mixed reality is the Florida-based, Magic Leap. In MR, killer whales can suddenly emerge from beneath the wooden floor of a gymnasium and splash below the surface again. Spreadsheets can be projected on to a real desk top, while the planets circle the sun above your head. Virtual objects can appear or disappear depending on your relation to physical pieces of furniture or walls.

Magic Leap and Microsoft’s HoloLens are in a multi-billion dollar race to create the next computing platform. But whereas the first iteration of cyberspace, roughly the past 25 years, has brought data and images and video to our many and varied screens, MR promises to combine data with presence and even experience into a single pair of glasses or visor. It’s as if the Internet of the head will be linked to the web of the heart to create a new universe of emotional intelligence. It will have a profound impact on how we see and co-create the world with each other and with an infinite variety of virtual and real objects and encounters.

MR will require vast new databases, server farms, bandwidth and power to fuel its huge processing demands. It will transform how we think and remember and invent and learn.

From what I’ve seen so far, there will have to be serious consideration about safety, privacy and security and a recognition that the compelling nature of the experience will be hugely addictive. Naturally, there are already companies devoted to producing artificial reality porn.

As always, we must attend to the potential risks and inevitable harms that these new platforms will create while staying focused on the remarkable rewards that could flow. I recently asked what happens when the Internet of Things becomes artificially intelligent. Well, add MR to the mix and throw in some chatbots for good measure, and you can begin to see the world our children will be inheriting.

Let’s ensure that this new technology is built with safety by design and that we are up to the considerable task of integrating MR into our lives and our childrens’ lives in a way that brings out the better angels of our nature and not the stuff of nightmares.


*This article originally appeared in the Huffington Post.


Written by

Stephen Balkam

For the past 30 years, Stephen Balkam has had a wide range of leadership roles in the nonprofit sector in both the US and UK. He is currently the Founder and CEO of the Family Online Safety Institute (FOSI), an international, nonprofit organization headquartered in Washington, DC. FOSI’s mission is to make the online world safer for kids and their families. FOSI convenes the top thinkers and practitioners in government, industry and the nonprofit sectors to collaborate and innovate and to create a “culture of responsibility” in the online world.

Prior to FOSI, Stephen was the Founder and CEO of the Internet Content Rating Association (ICRA) and led a team which developed the world’s leading content labeling system on the web. While with ICRA, Stephen served on the US Child Online Protection Commission (COPA) in 2000 and was named one of the Top 50 UK Movers and Shakers, Internet Magazine, 2001.

In 1994, Stephen was named the first Executive Director of the Recreational Software Advisory Council (RSAC) which created a unique self-labeling system for computer games and then, in 1996, Stephen launched RSACi – a forerunner to the ICRA website labeling system. For his efforts in online safety, Stephen was given the 1998 Carl Bertelsmann Prize in Gutersloh, Germany, for innovation and responsibility in the Information Society and was invited to the first and subsequent White House Internet Summits during the Clinton Administration.

Stephen’s other positions include the Executive Director of the National Stepfamily Association (UK); General Secretary of the Islington Voluntary Action Council; Executive Director of Camden Community Transport as well as management positions at the Institute of Contemporary Arts (London) and Inter-Action. Stephen’s first job was with Burroughs Machines (now Unisys) and he had a spell working for West Nally Ltd – a sports sponsorship PR company.

Stephen received a BA, magna cum laude, in Psychology from University College, Cardiff, Wales in 1977. A native of Washington, DC, Stephen spent many years in the UK and is now has dual citizenship. He writes regularly for the Huffington Post, appears often on TV and has appeared on nationally syndicated TV and radio programs such as MSNBC, CNN, NPR and the BBC and has been interviewed by leading newspapers such as the Washington Post, New York Times and The Wall Street Journal, radio and in the mainstream press. He has given presentations and spoken in 15 countries on 4 continents.