I was fortunate this May to be able to head over to Silicon Valley for the world’s largest AR and VR conference with support from Internet NZ. Firstly, thank you Internet NZ. You rock. They supported my very first mixed reality hackathon two years back and again this year helping me get over to SF for this expo. I wouldn’t have been able to attend without their support, and the learnings I’ve gained have been massive so now it’s time to share them.

The event attracted nearly 6,000 attendees, 400 journalists and 250 exhibitors and featured more than 400 speakers in a 100,000-square-feet expo. It was big. But not overwhelming. With so many exhibitors it was my key goal to try and get around everyone over the three days, and I believe I successfully achieved this. It’s tricky to gain access to the latest hardware in NZ so being able to try all the “toys” was a buzz and I did feel like a kid in a candy shop. Here’s a review of highlights and my take on the experience.

 

Mixed Reality Gaming and Location Based Entertainment is about to boom!

 

If you’ve been lucky enough to experience The Void or any other mixed reality gaming experience you’ll know just how awesome this is. Coming from an entertainment background, I love the opportunity that VR and AR present us in the future of location-based experiences. One of the first experiences I was eager to try was The Globacore Heist by Globacore. With our backpack computer and headset on, myself and two others had to work together to commit the heist of the century. We only managed to get through the first two rooms where we had to dodge lasers then work together to avoid sharks.

Honestly, you can’t get much more fun than this when it comes to interactive gaming. I’m excited to be working on an experience of this kind our right now thanks to funding from The New Zealand Film Commission as part of their interactive development fund.

 

Enterprise solutions and wearables are leading the growth of the sector.

 

It was pretty clear looking around the demo floor that enterprise solutions are on the rise and that’s where the willing customers are currently sitting. With the ability to improve processes, create safer environments and reduce costs, AR glasses or assisted reality ones where a clear majority of hardware on display.

Selection of headsets utilising Qualcomm Snapdragon chips

We tried them all. I think there were 50 different kinds, all with varying quality and features.

On the main stage, Meta’s Joe Mikhail kicked things off after lunch. “AR is the first technology that doesn’t try to describe experiences; it actually delivers them.”

I was excited to try out the Meta having followed their progress from afar but was a bit underwhelmed by the experience especially the fact it had no voice control and the hand tracking wasn’t great (it wouldn’t pick up your hand if you had a ring on?!)

I was, however, impressed with the ODG R9 and Daqri. Linked data with handsfree controls makes Daqi a winner when it comes to maintenance and inspections and is the market they are after.

Mixt Co-Founder Anton trying the ODJ R-9

We’ve been using and building for the Microsoft Hololens for a couple of years, and despite its limited field of view, it was clear at this conference that Microsoft is still leading the way when it comes to redefining the modern workplace. Now that MagicLeap is out it will be interesting to see what happens to the market

“To the developer community, drop your phones and tablets and start developing for smart glasses. Funders invest in app developers so they can go beyond phones.” Said Lance Anderson of Vuzix.

 

 

We’ve mentioned it before in a previous post. Wearables are coming at us fast.  

 

Stuff that’s good for the world

 

Having worked on a couple of projects aimed at using VR for wellbeing, I wanted to see what was happening globally in this space.

In the Playground area of the expo, I met the CEO of StoryUp XR, Sarah Hill and tried Heal-ium: Positivity Tree which is kind of like virtual telekinesis where you use your mind’s gamma asymmetry to grow a VR tree or an AR butterfly from a chrysalis to a beautiful monarch. It was an interesting concept aimed at workplace wellbeing an area which I see a lot of potential for. I’m not sure this will be the winning product, but I hugely admire Sarah and her team for pushing the use of AR/VR in mental health.

I also met a Doctor from the UK who has been using VR rooms to treat Autistic patients to deal with their fears which was having some excellent results. Unfortunately, the cost of setting up the room wasn’t a scalable solution so you’d need a hospital behind you to support this. It would be fantastic to see how this solution could be transferred into a headset and measured to see if the results are similar.

On the main stage, among the many verticals that HTC addressed, they showed an example of how VR is being used by medical professionals and their patients to review tumours. It was a touching moment between a young girl and her doctor that showed the power of virtual reality outside of gaming and entertainment.

Bring on more technology for good I say.

 

Volumetric capture options expand with Microsoft.

 

Microsoft volumetric capture studio at Reactor SF

Along with meeting the Microsoft team at the event, we were lucky enough to get a tour of their 3D capture studio in San Francisco a few days later. I didn’t realise they’d spent eight years working on this technology, but I can see this hard work has paid off as the results are fantastic.

We are having more and more requests for human interaction within our experiences so having the ability to capture quality 3D footage is vital. Of course, we are lucky to have the 8i studios on our doorstep, but it sounds like Microsoft will be opening a studio in Australasia shortly giving us options.

 

What’s coming up? It’s all about 5G.

 

With development well underway, 5G networks are expected to launch across the world by 2020. 5G networks are the next generation of mobile internet connectivity, offering faster speeds and more reliable connections on smartphones and other devices than ever before. Why is this such a big deal for VR and AR?

VR and AR experiences require a lot of data to be processed. For local applications, this is generally not an issue, but if you want to feed in data remotely, it can place a strain on a network. Not so great when the user is on the move using AR or away from a fixed internet connection.

Latency issues will also be resolved. It is generally accepted that the highest acceptable time it takes to update the screen from a users movement is roughly 20ms (motion-to-photon latency). Typical displays on VR/AR devices have a refresh rate of between 60Hz and 90Hz or 16.66ms to 11.11ms. This is the time to update pixels on a screen.

The fastest networks in the US provide 40-200ms latency round trip. On 5G, however, this is a theoretical single ms latency. It’s estimated that smart glasses will increase to 48 Million by 2021, that’s a pretty good reason alone for us to be pushing in NZ for 5G if we want to stay up to speed with the latest in workplace and entertainment technology.

 

Cloud AR  and Real-time computer vision. A hot topic.

 

For the last while, GPS has been a stand-in for Cloud AR. On any enabled smartphone, GPS does a pretty good job of figuring out what venue is where and access to more information about it. AR Cloud offers visual recognition of that venue and will be a much more precise alternative.

It get’s you thinking about the opportunities but also the risks. Imagine looking at a person and having instant real-time access to all kinds of information about them? It raises a lot of privacy questions and highlights the need for amendments to the privacy framework to provide adequate protection for people.

On the other hand, it opens up a whole lot of opportunities in augmented reality. No doubt those working in marketing are already rubbing their hands together at the prospect it offers.

AWE organiser Ori Inbar is very focused on this area and the need for the XR community to take ownership of the development of the Cloud.

AWE 2018

“There is a growing concern in the industry that [the AR Cloud] could be controlled by corporations and also be fragmented across too many instances,” says Inbar. “For this reason we are kicking off a historical endeavor to ask the XR industry to come together to collaborate on an Open AR Cloud that will act as a standardized, interoperable, and accessible foundation for all developers and creators to build upon.” Watch this space!

Product highlights

For those working in the industry here’s a wrap up from e-week on the new product highlights from the event:

  • Kopin unveiled the world’s first gesture and voice-controlled wearable smart screen, weighing 1.5 ounces and surpassing any other market product in modularity, battery and cost.
  • RealWear revealed the first commercially available ‘ruggedized’ AR head-mounted wearable certified for ATEX Zone 1 and C1/D1 Potentially Explosive Environments.
  • eSight, known for its high-resolution and high-camera contrast technologies, introduced a breakthrough augmented reality headset platform designed to provide enhanced vision in precision environments.
  • Wikitude revealed SDK 8, the 8th version of its award-winning augmented reality SDK, now featuring Scene Recognition, Extended Object Tracking and Instant Targets.
  • Vyking introduced what it bills as the ‘first mobile AR marketing platform,” a lightweight SDK that delivers 3D creative deployment and analytics.  
  • Vuzix revealed their partnership with Qualcomm on the new Snapdragon XR1 platform to develop next generation smart glasses, as well as a new partnership with Plessey Semiconductor.
  • Kaayatech showcased its groundbreaking HoloSuit motion-capture suit, allowing users to fully immerse themselves.  
  • Ultrahaptics launched STRATOS Explore, a new SDK for creating 3D haptic sensations in mid-air.
  • NuHeara demonstrated its new augmented hearing solutions, which amplify digital audio while subduing inessential ambient noises, all while keeping the wearer aware of their surroundings.  
  • Zappar announced ZapBox 2.0, a Mixed Reality kit allowing developers and businesses to reimagine how they design, build and create room-scale interactive 3D experiences.  
  • ScopeAR debuted a new AR platform to deliver both real-time remote assistance and AR smart instructions.
  • vSports launched and demonstrated for the first time vGolf, the first Mixed Reality golf simulation and training system.
  • Resonai, in collaboration with The Tech Museum of Innovation, presented The Animaker, an immersive exhibit that invites visitors to collaborate with AI-powered robots.
  • Varjo revealed the world’s first cinematic-quality (40X resolution) Mixed Reality technology to seamlessly merge real and virtual worlds in cinematic fidelity.
  • Mantis Vision demonstrated its 3D sensing solutions based on structured light and a smart decoding algorithm for front 3D facial cameras.

If you are keen to explore the possibilities of virtual and augmented reality hit us up!