Four Kitchens, a distributed digital agency, has started exploring virtual and augmented reality on the web. I talked to its CEO and co-founder, Todd Nienkerk, about the possibilities and his advice to agencies interested in working with VR and AR.
I first met Todd back in 2009 at DrupalCamp Stockholm where he was one of our guest speakers. Todd and I have stayed in touch over the years, and it’s been exciting to see his and co-founder Aaron Stanush’s agency, Four Kitchens, grow, innovate and experiment.
“We want to be there when the market arrives,” – Todd Nienkerk
Four Kitchen’s mission is to help customers “manage and publish great content” to “inform, educate and engage” their audience. VR and AR content is still in a nascent phase, but that doesn’t discourage the agency. Their goal is to establish themselves as one of the experts in this area and to be ahead of the mainstream trend: “We want to be there when the market arrives,” says Todd.
What is VR or Virtual Reality?
VR stands for virtual reality and is a name for a range of technologies that create fully immersive experiences that feel real. Wikipedia defines it as: “Virtual reality (VR) is a computer-generated scenario that simulates experience through senses and perception.”
Examples of VR include exploring alien planets or visiting digitally recreated places such as Roman ruins. Most VR technology uses head-mounted displays that cover the user’s eyes and show an alternate view. The headset tracks the user’s motions and updates the view. Using technologies like Google Cardboard, you can achieve VR with a simple headset and a smartphone.
What is AR or Augmented Reality?
Augmented reality is the sister of VR and uses similar technology as VR but applies it to add layers to reality. As Todd puts it: “You’re seeing the world, but things are being added to it.”
According to Wikipedia: “Augmented reality systems may also be considered a form of VR that layers virtual information over a live camera feed into a headset or through a smartphone or tablet device giving the user the ability to view three-dimensional images.”
The most famous application of AR might be the game Pokémon Go which had millions of people out in the world “hunting” for “Pokémons” to catch. Pokémon Go doesn’t require a headset. Instead, it overlays 3D graphics over the camera view of the user’s phone. But there are more practical applications for the technology such as offering a real-time x-ray view of a patient to a surgeon while operating.
You may also have heard of mixed reality (MR). It is similar to AR but it also integrates itself with the existing world in a way AR does not. While AR is more of an overlay, MR anchors virtual objects in the real world and makes reality and virtuality even harder to tell apart.
The Future is Already Here
“The future is already here — it’s just not very evenly distributed.” – William Gibson
Some of this may sound innovative, but we’ve been living with these technologies for quite some time already. One example Todd gives is the line of scrimmage in American football. For us Europeans who aren’t familiar with football, the line of scrimmage “is an imaginary transverse line (across the width of the football field) beyond which a team cannot cross until the next play has begun” (Wikipedia). TV channel broadcasts indicate it by showing virtual blue line.
Todd says that one of the most exciting improvements of smartphones is the addition of an extra camera, giving phones cameras stereo vision. This enables improved depth perception, just like in us humans, which is interesting as it allows the phone to do some calculations faster such as detecting edges and planes. These are critical for AR applications such as the IKEA app that projects virtual furniture in your home using AR.
VR and AR Will Be Commonplace
Todd predicts that VR headsets will not be a big hit soon: “It’s kind of like some people like to play video games on their iPhone or Android while some other people like to play video games on an XBOX. It’s the same thing with VR.”
As a result, Four Kitchens focuses on bringing AR and VR to the web making it more readily available on existing platforms and devices that everybody already owns to reach as many people as possible.
“The real breakthrough will happen when you don’t have to buy anything extra at all.” – Todd Nienkerk
He considers the tipping point, and he thinks we aren’t near it yet. Todd compares the availability of today’s devices compared to the forerunner technology of the ‘90s. But the real breakthrough, he says, will happen when “you don’t have to buy anything extra at all.”
How Content Intersects with VR and AR
Todd tells me that what Four Kitchens is particularly interested in is how content intersects with VR and AR. Their job is to help make content “useful, valuable and compelling.” The good news is that we don’t have to wait until viewers and headsets become commonplace. You can get far without using a head-mounted display or viewer. Holding your phone and panning to look around is usually enough to be useful. Todd tells me that what convinced him of the usefulness of VR was watching a CNN 360 degree video of French aircraft carrier Charles de Gaulle carrying out operations against ISIS in the Mediterranean.
What won him over was that he didn’t need a viewer to enjoy the content. “This content is accessible to people even in two-dimensional formats,” he says. His next thought was: “We’re eventually going to have a client that asks for stuff like this.” It had to be more than just a video upload to Youtube and instead more of an interactive experience. They started looking into VR tours with interactive hotspots, working like links, with popup content.
One of their first demos of this technology was a VR tour titled “Meet the Web Chefs” they presented in 2017. In this tour, two of their team members introduced the viewer to their workspaces. It was a way to show the potential of the technology as well as introduce the fact that they’d become a virtual business and how it affected the way they work.
These scenes work with on device, be it a smartphone, tablet or laptop. The viewer can interact with icons in the scene. “If you interact with them you’ll get a video that plays, or a text popup or a closeup photo or some way to interact with that stuff and those are really compelling for a lot of use cases,” Todd says. He mentions virtual tours at museums and university campuses as potential applications of this technology.
Todd’s Favorite Virtual Reality Use Cases
Todd says there are five main use cases that are particularly well suited for VR and which apply to many different experiences.
1. Empathy Building
360-degree video: “13-year-old Selam used to walk miles each day to bring water to her family, struggling to make time for her schoolwork too. Experience the transformation of her Ethiopian community as a well bearing fresh water is dug for the first time.” (from “The Source” by Charity: Water)
“VR is an empathy building machine, “ – Todd Nienkerk.
Todd says that “VR is excellent at putting a person in a space they don’t have personal experience with but allowing them to instantly feel empathy for someone in that position or in that place.”
One example Todd gives is a non-profit organization working to bring clean water to villages in sub-Saharan Africa. That’s a remote problem for a lot of people in wealthy countries. The idea of not having access to clean water seems remote to many, and few have any clear idea what sub-Saharan Africa is like. “What does a clean water system look like? What does it do? What does it mean?” Todd asks rhetorically.
He says you could send a video crew to make a short video about it, but that won’t make you feel like you’re there. “We have such a separation between the screen and us,” Todd says. However, when you have a headset (or viewer) on or look at a scene using a phone like a keyhole, “you get a sense of scale, you get a sense of understanding: this is where people live like I’m inside one of their homes,” Todd explains. With a video camera, “you’re only seeing what they want you to see.” Todd argues. He says VR seems “less contrived” than traditional types of media.
2. A Sense of Place and Scale
“This is really good for tourism, travel, hospitality and the real estate industry,” Todd says. He gives the example of a VR tour of an apartment before even going there for a physical viewing.
He says that a key element in renting out your own room using Airbnb is having great photos. “They send a photographer out to make your place look really good in the photos,” Todd says. However, to squeeze as much of the room as possible into each frame, they use wider focal lengths (i.e. ”fisheye lenses”) which makes space appear bigger than it really is. “In every photo on Airbnb, everything looks bigger than it really is,” Todd says.
“Well, if you had a 360 camera you wouldn’t need to fit anything else into the shot ‘cause you fit everything into the shot, and there’s no manipulation of focal length,” Todd explains.
Another example Todd gives is taking people to a place they cannot visit. Such as in the case of people with disabilities. “There’s this thing in the United States called Honor Flights. It’s when an airline or non-profit donates an airplane trip to veterans of foreign wars,“ Todd says. These trips help veterans travel to Washington DC for free to visit the war memorials. Many former service people are quite old, so traveling is difficult for them due to age, or illness. In that case, VR provides an alternative way to visit these sites.
3. Education and Training
This category includes campus and workplace tours, or safety tours for working in hazardous conditions such as a nuclear power plant. “It would be a lot better to train them in VR first before you put them on the floor, right?” Todd says.
Todd also mentions lab training, even remotely. You could show someone the location of the eyewash station even though they’re physically elsewhere. Another example is flight simulators, which have been around for a long time, and allows pilots to try things that would be deadly in the real world.
4. Therapeutic Applications
Another use of VR is cognitive behavior therapy for PTSD: “One use case we pitched to a hospital once was the idea of reducing the anxiety of a visit by providing a tour of the hospital from the patient’s home first,” Todd says. One example of such a tour would be a pregnant woman who hasn’t visited the hospital being shown in VR what will happen when it’s time for delivery, what equipment they use and for what.
These are examples of exposure therapy and there are cases of VR being used successfully to treat phobias of different kinds.
5. Gaming and Entertainment
This is perhaps one of the first applications people make when thinking of VR but may not be as common once VR hits the mainstream.
Todd says: “There’s an interesting statistic that I learned at a conference. Someone was talking about the adoption of VR and AR. He said the closest thing we have to compare it to is the adoption of smartphones. When the first iPhone came out, which was the first actual smartphone, people thought ‘this is super cool, this will be really good for gaming, it has an accelerometer, haptic feedback and so on.’ A lot of people thought it would replace the Nintendo Wii.”
“Well, if you look at actual usage statistics today, a mobile phone is used for gaming toughly 8% of the time. The vast majority of people’s time spent on smartphones goes into two categories. Messaging is one; the second is email. Texting and email represent more than 50% percent.”
The Possibilities of Augmented Reality
“When you blend reality with augmented reality, the possibilities just really explode and the market is just trying to figure out what will be useful,” Todd says.
One thing Todd says makes AR powerful is image recognition. One example he gives is going out shopping and checking of a teacup is available online cheaper. AR would allow you to look it up just by looking at the cup or pointing your phone camera at it.
Todd says that a constraint in the AR industry is being able to do image recognition on the device, not requiring a fast Internet connection. It’s a computation-heavy feature, which many manufacturers do in the cloud. The drawback being it seems slow to the user compared to doing it on the device itself.
AR and VR Meet in the Third Dimension
“The thing that AR and VR have in common is three-dimensional modeling. Both of them are using these technologies. How do you work with three-dimensional assets on the web and how do three-dimensional enhance your experience of content on the web?” Todd asks.
As a result, much of Four Kitchens’s investment in AR and VR has been learning about 3D technology and 3D modeling. “What are the tools you use to capture a 3D object in the real world? What are the file formats that are popular? What are the pros and cons of the various file formats? How do you display them in a web browser? All of that stuff!”, Todd says. He adds: “The 3D web is just a really interesting space” and mentions several applications of the technology such as being able to view 3D objects in e-commerce applications using multiple photos or 3D scanning. Todd says that it gets easier and less time-consuming to produce these kinds of 3D models of real objects.
EditVR, the VR Experience Builder
“We have built and will soon release an open source tool free and open search tool that allows people to create Web VR experiences very quickly.” – Todd Nienkerk
To make this even more accessible, Four Kitchens has created an open source project called EditVR, and they invite others to collaborate.
“It allows people to create Web VR experiences very quickly. So all they need is an inexpensive 360 camera or a camera that can take stitched 360 photos. Stitching is like doing a panorama, but you do it in all directions. Then you can upload it to EditVR and start adding hotspots and points of interactions and linking scenes to each other using transitions. These are essentially hyperlinks in VR. It’s like a portal,” Todd explains.
“If you’re a content producer and you’re already doing blog posts, and videocasts why not also do some inexpensive 360 experiences VR experiences?” – Todd Nienkerk
He adds: “We built this tool because we wanted to show that VR content is ready and that you can do it today and we want to help people integrate it with their larger digital strategy. So if you’re a content producer, and you’re already doing blog posts and videocasts why not also do some inexpensive 360 experiences VR experiences?”.
You can check out EditVR and sign up for updates here:
Todd’s Advice to Agencies on Getting Started With VR
I asked Todd about his recommendations to agencies eager to start with VR and AR.
“They will need to invest a fair amount in R&D to train up their team and to pay attention to how quickly changing this all is. That there is not high demand yet but I believe there will be,” Todd says. He continues: “Our strategy is we’re trying to find partners to help us a portfolio. It would be great if we could pay us to do it. That would be great. But we’re also willing to do pro bono work to fill out our portfolio.“
“My advice [to other agencies] is to be patient, stick with it and stay involved. Because this is going to arrive.” – Todd Nienkerk
“It’s a bit of a risk, but that’s the case with any new technology. I would at least recommend that agencies spend just a few hours reviewing the state of the industry ever so often. Because things are evolving so quickly. And also it wouldn’t be a bad idea to create a network of other agencies they can contract to if they have opportunities to do this work with clients but aren’t quite ready.”
Todd says there are lots of agencies that only do VR and AR but because they’re just doing that they’re currently in a better position to take on paid work. It’s their entire portfolio and all the marketing they do. For agencies that are less niched, it’s hard to find this kind of work unless you have a great and trusting relationship with a client, says Todd.
“My advice [to other agencies] is to be patient, stick with it and stay involved. Because this is going to arrive,” Todd concludes.
Todd says he’s more than happy to hear from other agencies who are looking into AR, VR, WebVR and 3D technologies. So if this topic excites you, don’t hesitate to get in touch with him.
Many thanks to Todd for taking his time talking to us. If you want to write or call Todd, check out his profile at Todd at Four Kitchens
Do you have any VR or AR projects planned? I’d love to hear about them.
Please post a comment. I read all the comments.
- epredator Predlets getting their #psvr fix via photopin (license)
- Фото Москвы Moscow-Live.ru IMG_6805 via photopin (license)
- Photo by mentatdgt from Pexels
- CNN: “Stand on an aircraft carrier battling ISIS”
- Four Kitchens: http://vr.fourkitchens.com
- With.in, “The Source”
- Video interview, Jakob Persson and Todd Nienkerk.
- Kozloduy Nuclear Power Plant – Control Room, Wikimedia Commons
- Photo by rawpixel.com from Pexels
- Janitors HTC Vive VR – Mobile World Congress 2016 via photopin (license)
- jimbob195 afternoon tea via photopin (license)