While voice assistants like Amazon Alexa and Google Assistant continue to become more commonplace and find their way into a variety of different devices, there is a growing movement to add artificial intelligence to “digital humans,” or virtual characters that go beyond doing our bidding, and become companions or even friends. Korean technology giant SK Telecom is the latest to experiment with the concept, and demonstrated its HoloBox at Mobile World Congress.
We spoke to SK Telecom and Reality Reflection, the development team behind the digital character inside the HoloBox, to better understand this fascinating, sci-fi-infused future.
HoloBox
The HoloBox looks like a compact, cylindrical fish tank, but instead of water and a few fish, you’ll find Wendy living inside. Wendy is just one of the characters, or digital humans, created by Reality Reflection for the project. She is Alexa, Siri, or Google Assistant given form, and will do everything you’d expect from a modern virtual assistant. She appears on a flat, 8.6-inch screen mounted at the rear of the glass cylinder, and is made “real” by a projector mounted just in front of it.
You’ll find all the usual technology inside the HoloBox, including a front-facing camera, multi-directional microphones, and touch controls on the top. Primarily though, Wendy reacts to your voice — using SK Telecom’s Nugu AI platform — and performs simple tasks, like turning on lights or reminding you of an appointment. It’s really only the beginning of her capabilities.
The HoloBox also fits neatly into SK Telecom’s 5G plans. Generating the holographic character takes a massive amount of visual data — one square centimeter of hologram is about 1GB of data, according to SK Telecom. That cannot be streamed over 4G LTE, but it is possible with 5G’s speed and capacity. The intention is for HoloBox to show how emerging forms of entertainment, from augmented and virtual reality to holograms, can benefit from 5G.
Who is Wendy?
SK Telecom may be using the HoloBox and Wendy to promote 5G and its technology, but like any good human — digital or otherwise — there are other levels to Wendy and her glass home. Reality Reflection built the Wendy character, and see her evolving to become a companion to those living alone. She’s capable of listening, understanding, reacting, and even emoting through her facial expressions. To do this, she has to look just right.
Wendy the digital human is based on Wendy, or Son Seung-wan, a singer in the K-pop group Red Velvet. Digital Wendy is more than just her namesake. Reality Reflection used its massive camera rig consisting of 160 DSLRs to scan Wendy’s face and body to come up with the final digital human we see inside the HoloBox. To render her into a single photorealistic model was a massive undertaking. A scan takes three to four hours to complete, and at least two weeks to create a basic render. To get the desired level of detail in the model can take two or three months.
Beyond the HoloBox
The model inside HoloBox is nowhere near as detailed as it could be. The technology to create such a highly realistic model, and project it in 3D, is beyond the scope of the project. Reality Reflection’s Vice President of Business Development, Don Lim, said he was disappointed that the final version of Wendy didn’t show off the true extent of the team’s hard work, while SK Telecom’s Immersive Labs Manager Gukchan Lim explained that a desire to keep the unit relatively small made the use of 3D projectors impossible.
https://twitter.com/AndyBoxall/status/968545896685334528
The trade-off comes when Wendy, or Aria, the other SK Telecom character shown to us, leaps out of the HoloBox and onto your smartphone. Using augmented reality technology, Wendy and Aria can live on your phone to help your through the day. The demo we saw looked great, and the higher-resolution phone screen helped more of the character’s detail show through.
Reality Reflection created another digital human, again based on Wendy, called Zena. This character lives inside virtual reality and can be seen on the Oculus Rift, HTC Vive, and Samsung Gear VR. Zena dances, emotes, and maintains eye contact in VR. The level of realism in her movements is impressive, and evidence of what the firm could do with next-generation holographic hardware.
Living with Digital Humans
It’s not the first time we’ve seen a project like this. Japan’s Gatebox was one of the first to attract mainstream attention, and messaging app Line has expressed considerable interest in pursuing the concept of using digital characters and artificial intelligence to combat loneliness, or to create virtual companions.
Artificial intelligence is making it possible for computers to think and act more like people, and by increasing the visual realism of the character itself, particularly in movements and facial expressions, it’s hoped we’ll be more willing to accept these characters into our lives. Wendy can make more than 100 different expressions and movements at the moment.
Holographic digital humans can be seen as a logical step beyond verbally interacting with a faceless smart speaker, but whether it’s a direction everyone will be comfortable with remains to be seen. The response to Gatebox was split, with many finding it disturbing, and it doesn’t seem like SK Telecom is convinced HoloBox and Wendy will quickly find more favor.
Although we were told the intention is to put HoloBox on sale in the future, it may not be for another year or so before it happens, and that’s a shame. Living with a device like the HoloBox doesn’t have to become a depressing episode of Black Mirror. For us, the concept of having a digital companion is a sci-fi fantasy we can almost truly experience, and HoloBox brings it one step closer to reality.
