There are two questions that I’m often asked: ‘What’s in store for the future of AR?’ and ‘What would you like to see in the future of e-books and tablets?’
My answer to both is haptics and tactile feedback.
In August 2011 I had the pleasure of visiting the Magic Vision Lab at the University of South Australia and experiencing their AR haptics demo. Wearing a head-mounted display (HMD) and using a Phantom stylus, I could feel the scales of a virtual fish which appeared before me. I was able to touch virtual objects and receive tactile feedback, as though these were real, physical objects I was interacting with. This completely threw off my sensibilities of the real, having difficulty distinguishing between what was real and what was virtual. This experience signified an important shift for me in the medium of AR: in the past, the only tactile component of AR was that which physically existed in our environment.
Image: Haptics Demo from the Magic Vision Lab, University of South Australia.
I was fascinated by the sense of touch and tactile feedback paired with AR. However, I was left desiring a more direct interaction in this experience, without the HMD or stylus.
Enter Senseg’s touch technology for tablets, which premiered at the Consumer Electronics Show (CES) last week. If we can merge this with AR, I truly think it can be a game changer and help push the medium forward in new important ways that are currently absent. To date, AR has been primarily a vision-based medium; we haven’t really got to augmenting touch and we can’t ignore these other very ‘real’ senses for much longer.
In the short video above, Dave Rice, VP of Senseg, discusses the technology as adding tactile effects to touch screen displays including smart phones, tablet computers, touch pads and gaming devices. He discusses the possibilities for gaming applications (I personally think this would be incredible to apply to storytelling as well) and describes a treasure hunt game in which a treasure chest is hidden and can only be found by feeling around on the screen. Dave says, “There were no visual cues there and that’s pretty exciting because now we can move to the world of feel to complement what you’re seeing, or to work independently from it and really create a new world to explore.”
For me this perfectly describes the future of AR and its potential. I think about this last quote and how it applies to my recent AR Popup Book “Who’s Afraid of Bugs?”, the first AR book designed for iPad 2. For me the next step in this book is to be able to touch and feel the texture of the virtual spider that magically appears. Imagine petting the spider and feeling each tiny hair.
(Also with today’s announcement from Apple on iBooks textbooks for iPad, “a new kind of textbook that’s dynamic, current, engrossing, and truly interactive”, imagine how haptics and tactile feedback could change the future of education in e-books, as well as AR. Talk about ‘bringing the curriculum alive’.)
“A Brief Rant on the Future of Interaction Design”, is a very relevant and excellent article in which Bret Victor asks us to aim for a “dynamic medium that we can see, feel, and manipulate”. Bret’s article immediately resonated with me when I read it in November and I shared it with the AR community via Twitter as something important we needed to be aware of and really work towards.
Image: Bret Victor
Bret emphasized the use of our hands to feel and manipulate objects. He writes, “The sense of touch is essential to everything that humans have called ‘work’ for millions of years.”
“Now, take out your favorite Magical And Revolutionary Technology Device. Use it for a bit. What did you feel? Did it feel glassy? Did it have no connection whatsoever with the task you were performing?” Bret calls this technology, “Pictures Under Glass”, noting that it sacrifices “all the tactile richness of working with our hands”.
Bret links to research that’s been around for decades in haptics, tangible-user interface (TUI) and even Touchable Holography. He comments on how this research has always been marginalized, but that “maybe you can help”.
AND WE CAN. As Bret so wonderfully states, the most important thing about the Future is that it is a choice. As an AR industry and community, it is our choice as to how this medium evolves. “People choose which visions to pursue, people choose which research gets funded, people choose how they will spend their careers.”
Let’s do this. Kindly get in touch if you’re keen to collaborate!
Let’s also continue the conversation on Twitter; find me, I’m @ARstories.
*Hat tip to Stephen Ancliffe for sharing the Senseg video with me.
I’d like to share with you the slides from my recent talk “Augmented Reality: Art, Magic and the Machine” at The Museum, Waterloo, Canada as part of the Rethinking Art and Machine Speaker Series.
I’ve talked about wonderment and imagination in the past in regards to advancing AR as a new medium, but this talk was about the importance of curiosity, beginning by looking at AR through the lens of Alice in Wonderland. AR presents us with very curious awe-inspiring possibilities; we must engage our curiosity to push the medium forward in novel ways, continually questioning, creating and collaborating. Here is a link to a wonderful video I included in my talk by Skillshare on feeding and nurturing our curiosity.
In my presentation I also spoke to the importance of engaging the other senses in AR, not just vision alone. Magic also played a central theme throughout my talk from Georges Melies and early cinema to Arthur C. Clarke and the magic of advanced technologies. AR is an inherently magical and enchanting medium. I suggest that we allow stories and content to evolve from and be inspired by the magic of the technology, not lost in it. I also discuss the role of the artist as central to advancing AR and evolving new forms.
Have a look and then join me in conversation on Twitter.
I’m very pleased to share video of my recent TEDxDubai talk! Naturally, I discussed Augmented Reality 
but I also tied in my Tilt Shift photography to this year’s TEDx theme of “The Beauty of Small Things”.
Fighting a fever and a flu bug, I mustered all my might and was so happy to make it on that stage! Let’s say, it really was an out of body experience!
My talk featured my children’s AR pop-up book “Who’s Afraid of Bugs”. The book was created using AR software by Junaio. (You can also read more here in a recent interview I did.)
I spoke to the importance and power of imagination and make-believe, and how they pertain to AR at this critical junction in the medium’s evolution. When we make-believe and when we imagine, we are in two places simultaneously; make-believe is about projecting or layering our imagination on top of a current situation or circumstance. In many ways, this is what AR is too: layering imagined worlds on top of our existing reality.
As a designer/artist/PhD researcher I am both a practitioner and a researcher, a maker and a believer. As a practitioner, I do, create, design; as a researcher I dream, aspire, hope. I am a make-believer working with a technology that is about make-believe, about imagining possibilities atop actualities. Now, more than ever, we need more creative adventurers and make-believers to help AR continue to evolve and become a wondrous new medium, unlike anything we’ve ever seen before.
Make-believe is an extension of wonderment for me. In my previous TEDx talk, in 2010 at YorkU (embedded below) on wonderment and the creative process, I spoke to wonder guiding my work in AR, and the importance of developing wonderment in technology alongside compelling content/storytelling as the medium advances.
I am grateful to the superhuman TEDxDubai curators Giorgio Ungania and Natascia Radice, all of the wonderful volunteers, the mega talented fellow speakers and each of the incredibly engaging delegates for a truly stellar event. Thank you, thank you, thank you.
I was mega flattered to be invited to write a guest column for The Creators Project (a partnership with Intel and Vice). The topic? AR, but of course! In “The Magical Magic of Augmented Reality”, I draw parallels between AR, magic and early cinema, discussing the work of one of my creative heros, Georges Méliès.
Prior to becoming a filmmaker, and earning the title of ‘the father of special effects’, Méliès was first a stage magician; he used the newfound medium of cinema to extend magic into novel, seemingly impossible visualities. AR, too, is about creating impossible visualities. In some ways, we can consider AR to be a real-time stop-substitution that layers content dynamically atop the physical environment and creates virtual actualities with objects—shifting forms, appearing and disappearing—as Méliès first did in cinema.
Read the full article here and share your thoughts with me on Twitter. Send me a tweet and let’s discuss: @ARstories
A great big thank you to Bruce Sterling for blogging my article on Wired.com!
*Update: Dec 8/11: I name Melies the ‘Patron Saint of AR’ in a follow-up article on The Creators Project celebrating Melies’ 150th birthday.
Introducing a few sample pages from my new Augmented Reality (AR) Pop-up Book, “Who’s Afraid of Bugs?” with video and images below.
I’d like to present the first AR Pop-up Book for mobile devices using image recognition (a.k.a. regular images to trigger augmented content, as opposed to the black and white square glyphs that are common in AR). Integrating image recognition in the design, the book can hence be enjoyed alone as a regular pop-up book, or supplemented with augmented digital content when viewed through a mobile device equipped with a camera, such as an iPad 2 or iPhone 4.
Playful rhyming text takes you through the storybook where various ‘creepy crawlies’ (spider, ant, and butterfly) are awaiting to be discovered, appearing virtually as 3D models you can interact with. A tarantula attacks when you touch it, an ant hyperlinks to educational content with images and diagrams, and a butterfly appears flapping its wings atop a flower in a meadow.
(Above: Screen grabs from AR Pop-up Book working on iPad 2.)
The story for the AR Pop-up Book was inspired by AR psychotherapy studies for the treatment of phobias such as arachnophobia. AR provides a safe, controlled environment to conduct exposure therapy within a patient’s physical surroundings, creating a more believable scenario with heightened ‘presence’ and greater immediacy than Virtual Reality (VR).
[Article: Juan, Carmen M. et al. “Using Augmented Reality to Treat Phobias.” IEEE Computer Graphics and Applications 25.6 (2005): 31-37.]
In some ways, the AR Pop-up Book, “Who’s Afraid of Bugs?” provides a playful method for exposure and learning through story and technology to help children overcome their fear of bugs and discover that they aren’t so scary after all.
A portion of the AR Pop-up Book was demoed live during my talk on AR and Storytelling at ARE 2011 in Silicon Valley – watch video of my talk and slides here.
Design, concept, paper-engineering and production: Helen Papagiannis
Special thanks to Lukas Gruber for support with 3D model conversions.
AR Software: Junaio
*UPDATE: I’ve been so flattered by the wonderful response to my AR Pop-Up Book! Great big thanks to everyone for blogging, tweeting, and sharing my work, I really appreciate it! Here’s some of the coverage, thanks again: WIRED, Games Alfresco, Augmented Reality World, Adafruit, VR News, Junaio
Back from the awesomeness that was ARE2011 in Silicon Valley. The wonderful team at Layar recorded my talk and posted it with a very nice write-up on their website here. YouTube video of my talk and my slides are embedded below:
Talk description:
Augmented Reality (AR), like cinema when it was first new, commenced with a wonderment of the technology, with content being secondary. It’s now time for content to catch up to this emerging technology. Helen looks to the past to media theorist Marshall McLuhan and filmmaker Georges Melies for clues as to how we can advance compelling content by harnessing AR’s unique characteristics. (In addition, when presented at ARE 2011 in Santa Clara, CA, Helen showcased a live demo of one of her interactive AR storybooks.)
A big thank you to the ARE2011 organizers Ori Inbar, Chris Grayson and Tish Shute for putting together a stellar event!
I’m super flattered to be included in the Top 10 Augmented Reality & Gamified Life Talks! Wow, what great company to be in, I’m totally floored: Bruce Sterling, Blaise Aguera Y Arcas, Jesse Schell, Pranav Mistry and Julian Oliver, to name a few! Thank you kindly to Gary Hayes for including me among these luminaries!
You can catch many of these AR heros discuss Augmented Reality at ARE 2011 in Santa Clara, CA, May 17-18. I will also be speaking! Looking forward to it and hope to see you there!
