A Story of Augmenting Reality

At VIKA we make extraordinary books that blur the boundaries between real and virtual formats. Our first production, Where is the Bird? is the first Augmented Reality (AR) book to promote British Sign Language (BSL) as a language for all children, d/Deaf or hearing. It’s a playfully immersive book and app designed to go wherever you go. Turn the pages and wake magical ‘pop-up’ AR animations paired with videos in sign language.

Where is the Bird? was conceived after a frustrating experience learning sign language with my hearing son. It was tricky to learn a new language and learn to be a parent. Teaching materials for BSL are surprisingly limited, even though the impact of signing with children under two is huge. This is a particular problem for the 90% of d/Deaf children born to hearing parents. For hearing families, complexities of speech are a possible contributor to the ‘Terrible Twos’, a time when children have independent thoughts, but cannot adequately express them. Where is the Bird? capitalises on this small window of opportunity, when hearing parents are proactively willing to learn BSL with their children, to increase national awareness of BSL.

In 2019, I received Innovate UK’s Audience of the Future Award to research the potential for AR to inspire them to use BSL in the homes of hearing parents. The design challenge was to design a new publishing format flexible enough to be used on a smartphone or on a book, giving new parents an on-demand choice between reading print and viewing curated media. The award stipulated a user-centered design process constructed around the Double Diamond design methodology.

I proposed two design rounds: R&D to prove the concept followed by fine-tuning a working prototype, each following four design phases: Discover, Define, Develop, Deliver.

1. Discover

I conducted market research into the tech, subject matter and the competition. I wanted the product to be fun and aimed at young parents, so no scrolling dictionaries or childish graphics. My biggest challenge was connecting with the d/Deaf community. VS1 Productions became my lifeline, introducing me to influencers and teachers at Elmfield School for Deaf Children. Clear feedback was to ensure the d/Deaf community was consulted and their language was respected. “If a Deaf person can do the job, they should do the job!” was advice that turned inclusivity into a priority. I am proud that 50% of the production team for Where is the Bird? are native BSL users.

2. Define

Four sample AR animations were created to check the AR worked before building the final ten. I’d planned parent workshops, but soon realised I needed a system that didn’t require parents to come to us. My user-testing group was small and comprised fellow colleagues at the Pervasive Media Studio, Bristol and busy, working parents familiar with tech development who could see the vision behind my buggy, draft UX. Feedback was personal and was collected over cups of tea, WhatsApp and email. Everyone gave honest, practical, technically savvy feedback. All suggestions that could be added to the app were integrated straight away. This meant our user-testers were really engaged with the process. Once the feedback became less conceptual and more specific, I knew I could start developing the full prototype.

3. Develop

We developed twenty AR/MR animations and then expanded the user-testing group with a callout on The Bookseller. The hardest part of testing is to get the user to download the app, so a lot of work went into a streamlined and accessible onboarding process. I photographed children using the test booklets and app super early so our instructions were accessible for the d/Deaf community—written English is their second language—and increased user confidence and posted detailed instructions on a learning resource website (babybsl.com). My son and his Brazilian relations feature heavily in these early pictures.

All feedback was logged and informed every design iteration for the UX and the booklet with parents, pre-schoolers and members of the d/Deaf community. Each round supplied invaluable insight, guidance, and first clues for a commercially viable design. Had I not had user-input from the beginning for such a complex concept, my assumptions would have guided me in the wrong direction. For example, in the picture below there is a traditional style illustration of how to make the BSL sign for ‘bird’. These were created for all twenty words, but we took them out because members of the d/Deaf community said they not working.

All efforts to make our communications accessible to the d/Deaf community paid dividends. As a hearing person with no prior contact with the d/Deaf community, I was oblivious to the now-rather-obvious-fact that English is a sonic, linear language entirely separate to BSL. I’m sorry to say that I never thought about the relationship between d/Deaf children and reading stories, and that the foundation of literacy in hearing families is based on sound as we read stories aloud. With no easy route into phonics, d/Deaf children often find books static and exclusive. The d/Deaf community showed me how beautifully matched Augmented Reality and BSL are, as the tech enables a silent page to spring to life, enhancing a uniquely visual language. By creating a technology that uses illustration to trigger BSL videos next to words, we can start to translate stories into BSL and perhaps increase d/Deaf literacy. This concept is the core of my patent-pending technology.

4. Deliver

Since the SWCTN showcase and until I’ve released the book to trade (we’re looking for distribution partners!), I continue to test the book within communities. As a small publisher, I can’t rely on one big launch, so I’ve been conducting a series of launches to different communities, from book lovers to play specialists at the Bristol Royal Hospital for Children, collecting feedback along the way. When C19 hit, I created A4 colouring sheets for home-schooling and NHS waiting rooms (https://bit.ly/C19homeschool). I am particularly proud of our latest user-tested review in the National Deaf Children’s Society Magazine.

Where is the Bird? is still being developed and we are hoping to find funds to develop the full interactive learning system. The book can be used as a storybook without the app and it can be used alongside the AR and the videos. Extra information can be accessed through the app enabling readers to choose how they want to engage with the world of Baby BSL at various points in their daily routine. It is designed with a soft-touch approach to introduce immersive technologies in a gentle, ethical and unobtrusive way to young families. Where Is The Bird? is a book which is enhanced by the power of AR and accomplishes its purpose more successfully because of the use of technology and the user-centered approach to its creation.

Written by Victoria Forrest. Published in BookMachine.

Where is the Bird? is available on Amazon: http://bit.ly/babybsl. The BabyBSL app containing all 20 signs is available for free download from App and GooglePlay Stores.