©2019

An Augmented Reality Documentary of the 2018 Nicaraguan Resistance

The Amaya AR Project, is a 7 to 10-minute-long mobile Augmented Reality documentary-experience inspired by the Nicaraguan April 2018 civic uprising. This experimental art piece presents 2D images and source audio gleaned from viral videos and photos taken by Nicaraguan impromptu citizen journalism. Through my own personal experience from within the Nicaraguan diaspora, I bring to Toronto the story of Amaya Coppens, a Belgian-Nicaraguan political prisoner incarcerated in September 2018 and released in June 2019. The documentary is experienced in the physical space of Lisgar Park in Toronto through a hand-held mobile device and headphones.

THE AMAYA AR PROJECT

An Augmented Reality Documentary of the 2018 Nicaraguan Resistance

The Amaya AR Project, is a 7 to 10-minute-long mobile Augmented Reality documentary-experience inspired by the Nicaraguan April 2018 civic uprising. This experimental art piece presents 2D images and source audio gleaned from viral videos and photos taken by Nicaraguan impromptu citizen journalism. Through my own personal experience from within the Nicaraguan diaspora, I bring to Toronto the story of Amaya Coppens, a Belgian-Nicaraguan political prisoner incarcerated in September 2018 and released in June 2019. The documentary is experienced in the physical space of Lisgar Park in Toronto through a hand-held mobile device and headphones.

From April 2018 to February 2019, I compiled an unstructured media archive from Nicaragua, drawing from impromptu citizen journalism including viral images and sounds circulated on social media, television, newspapers, magazines and radio as events unfolded. Through two different methodologies: Research-Creation and Envisioning, I experimented with the source material in Augmented Reality (AR). The approach to editing material from the archive is informed by Hito Steyerl’s essay “In Defense of the Poor Image,” leaning into the glitchiness, blurriness and low-quality of the viral content to express the affects of the situation in Nicaragua.

 

The unstructured media archive-inspired the main questions for this project: How might be written and verbal testimonials be used to create an embodied experience for an Augmented Reality documentary once they have been published in television, radio, newspapers and social media platforms? And how might media archive materials, 2D and low-resolution content from television, radio, newspapers and social media platforms, be effectively included in Augmented Reality as a rich source of experience and perspective?

A COLLABORATIVE PROJECT

Why AR? Why just Image Tracking?

In the case of The Amaya AR Project, I wanted to share my experience of feeling I was living in two realities at the same time. I was in Toronto's physical space but my constantly viewing this violent content from my digital space (social media). AR reflected this sentiment as it allowed the digital same images and sounds to exist layered on my physical space and other people could also experience it.

Mobile AR was also inclusive with my fellow Nicaraguans, as most people have access to a smartphone but not to a VR headset, and The Amaya AR Project is meant to be used by Nicaraguans in the diaspora. Those who want to inform other people about what is happening and they might not have programming skills. The easiest way to create an experience shareable with Nicaraguans in the diaspora was to strip-away part of the technology and use image targets. I provided the option for people to download image trackers, and Nicaraguans around the world would only need to print the signs and I would provide instructions to download the app to their mobile devices without the need of tech programming experience.

UX

Initially, the app on the tablet included the UX instructions. After several tests, I realized most people have never used Augmented Reality as a way to follow instructions and encountered the next issue:
People were distracted by the technology, in most cases were more interested in the technology than Amaya’s story. For this reason, I decided to remove the wayfinding instructions from the app and instead use numbered physical objects to mark the places where people should stand.

OBSERVATIONS

-Participants enjoyed doing the experience with someone. With the use of an audio splitter, I connected two headphones to the tablet. Two people could walk around the park seeing the same visuals and hearing the same audio assets.

-Participants got very emotional, some of them cried.

-I received a significant number of hugs from strangers.

-Toronto is a city with a high number of immigrants and refugees, several participants told me they related to The Amaya AR Project. Some participants shared their personal experiences of living in the diaspora while similar situations happen in their home countries. They were mostly from Ukraine and Pakistan.

-After doing the experience, a significant number of participants researched on google Amaya’s story and then came to me and asked additional questions about the situation in Nicaragua.

-By removing the wayfinding AR instructions on the app and using numbered physical objects to mark where participants should stand, I noticed people felt extremely comfortable following the instructions that way. It allowed participants to fully immerse themselves into Amaya’s story. Only two participants required me to explain to them where to go.

-Participants and people walking around the park thought the signs were part of the park. Participants said the signs blended well with the environment and were not distracting.

A COLLABORATIVE PROJECT

The Process

The narrative for The Amaya AR Project was inspired by the interviews Amaya gave to the Nicaraguan and international press prior to her arrest. The interviews were gathered, summarized and translated from Spanish to English. My personal experience as a Nicaraguan living in the diaspora was added to fill the social and political context gaps and to center the audience to the Toronto Location.
 

The script consists of six scenes, in which I relate my feelings about unfolding events, and Amaya explains the situation in Nicaragua. It includes chants from marches, phrases from protests and pieces of advice from social media gone viral, voiced by Nicaraguans in the diaspora.

Audio Experimentation and Collaboration

The sound design of The Amaya AR Project is the result of collaboration with Kiersten Depina, a local sound designer.

Amaya’s words were performed by a Nicaraguan woman, vocal against the government and living in the diaspora. The Amaya AR Project uses sound effects from original audios in Nicaragua that had affective resonance, iconic phrases spoken during the protests and marches, and readings of written phrases that went viral on social media. This resulted in the project including three sets of voices - Amaya’s, mine, and those of the multitude (spoken by people from the Nicaraguan diaspora).

The final script was sent through email to Nicaraguans who met the parameters I had set (Living in diaspora and vocal about the Nicaraguan government injustice and human rights violations). Performers recorded their voices with their cellphones on their own time and gave their own voice emphasis the script. This resulted in imperfect, glitchy sound recordings, as discussed by Hito Steyerl and as encountered in my everyday connections with Nicaragua in Toronto.


Many of the recordings include voices with heavy accents. Most people told me it was truly meaningful and emotional for them to scream and vocalize the march chants and protest phrases. In a way, they formed a virtual protest, participating individually without knowing who else was involved, yet connected through their interest in doing something to support the Nicaraguan resistance. In this way, the project supports giving voice to the network of Nicaraguans in diaspora.

Image and Sound Citation/Credit Tables

Transparency in the project

Due to the political situation in Nicaragua and my schedule in Toronto, I was not able to travel. The Amaya AR Project was created with pieces taken from the internet, newspapers, websites, and social media. It is important to give credit and recognition to those brave journalists and photographers who had the courage to risk their lives to document the news. 

As part of the research-creation process, I created a series of citation/credit tables to keep track of the origin of the images and sounds. Though there are certain photos and videos I was not able to locate their origin.