First premiered at Sundance, “Kusunda” is a virtual reality experience about the sleeping Kusunda language in western Nepal. Co-created with Kusunda shaman Lil Bahadur and his granddaughter Hima, this intergenerational story invites you to support the challenging task of revitalising a dormant indigenous language. We spoke to the creators Gayatri Parameswaran and Sönke Kirchhof about making this unique volumetric film.

What is “Kusunda” about?

‍Gayatri Parameswaran — This is a voice driven virtual reality experience that's co-created together with the Kusunda community in Nepal, which is an indigenous group who are working to revitalize their language. So in the VR experience, you meet some really special people. You have to learn a few words in the Kusunda  language and speak them out loud in order to move further in the narrative.

Why did you want to tell this story?

GP — The journey of this piece started 10 years ago when my co-director/producer Felix and I were in Nepal and met the matriarchy figure who was one of the last fluent speakers of the language. She's this amazing badass woman who really wanted to keep the language alive. At the time, we made some journalistic pieces, some radio, TV and multimedia work. And then we wanted to do something deeper. So together with her, we started conceptualizing this VR piece.

‍

‍

What would you say makes this project really special?

‍GP — There's nothing more human than language. When we started researching the theme, we found out that half the languages that we speak today are at threat. I cannot speak my mother tongue every day because I live far away from my family and from the community where I can speak the language. I'm very aware that this might mean that the next generation probably doesn't get and keep the language.

The number of languages that we speak today is approximately six to seven thousand, and at the turn of the century, it could be down to as low as 3,000. It's shocking that we lose a way of looking at the world. We lose a lot of knowledge that is embedded in a language. Many languages that are under threat are are spoken by communities that are marginalized. So it was easy for us to understand the global implication of this story.

‍

Those numbers are devastating. It’s in the same order of magnitude of habitat loss or the extinction of species. With the thousands and thousands of years that went into building the cultural identity within those languages, it's truly quite tragic and quite beautiful that you're able to preserve this particular story within the Kusunda language.

‍GP — We were speaking to Kent Bye on his podcast and he called it “spatial anthropology”. This is a field that's growing and it needs to be taken with a lot of responsibility. But at the same time, I think it's a really exciting place to spatially be able to represent a community’s stories.

‍

‍

‍

‍Do you identify that you are now a spatial anthropologist, and it's the lens that you're going to use going forward in your future work?

GP — I'm a storyteller and a creator. In this piece, we worked closely together with the community who are the storytellers. So they tell the stories, and we are kind of the messengers or carriers who take it to a global audience.

‍

‍Could you each share a little bit about your own creative practices that you brought you to working on this story?

Sönke Kirchhof — I created a company 15 years ago that specialized in stereoscopic 3D filmmaking. This was five years before “Avatar”, so very early days and a little bit too early. In those days we couldn't just shoot something and give it to a post-production house, because nobody really knew about how to process this stuff. Because of that, we started to go into research and development and work on workflows and pipelines and software tools. So that’s the tech side where I came in. What’s always interesting is to push the boundaries.

‍

I started first in the field of political and social science, and then went to the film school and studied film production. I decided to do something where I'm able to do my own calculations and schedules and hire people for projects that I want to work on, coming with the mindset from social science being interested in the issues of our global world of neoliberalism capitalism. As an artist, of course, the stories from around the world about interesting people that you never met before, trigger something. The medium of VR makes it possible to get much closer to there, but tell stories in a completely new way. That makes everything in VR appealing for me.

Now to Kusunda. A language that I spoke at home growing up in the countryside in North Germany is Plattdeutsch. It’s close to the Danish language and to Dutch. I was still learning that in school. I got first prize in reading competitions and now, nobody speaks it anymore. It's lost. So that's why I was interested in this specific project as well because I know the story of languages that you would like to speak with people, but nobody is out there to speak to.

‍

‍

‍GP — Felix (the Kusunda co-director) and I met each other when we were doing a masters in journalism. We specialized in war and conflict reporting, so we started going out in the world and telling really difficult stories about human rights. One goal that we had always was to kind of represent underrepresented voices in mainstream media and to reverse the power dynamic that exists in the media landscape. How can we give voice to those who do not always have the means to express themselves and be heard? We started experimenting with a lot of different formats. Although we started out with the written word, Felix comes from a much more visual background, and I was a radio reporter for a few years of my life. So it really made sense for us to combine our skills and start experimenting with format. And what we realized is that each story lends itself to a new format.

We stumbled upon immersive media like virtual reality. I remember putting on the headset for the first time seven or eight years ago and being blown away. In my work, it was very hard to take people to the places that I have been to and to tell them exactly what I had experienced. And with virtual reality, it was kind of a plug and play. I had never experienced something like this before, and I had never found a tool like this. And that made me think this can transform the impact our work has. With that in mind, we co-founded the studio and started working in this intersection of storytelling technology and impact. A lot of the work we do wants to achieve real life impact. It's a great day to have this interview as well because talking about we've just kickstarted a language revitalization course in the Kusunda community, run by the Kusunda community today with the help of funds that we were able to raise partly because of our virtual reality work.

‍

‍Were there any particular creative references or inspiration that informed what this VR project looks like and how this story was told?

GP — We were part of the Sundance New Frontier lab. Through the amazing conversations that we were having there it became clear that we are not going to do justice to our audiences if we do not tell them how this was made. It's not a piece that takes shape in vacuum, you know? There is a context in which it happens and it became very important to make references to the process itself. A lot of the early virtual reality documentaries were neatly edited and fantastically put together as though it all happened out of nothing. For us it was actually really important to show the rough edges and talk about the challenges of this as well. There is no fourth wall.

‍

‍

‍What tools, techniques and technologies did you use to create “Kusunda”?

‍GP — Normally our process involves a text and audio prototype of the story first, because the story is so important. We take everything that's going to contribute to the story and just read it out and see if that makes sense. It's a cheap way to prototype. After that we go into a linear version of the piece, which is a video version with a lot of storyboards and animatics.

Then we take it into the game engine – this project was created with Unity and also other 3D software like we used Blender and Tilt Brush. All the animations were captured using motion capture, so there was an OptiTrack system. For the volumetric video, we used Depthkit. We did color grading in Resolve. Sound design was done in Sweden as part of a co-production with a Swedish company called We Are Machine, and then we  did a lot of work on the Unity timeline with the audio as well.

‍

‍How did you use volumetric video in this project?

GP — We wanted a really impressive way of capturing these stories and capturing them spatially. At the same time, the biggest challenge of this piece was that it's located in a remote location in the Himalayas. There's not even a road leading up to the place where one of our storytellers, Lil Bahadur, the shaman, lives. So we had to lug up all the equipment and build up a studio in their home. Our laptop sucks so much energy that the generator blew out.  We had to kind of hire the local village electrician to be part of the crew for the five days. We used Depthkit with one sensor and a DLSR to capture the volumetric videos. We were able to capture in 2.5D and we found that this is the best possible solution in a location like this, given the constraints and given the independent nature of the project.

‍

‍

SK — Beside the creative aspect, we were very interested in how we could improve the quality of what we are getting out of there, like pixel bending aspects, and the shapes of the body.  In a 16:9 TV documentary, we're totally used to seeing low light or grainy images in a documentary. But in VR, people are a little bit more sensitive. We try to keep an eye on these aspects to modify as much as possible, while not wasting resources and trying something that is kind of impossible with the setup.

‍

‍Given what you know now about telling this very complex story, with this beautiful collage of techniques, what would you do differently? Or what will you change about the next project you work on?

GP — Actually one of the next projects that we are working on is a simple immersive audio story and it is so nice to not have to deal with a hundred and fifty million problems, but only like 20 million problems. You also realize when you're telling a good story it's completely okay to keep it simple.

We were very, very ambitious with Kusunda. We couldn't do it without all the amazing partnerships that we had. We have so many different players. It was this matrix that brought in its own kind of magic and we created something that was beautiful. But it needed three years of work and raising financing for this. And it's very hard to do that for every project. This is also an impact project and it's not a classic entertainment piece, so it's going to be very, very hard to see any revenue coming out of it. I'm very satisfied with the result of this work but it raises the question on how sustainable this might be and how we can keep this ecosystem thriving with works like this which are not classically profitable.

‍

‍

Where to next for the project?

GP — This year, we had a big screening: multiple days of screening in the Kusunda community in Nepal. That is our first audience and commitment with respect to this piece. It's very, very hard to reach an audience which doesn't necessarily have resources for a PC VR experience. We have now been able to create a non-interactive 360 version of the piece, which is going to be distributed on cardboards in the community. We are going to create backpacks with headsets and mobile devices that's going to travel around villages in Nepal. We also created a 2D version of the piece which will be screened in communities in Nepal as well. So that's one part of the impact strategy and distribution that's running right now. And we want to reach more. Because of Covid, we haven't really been able to distribute it at indigenous festivals where people who face similar challenges within their communities come together and discuss these topics.

What we realized is that although there will be an online release of the piece on Steam, not so many people will be viewing that version of the experience. So we have to take it to places like museums, public installations, perhaps schools and universities to show it to audiences.

‍

‍

‍