Living in the Future, Lost in the Snow. Finding Closeness in Remote Collaboration in a Pandemic Hacklab

We know performances can bring people together in the same room – can they do so over distances? Can you make a meaningful performance when a room is empty?

Berlin’s CTM Festival has evolved over two-plus decades, shifting shape and form – “adventurous music and art” is the banner CTM takes for itself. The pandemic brought some of that adventure crashing down on us. The disruptive impact of COVID-19 on CTM is hard to overstate, even relative to other events. Multi-faceted, obsessively international, and enormous in scale and density, CTM tends to represent the opposite of what works in pandemics. It involves long-term planning, not spontaneous action, a patchwork of visas and travel that can be complex even in non-pandemic times, and crowded, intimate space was always a hallmark.

But the pandemic also reveals the fragility of international performance in general. It exposes the privilege of Berlin life versus a world that often is disrupted by disease, environmental calamity, and politics. From Tehran to Minsk, those political disruptions can drive culture online and out of physical space just as surely as a virus can. And we know that the coming health, ecological, and geopolitical landscape is likely to bring more challenges in our lifetime than just a novel coronavirus. So without arguing that any online experience is equivalent, there is an opportunity to turn a catastrophe into a training camp. It’s necessary.

LIVE FROM BETONHALLE

It’s the final Sunday of CTM Festival, and a handful of people scurry around Betonhalle, the cavernous black box converted from the former crematorium facilities of the silent green venue. Show time arrives, the lights come up, there’s that familiar rush of adrenaline and focus – but the stage remains empty.

The MusicMakers Hacklab has been a playground for experimentation inside the CTM umbrella since its debut in 2013. The program is an intensive, 7-day environment for open collaboration and invention. It’s setting up a moon base inside the festival, where artists from an open call can pursue ideas they’ve never tried before, without the usual rules, working with one another.

In the past, MusicMakers Hacklab participants always thrived on closeness. They worked in intimate collaborations, and shared space and experience not only while working but also while squeezing into Berghain, HAU, and other venues to absorb the festival. 2021’s edition reminded us that this is a luxury, a privilege. And so, to respond to these new pandemic challenges, it wasn’t enough just to stream some content or have some Zoom chats – the question was whether actual working relationships could form and thrive at a distance, even through crude technologies. It inverted the whole event; rather than investigating what can happen when people play in a room for an audience, it asked what is possible when those elements are lost. If hacklabs are meant to respond to unknowns with spontaneity, the task was to answer this uncertainty, too, with invention.

»I felt like I was living in the future, questioning how the future will be in a not-so-distant time,« says Pama (Ce Pams, aka Pamela Canoles), who joined from Chile. »I think the pandemic has made – or is making – technology advance more abruptly, or maybe we try to mimic reality with virtual tools,« she says.

The number of overlaid realities in that Sunday go-time moment would be evident to anyone in the room – like observing a sporting event involving ghosts. CTM’s video crew managed multiple camera operators and livestreaming servers. Betonhalle’s light and sound crews ran cues as if performers and an audience were present. But the performers had to play with one another – and the room in Berlin – across several continents and time zones and sometimes-unstable internet connections.

...rather than investigating what can happen when people play in a room for an audience, it asked what is possible when those elements are lost.

Olivia Jack, this year’s guest host, juggled audio chat with performers in a private virtual green room, swapping video signals from multiple online endpoints, switching audio streams, and hosting live coding visual and sound tools. I also called cues and triggered events between the real-time performance in Berlin and the tech crew, plus performers and public who had various degrees of lag. (By the time the public sees it, the performance is delayed by a few seconds – a side-effect of the streaming technology used.)

The challenge was to feel something in those virtual spaces, to make them performances again.

A FRAGMENTED LAB

Video calls with everybody at once? Tough, especially with inconsistent internet access. But working in smaller groups went better, and even drove new artistic outcomes. Surprisingly, intimate unstructured meet-ups mirrored in-person interactions, even when they were moved online.

Mint Park (BAKÁH): »For me, the whole week was like a trip to another underworld or a planet or something!...I think it's because I was joining from a different time zone and was up daily from 10pm–4am behind the computer in my basement studio after having a somewhat regular workday, funnily enough, at a closed-down techno club called Faust.« (In her reality and full-time day job, it was a dance rehearsal week.)

Olivia Jack: »To me, asynchronous collaboration worked really well during the hacklab. With everyone in different time zones, differing internet connectivity, and working with different hardware and mediums, non-parallel collaboration meant that people were able to generate ideas and then pass something on to another.«

Pamela Canoles (Ce Pams): »There was an immediacy of question and answer, cause and consequence, that created a very rich dialogue.«

The MusicMakers Hacklab was first organised in Kunstquartier Bethanien’s Projektraum in 2013, and so – minus participants – the co-hosts met to work in that room for the week also in 2021. Other than this, no two participants were in the same space; the hybrid was Olivia and myself as in-person co-hosts, and collaborators online. But collaboration remained the essential goal, meaning artists focused on working with one another online in order to realise shared visions.

Suren Seneviratne (My Panda Shall Fly): »I really enjoyed the early stages of group brainstorming. There was a flurry of activity spread across different breakout groups and it was exciting dipping in and out of separate discussions, trying to piece together what everyone was trying to say.«

Ruhail Qaisar: »The best thing was how the ideas weren’t allowed to fade into perpetual limbo; each project starts as a nervous germ in the head and, unless nurtured, dies instantly. Here I was, living in the Higher Himalayas in the northernmost remote frontier of India during a pandemic, and yet I was able to successfully take part in multiple projects during the Hacklab, execute them perfectly, and also develop a very intensive functional workflow through all the different modes of collaborations explored in the performance pieces.
Reflecting on language and translation led one group to deconstruct language itself and challenge the dominance of English in the programme – dubbing the result, across participants’ varying alphabets, »Quz民hi辩शन化證कॉ.«

Hui Ngo Shan Steve (Nerve): »The creative process of ›Quz民hi辩शन化證कॉ‹ was really challenging. We had a long discussion on the concept of language and dialogue. We were using a language – English – which is not our first language, to discuss our own home languages. The turning point was when we found a way to hack the ›dictate text‹ function of iOS, and we started to use our different voices and languages to jam virtually. The mix of iOS text and our voices going through OBS Ninja to the screen, speaker, and space in Berlin, resulted in what I’d call a dialectic work.«

THE PHYSICAL WORLD

Participant groups found varying ways of incorporating their surrounding physical environments into their pieces. In doing so, they faced a patchwork of varying pandemic restrictions. Turkey had an early curfew. Artists who expected to be in Europe were displaced. To add to that sense of location, one group invited all the hacklabbers to record impulse responses (a way of recording the audio profile of a physical space) to use as a reverb.

Ruhail Qaisar: »I drove to record the interiors of huge canyons at a bouldering range, which I later morphed into a reverb convolution sequence for the ›I am Morphing in Your Room‹ piece.
As a part of the ›Total Astral Recall‹ performance, I added a contribution from my family archive of vernacular audio-visual material, and it was very surreal to hear the audio from my local dialect bounce off against the walls inside Betonhalle. [Mics in the space returned the audio to the participants.] With the language piece, it was amazing to see the live feedback of the text-to-speech AI processing input from multiple dialects with our cut-out heads on the screen. It certainly felt like a futuristic public address system, glitched-out, making multilingual announcements, its own corrupted memory feeding back into the input. Almost something out of a cyberpunk narrative.«

Mint Park: »I thank our microphones and speakers! Thanks to these tools we were able to communicate and also morph around different spaces around the world.

It was nice to jam with people at the end of the day, not through thumbnails on webcam, or some code, but through sound. We ran into some syncing and port issues, but other that that I was surprised at how easy and smooth it was to connect and play. I think the jam sessions were our main outlet for live collaboration.«

TOOLS

The hacklab employed various tools and platforms to collaborate and play – both for the organisation and development project, and within each group as they selected and modified the technologies needed to perform. Co-host Olivia Jack is known for her Hydra live-coding visual and audiovisual tool. She has also developed her own LiveLab tool and worked with it in the remote Hacklab environment, which gave her a deeper technical understanding of how other tools function optimally.

Different tools also lead to different interactions. Some tools involve live coding–executing musical ideas, for instance, as bare-bones instructions that embody patterns of melody and rhythm. These environments are well-suited to improvisation and collaboration even in poor connectivity situations, and found their way into the live performances. Tools like Discord allowed simultaneous audio chat in informal rooms – essential for putting heads together in working sessions, but also a remote equivalent to wearing headsets or using walkie-talkies on show night. Various options piped live video and audio; some music-specific ones were geared to synchronise time and beats. And then a tool called Work Adventure allowed random encounters as if in a game overworld.

A few of those tools – all of them free and open-source software with the exception of Discord:

• Jitsi: video conferencing
• 
  
• Discord: community text/video/audio chat (proprietary)
• 
  
• LiveLab for CultureHub – »media router, video chat, web-based tool, long time in development, open-sourced at the pandemic,« developed by co-host Olivia
• 
  
• MediaSoup: WebRTC video conferencing
• 
  
• NINJAM: audio live jam client from the developer of Reaper
• 
  
• Jamtaba: audio and MIDI live jamming tool with rooms and chat
• 
  
• Hydra: live-coding visual platform in-browser, developed by Olivia
• 
  
• OBS Ninja: (now renamed VDO.Ninja since the hacklab) – open source tool for creating persistent rooms with video and screensharing
• 
  
• WorkAdventure: social interaction space presented as 16-bit RPG video game (open source with freemium server)
• 
  
• SonoBus: high-quality, low-latency peer-to-peer audio streaming client with the ability to run as a plug-in as well as standalone
• 
  
• Estuary: live-coding platform with multiple interface and language support
• 
  
• SuperCollider: code environment for sound synthesis and generative composition

Some things worked reliably and creatively, some didn’t. The participants each had some favourite interactions.

Mint Park: »If I were to pick a moment, I think it has to be the time we were trying to make this live feedback loop with our videos and audio in Jitsi and LiveLab together – just putting things together and trying different ideas out on the fly. It wasn't indeed the most glorious moment, of course – it was rather challenging. But I got a kick out of the thorough and arduous focus that we got to put together as a team, whether or not it worked out in the end. Perhaps it's the pandemic that allows us this experimental spirit or mode of doing and thinking – just trying for the sake of trying.«

Olivia Jack: »I think it's no accident that NINJAM or Jamtaba as software kind of has that philosophy where it doesn't try to synchronise everyone, but actually keeps everyone in a predictable delay to the other participants in the jam. And so when you're jamming, you're jamming with the past interval of your collaborators. I think that's kind of inspiring as an approach to developing tools; when faced with a limitation like delay, don't try to, you know, pretend like it doesn't exist. Accept it and work with it. I think there is no such thing as zero delay. Even with sound in a large space, there's still some sort of delay. And that can be exciting.«

Suren Seneviratne: »I had never used Discord before, but the frequency in which our group was in contact, especially in the voice channels, really did feel like we were all in the same room! It took me by surprise. We were all spread across the world yet found pockets of time in which we all came together to meaningfully collaborate, despite intermittent Wi-Fi connections. Of course, nothing beats real physical interaction but for me, as the week became more focused and intentional, Discord text/voice channels were irreplaceable.«

Yifan He: »I enjoyed the voice channel on Discord a lot. When there's too many messages to read through, it's always better to catch up on everything by talking to one another. OBS Ninja rooms are great practical tools; I'll go back to them as well.«

Hui Ngo Shan Steve: »To me, a favorite was OBS Ninja. The tool itself is simple and basic. But it opened up many new possibilities for us to explore audio and video collectively. And I think artists can definitely use it in a way that is beyond its originally designed function. Also, although in the end we used Jamtaba during the performance, I found that I really liked Sonobus for sound jamming, mainly because it has a better mixing and monitoring function. I'm glad that tried this tool out during the week.«

Bruno Gola: »I think all the tools that enabled some sort of direct jamming were really meaningful to me: Sonobus, NINJAM/Jamtaba, and Estuary. Each of these has its own characteristics but in general, the feeling of collaborating through jamming, even online and at a distance, was really special after many months of almost no jams or collaborations, just working on my projects by myself. After a day of video chats for planning and discussing all the different projects we were working on, the jams felt like the perfect way to end a work session.«

Mint Park: »It was nice to jam with people at the end of the day, not through thumbnails on webcam or some code, but through sound.«

Mamady Diarra: »Estuary/ SuperCollider stood out for me. I was able to learn and share some new Supercollider stuff with the help of Mint and Bruno. Favourite moment? My time with Bruno. We're looking for a way to jam together.«

Olivia also reflects on her own tools and the ones that were most effective.

Olivia Jack: »We also used a streaming server that I created based off of an open-source project called MediaSoup. We used that just to share video streams from one place to another. In some ways, it was really stressful for me to use tools that I had developed before. I think when I use something that someone else made, I'm not thinking about all the bugs that it contains or could possibly contain... I thought it was great that we used a lot of open-source tools, specifically OBS Ninja, OBS, and WorkAdventure, which is a 2D kind of game-based video chat where you can walk around and encounter other people.«

Ruhail Qaisar: »My favourite was the gamification of the brainstorming sessions in WorkAdventure, where all the participants were routed to a beautiful two-dimensional old RPG minigame window. Once we would come across other participants, it would trigger a video chat. At times it was very funny and awkward, as you would be exploring the game world and bump into somebody while on screen with an awkward webcam face. Yet it really helped keep the interaction very natural, almost as if in a real space.«

THE NEW PLANETARY METAVERSE

What seemed to be a temporary accommodation early in our Hacklab planning in retrospect has become more of a lasting lesson – especially as we are mindful thinking ahead to 2022 and beyond, with a planet still in crisis. And in spite of the remote event format, what sticks with us are images of snowy Berlin mid-lockdowns, lights cutting through hazers, and the portals through the screen to the scattered locations of Hacklab participants around the world.

Olivia Jack: »I think it's so great to be able to work and learn with people in all parts of the world. And I think now there's really no excuse to host events with only US- or Europe-based participants. Not that every single event has to be some interconnected thing, but I would encourage people to look beyond their local or usual community when hosting an online event.«

Yifan He: »The language project group was supposed to reconvene at a certain time in the middle of the week. One day, after a short break, we lost Ruhail for several hours. Later we found out that his cat ran out of the house, into the snow. Because of the weather, Ruhail had to go out and search for his cat for hours.

It was nerve-wracking not being able to reach a member of the team and make decisions. But things like this happen. Not being physically by one another's side, we just have to trust and also be extra caring and considerate.

Poor cat, we are all glad she made it back home.

This text is an adaptation of an article originally published in the online magazine of CTM Festival.

Credit Title Image: Peter Kirn