GU Journalen Coverage

With the recent work of Swen within a Vinnova funded project, we were able to look into generative approaches to create sounds. During the summer, Swen was interviewed by GU Journalen for their Winter issue.

• For the Swedish version: read here
• For an English version, read on...

Movements can not only be seen, but also heard.

This is the basis of the Sonic Dancer project. With the help of a small device, which translates movement into notes, people are able to dance together – even if they cannot see one another.

Swen Gaudl, Senior Lecturer at the Department of Applied Information Technology, previously worked at the University of Plymouth. It was there that he met dancer and teacher Silvia Carderelli-Gronau towards the end of 2020.

• Because of the pandemic, there were a number of new contact restrictions in place which obviously made things difficult for dancers. Meeting via Zoom, a solution that worked in many other fields, was less helpful to them as dancing usually means there will be some level of physical contact involved. As such, the pandemic led to most dancers being able to neither perform nor practice together, Swen Gaudl explains. One way to attempt to bypass this might have been to place out cameras inside the studios where they practiced, so that everyone could match one another through a computer screen.
• But to try watching a screen while moving about isn’t all that easy. Besides, a singular camera would focus on one spot alone and wouldn’t be able to follow the dancers around as they go. Adding a greater number of cameras quickly becomes expensive and it’s all rather clunky in the first place. Alongside sound designer Dan Pollard, Swen Gaudl and Silvia Carderelli-Gronau began experimenting with sound. This culminated in the creation a little blue device, called the Sonic Dancer. During the GU Journal’s visit, Swen Gaudl placed the item on the floor in one of the department’s rooms.
• What we wanted was a cheap and easy-to-use item, something that wouldn’t require the user to possess any particular set of technical skills. What we arrived at was this device, containing eight microphones that, when their sounds are combined, can track the movements of up to ten people in the room. Each device can be configured in their own unique way, almost like an orchestra where there are strings and wind instruments, percussions and so on. The device is comparable to a lighthouse, which the dancers move in accordance with, Swen Gaudl says.
• It can also be likened to one half of an onion that has been cut into eight equal pieces: each piece consists of several layers that one can move further in or out of in order to produce lower or higher tones. With the help of these sounds, the dancers can then accompany one or several of the other dancers and in that way attain a sense of moving together. Several devices can be linked together through Wi-Fi so that dancers who are in different locations can follow along with each other’s movements, Swen Gaudl continues.
• It’s like a sort of video conference, where the connected dancers can move around inside whatever room their device is in and create a sound response to all other devices. When the dancers then interpret the sounds that are played, they get an idea of how the other participants are moving while also receiving feedback on their own part in it. As they cannot be distracted by anything visual going on, it is also easier to concentrate on following along with the movements. Another perk is that even those visually impaired can participate in this type of practice.
• You can also dance in poorly lit rooms or even outdoors in nature, since Sonic Dancer can run on batteries. Effectively, Sonic Dancer makes it possible to experience different rooms in entirely new ways, as it picks up sound, for example traffic outside the window, in a way one might not otherwise think of. The Sonic Dancer can also register the user’s breathing.

• What effect that might have, for instance during meditation, is something we wish to examine more thoroughly. Swen Gaudl further proposes that the Sonic Dancer could be used in an array of different circumstances.
  • We believe this device could prove valuable in other situations too, for example by connecting people who are very far apart through the use of movements. It’s also possible to record one’s own or other people’s movements and then integrate them with things that have been made previously. With the help of Sonic Dancer, you can collaborate with people, not only though space, but through time as well. The bodies in motion become a part of the very room in which the sound vibrates.

Facts: Sonic Dancer is a small device with eight microphones inside, that track the movements of multiple people inside a room. Several devices can be linked together, where each device can sound in their own way, so that people in different locations can move together and hear how the others are moving. The device is developed by Senior Lecturer Swen Gaudl, and dancer Silvia Carderelli-Gronau in collaboration with a team of researchers, dancers, sound designers, and accessibility experts. With support from Vinnova, the project team, including an AI researcher from the USA (Dr Amy Hoover), currently investigates using generative AI to help designers engage with the novel control for creating sounds.