Ryan Edwards, a lifelong musician, media artist, and sound designer based in Boston, has always been interested in music + “something else” as he puts it. He was never interested in just playing music with other musicians or singers, it had to be something interactive. In fact, his work is often haptic, creating community, agency, and participation through new-media and technology like IoT.
Courtesy of MASARY Studios
His focus in music, interactivity, and technology has actually led him and his colleagues at MASARY Studios to create many media-based installations that force audiences to reconsider how environments can be used for performances. In one of their newest installations, they have accomplished just that, while harnessing the power of IoT to bring audiences into the musical fold without years of training. Welcome to the world of Sound Sculpture.
Sound Sculpture — An IoT-Enabled Sound Installation
Sound Sculpture is an interactive sound and light instrument composed of 25 location-aware blocks. Each block represents a note and participants can freely move them around to make new pitches and rhythms, enabling them to be the conductors of their own musical experience.
“One of my passions is to bring people into a musical experience without needing years of training,” Ryan said about the project. “Sound Sculpture allows us to do just that — we can put music in people’s hands and give them an understanding of what composing means without needing years of musical experience.”
How Does Sound Sculpture Work
How does the installation and each block work you may ask? The blocks are made of a high-density poly-ethylene are 17” on each side, and are durable enough to be dropped, stacked, and stood on.
Inside each block is a core that is about 15 inches tall and includes a positioning tag, a lipo battery, an LED array, and a Particle Photon. The Photon is an integral part of each cubes’ communication to a controlling computer. The controlling computer communicates to the Photon what color each block should make based upon its position, including any color interpolation, fade-in/fade-out, and any of a variety of additional custom-built lighting effects.
Using the POZYX positioning system along with their freestanding “location anchors” and MASARY’s own custom software, the blocks are trackable within the activated space and their movements and location are constantly updated. The software tracks across the space and sends a signal to each block in sequential order — activating it with sound and light.
Courtesy of MASARY Studios
Ryan said that before the Photon, they were using a Pixel Pusher, but they found it to be too costly and large for the project.
It wasn’t until Jeremy Stewart, the Technical Director of MASARY Studios, was brought onto the project that he recommended they make the switch to the Particle Photon. “The Photon was the perfect form factor” he said. “It came at an affordable price point and wasn’t as bulky as the Pixel Pusher.” In fact, this wasn’t the first time Jeremy had used the Photon for an art-based installation. He has used IoT to create two types of musical-based media installations:
Field Cuts Encounter 6
Jeremey’s first project titled “PL:2” forces participants to consider their role and shared agency when interfacing with digital systems. He decided on this theme because his dissertation work focused on examining agency and effect of individuals within distributed multimedia systems.
In this project, audience members are able to use their personal mobile phones to interact with a digital black box. Through tapping and dragging, users can influence the performers, sound, and video within the installation. The user’s feedback is sent to one of several Particle Photons worn by the dancers, which in turn sends signals to specific vibration sensors sewn into the performer’s clothing, influencing performance.
His second project titled “Field Cuts: Encounter 6” uses a similar basis. Field Cuts is a multimedia movement performance that explores the invisible connections and communications between performances. The purpose being to challenge the performers to think about his or her movement in new ways.
Each performer is attached to a Particle Photon, which collects IMU sensor data and signals from mechanical relays attached to ten units on the performer’s bodies. The attached IMUs report back accelerando and gyro movements to the Photon while the mechanicals relays supply electrical impulses to the performer’s bodies as tactile feedback. Each performer’s movement is altered through the signals they receive from these various components.
What can you create with IoT?
From interactive sound sculptures to connected improvisation, these artists are finding new and inventive ways to bring audiences into the interactive performances with IoT. However, these projects just scratch the surface of what’s possible with IoT. If you’ve recently built an IoT project, feel free to share your own stories and tips for others who are building their own IoT driven-initiatives.
