Massive Robotic Instrument Turns Smartphone Data into Sounds

Using hardware and software, artist Andrius Šarapovas created a massive musical composer.

DJ Pangburn

DJ Pangburn

Images courtesy of the artist.

Imagine the 4G data from a million users—it's chaos. But when filtered with the right tools, interesting things happen, as is the case with Vilnius-based artist Andrius Šarapovas's room-sized kinetic sculpture that turns 4G data into music. The installation features 77 individual segments—some fixed to walls, others suspended from the ceiling—that make generative music, each featuring a metal bar, sound activator, sound damper, resonator, and mechatronics (technology that combines electronics and mechanical engineering). In a sense, it it is a room-sized, networked robot or a massive interactive musical sculpture; an ever-evolving one, given his access Tele2's 4G network from across Lithuania—itself governed by various algorithms from apps—which activates the installation's algorithm.

The installation's harmonic range, as Šarapovas tells Creators, covers 16 notes: four notes—C, D, F, and G—spread across four octaves. The sculpture's algorithm link the notes to the north (G), south (D), east (C) and west (F) of the Lithuania. The notes are distributed throughout the space so that, as the visitor walks through the installation, they hear a different harmonic composition at each point.

"The rhythmic pattern produced by the statistics is based on a few rules that are set in milliseconds," explains Šarapovas. "It is quite impressive what a great complexity is generated using a set of simple rules. The probability of the repetition of an identical four-note combination is about once in every 98,304 activations."

The algorithm, written for installation, uses the extremes of 4G statistical data, with one second of data generating one second of music. The number of connected sessions (from Tele2's 4G network) determines the rhythmic musical pattern, as well as the velocity of the activation (sound volume). Each note's pitch is determined by the amount of data being downloaded, while the number of connected sessions also determines the brightness of the installation's lighting and the speed of the shadow animation.

Early on, Šarapovas settled on a minimalist spatial object that would consist of a number of identical segments. He used materials of various hardness and adjusted the mechanical parts to balance the sound, control noises, dampen different notes, and activate sounds.

Šarapovas made several different segment prototypes before opting to use aluminum bars for their acoustic qualities. These bars, he says, can maintain long sustained notes and sound rich in harmonics. This allowed Šarapovas's team to use a unified wooden resonator, which provides the subtle yet quite significant lower frequencies, or what he calls "the body" of the evolving composition.

"The installation can also be viewed as a single large musical instrument that sets its own routine," says Šarapovas. Every Monday, when the gallery was closed, he checked and calibrated the key-stroke strength of each of the installation's 77 segments. On average, visitors spent an average of 15 to 20 minutes in the exhibition room, while some stayed for an hour or longer, sitting or lying down and walking in circles.

As an artist, Šarapovas is interested in the boundary where chaos and structure, perceptible to humans, meet; the field where both of these systems overlap, balance, and are expressed simultaneously. For him, this new kinetic sculpture is interesting because he and visitors can see the intricate complications that occur because of variations in a few simple rules.

"What's interesting is that not all of the data used in the project is generated directly as a result of human actions," Šarapovas says. "Some of it is determined by long-term choices to use one device or another, by the apps they've installed, or by the settings they use."

"Our smart devices are always on, updating their apps, refreshing their news feeds, and synchronizing their data," he adds. "This means that our kinetic installation's algorithm is also being activated by other algorithms, which can be affected by yet other algorithms, and so on."

Click here to see more of Andrius Šarapovas's work.


You Can Play These Giant Sculptures Like Musical Instruments

Massive Kinetic Sculpture Mimics the Motion of a Flower

Kinetic Sculpture Moves And Changes According To The Weather