Imagine listening to your favorite song and watching it transform into a glowing painting created by moving robots.
Researchers at the University of Waterloo have developed a new system that does exactly that, using a group of small robots to turn music into colorful light art on the floor.
The system uses several wheeled robots, each about the size of a soccer ball.
As music plays, the robots move around within a defined area, leaving behind trails of colored light.
A camera above records their motion, capturing the glowing paths as they twist and overlap to form a visual “painting.” The result is a unique artwork that reflects the mood and emotion of the music.
Dr. Gennaro Notomista, a professor of electrical and computer engineering at the University of Waterloo, explained that the goal was to program a swarm of robots to respond creatively to sound.
The robots “listen” to features of the music such as tempo, rhythm, and chord changes. Faster sections may make them move more quickly, while calmer parts can slow them down. Different musical tones influence the color, brightness, and thickness of the light trails they leave behind.
The robots also change where they move on the floor, creating patterns that evolve as the song progresses. Together, their movements produce a coordinated display that acts like a living painting shaped by sound.
According to the researchers, the system is designed not just to follow music mechanically but to express its emotional qualities in visual form.
Humans can also take part in the creative process. People can adjust certain controls while the robots are working, such as changing the width of the light trails or shifting where the robots draw on the “canvas.” This allows a person and the robots to collaborate, influencing each other to create a shared piece of art in real time.
The research team first had to solve the technical challenge of controlling multiple robots safely in the same space so they would not collide. They successfully tested the system with up to 12 robots, and it could potentially be expanded to include many more. Another key step was developing software that could analyze music and translate it into movement instructions for the robots.
Although the project focuses on art, the technology behind it could be useful in many practical fields. Coordinating large groups of robots is important for tasks such as environmental monitoring, farming, search and rescue missions, and even exploring other planets.
The researchers hope to work with professional artists and musicians in the future and plan to hold public exhibitions so people can experience the technology firsthand.
By blending robotics, music, and human creativity, the project shows how machines can help people see and feel sound in an entirely new way.
