the sound catcher


This project was developed in collaboration with research student Max Worgan. He used a variety of computer vision techniques to analyse the shadow puppet performance and generate an algorithmic musical accompaniment. Surveillance techniques for detecting, recognising and tracking objects were re-appropriated to map the visual information of the performance into a musical domain. Using a custom built computer vision system that identified the movement of the puppets, the structure of the piece and the overall activity, he produced real-time, generative music that reflected the puppetry.

The generative music score Max developed was inspired by gamelan, a form of musical ensemble from Indonesia. Chosen because of its historical association with shadow puppetry, its semi deterministic form and structure lends itself readily to algorithmic interpretation. Striking a balance between creating a pleasing piece of music and one that represented that action on stage was the main aesthetic challenge of the project.

Our first public performance was on 21st June 2009 at Blank Studios, Brighton. It was later part of The House of Fairy Tales event at Clumber Park, Nottinghamshire on the 8th and 9th of August 2009.