Tonesight has been created at Simon Langton Grammar School for Boys by Cal Hewitt and musician Richard Navarro-Pollott.
It uses gesture recognition technology to detect a user's hand movements, conjuring music from the artwork based on the position of the hand in relation to the painting.
Moving the hand back towards the body will play a greater number of the sounds – giving a wider perspective – and moving in will ‘zoom’ in on a smaller area, and so a smaller number of the sounds.
This is giving visitors to galleries the chance to explore paintings and create their own arrangement of the music, with users describing the process as allowing them to build a mental ‘map’ of the image in their mind’s eye.
Created alongside blind and visually-impaired young people at the Beaney House of Art and Knowledge, Canterbury, the system has been used in Canterbury Cathedral, The Lilford Gallery and Turner Contemporary, Margate.
With an installation set for launch at Greenwich's Painted Hall (June 2019), a project underway at the Wallace Collection, Central London, Tonesight is inspiring collaborations between musicians, dancers, visual artists and audiences.