This project was an extension of a "Noto the Talking Ball" tutorial, which explains how to create a 3D grid with each cell moving up and down based on mouse interactivity.
This version adapts the visual elements of the tutorial into a beat machine. A white-on-black image is input into the grid, where white pixels are raised and black pixels remain at the default height. I then divided the grid into multiple rows, with each row corresponding to a different sound. A vertical bar sweeps across the grid from left to right, acting as a timeline. As this timeline passes over an active white cell, the corresponding sound is played, depending on the row in which the cell is located.
The concept involves using a depth-sensing camera positioned above a table to allow people to alter the beat pattern by placing items on the table. A projector could also be utilized to display control areas on the table, enabling users to manipulate aspects such as volume, instrument, tempo, and pitch by placing objects within these areas.