Visual Space is a project which uses a Microsoft Xbox Kinect to control video clips, visual effects and sound via hand, head and shoulder movement.
According to the developers, tracking data is extracted in processing by using the SimpleOpenNI library and sent over to Max/MSP Jitter as OSC messages via Maxlink. In Max/MSP, the joint position data is used to calculate a manipulation range for each limb. This allows gradual control of colors, sound, live video input and various other effects.
Here are some additional details from the hack’s creator:
“The program I wrote also specifies positions for different joints in which new footage is being triggered. For example, moving the right hand or the right shoulder triggers pre-recorded clips of a lady reaching forward and turning, which imitate the user’s actions. The application also features a customizable interface that allows loading in videos and sound files as well as to select effects. Clips and visual effects can be deselected, changed or linked to different joints at any time. Therefore, Visual Space facilitates motion controlled video mixing in real time, allows for numerous modes of interaction and can be easily customized.”
The project was developed as part of the creator’s MSc in Creative Technologies at De Montfort University, Leicester.
