wiiPop

Unlike common dancing games, Wiipop allows the user to freestyle and improvise his dancing movements, instead of repeating a pre-defined rutene. The idea is related to the music game “SingStar”, but instead of singing the notes and lyrics of a song, you are able to hit the beats of the music with your body the way you want. The Application melts camera technology and accelerometer units in order to detect the fundamental features of the dancer’s freestyle. Our system allows you to play in Singleplayer Mode or you can freestyle versus a Friend in multiplayer. Additionally the Wiipop system and its underlying engine, can be used to explore different ways of performative art.
 

Common dancing games use human computer interface devices, like accelerometers or camera systems, in order to detect the dancer’s actions. The actual skill level is rated by the amount of precision; the dancer is able to repeat a predetermined choreography.However, today’s social dances like the urbanized Hiphop, Funk and House culture is raised and kept alive by a more spontaneous approach. The idea of improvisation and the unpredictable creation, fill the core of those dance forms since more than 40 years. The cultural fundament is based within the idea to honestly speak up, communicate and create instead of simply repeating and simulating. Therefore dancing games, consisting of the idea to repeat the movements of a choreographer, contradict with the key aspects of speaking up and creating.The WiiPop Prototype aims to measure fundamental qualities of a freestyle dancer, like spacing, musicality and the incorporation of individual body parts in order to evaluate the dancer’s skill level. This concept enables the dancer to express himself, to be creative and to monitor the level of tools he uses in order to move.Dancing consists of both, small and big movements. Therefore we combine the Kinect Camera, detecting spatial movements of the user, with the body mounted Nintendo WiiMote, measuring fine impulses of the body. The melting of the two sensor technologies, offer the possibility to evaluate a complete and detailed view on the dancer’s behavior. Since social dances are mostly related to music and percussive events, we decided to enhance the system with a beat detection algorithm, measuring the dancer’s synchronicity in respect to the beats of the music.Wiipop, implemented as a video game, demonstrates the possibility to analyze and categorize dancing behavior. However, the main idea of this project is to investigate different software and hardware solutions to digitally encrypt the language of dance. This project does not intend to capture or judge the full poetry of dancing art. Instead, it aims to push the level of physical detection and algorithmically categorization.

 

The Wiipop System aims to measure the qualities of a very specific dance style called popping. Unlike classical ballet, contemporary dance or ballroom dances, popping is closely related to robotic mime and consists of clean mechanical signals. Hence this dance, offers a lot of possibilities to extract discrete information from the performed movements. For example the simulation of mechanical tools bouncing, locking and unlocking. Therefore the dancer performs muscular impulses (Pops) to the beat, which are isolated from swinging arms or jumps. In between those impulses the Dancer uses different transitions in order to transform into the next pose.Our implementation, to analyze this dance style, is based upon three data rooms.The information which is:

  • extracted from the music.
  • detected by a body mounted accelerometer (Nintendo Wiimote)
  • detected by a depth camera (Microsoft Kinect).

By synchronizing the data of the camera system and the data of the accelerometer, we are able to evaluate a precise picture of the physical dancing behaviors. In order to monitor the musicality, we match the accelerometer values, indicating the pops of the body, against the extracted percussive events of the beat. By melting the detected musical and physical rating, we are able to approximate the dancer’s overall skill level.

The software is written in java based processing. The library to access the data forwarded by the Kinect camera is SimpleOpenNI (Max Rheiner).

The application is created in processing 1.5 (even using the processing IDE :) ) and utilizes a melt of kinect and wiimote sensortechnology.
The Kinect Camera
The Kinect camera is a structured light camera by Microsoft. We capture the camera’s data by the use of the wonderful Library “SimpleOpenNI 0.24″ from Max Rheiner.

The WiiMote

The Wii Remote is a controller for the Wii console by Nintendo. We use glovepie 0.43 form Carl Kenner to read the data of the wiimote and forward it to OSC. In order to read this osc stream in processing, we use the OSCP5 library from Andreas Schlegel.

The Music

In order to detect the beats of the music, we use the standard Minim library for processing.
In order to synchronize Wii and Kinect data, we use timestamp methods and the manual assignment of the user, referencing which Wiimote is attached to which bone of the sceleton. This process is simple and related to the way people choose sides in video games. In this case you choose Joints.Nearly all of the control elements in this video game are created by the use of the well known controlP5 library from Andreas Schlegel.This Video features a sequence at the end containing particle systems reacting to a dancer. This kind of particle motion is created by the use of the “MsaFluid” library from Memo Atken.
 

Our main motivation to develop different concepts for the analyse of dance, is to explore new ways of performative art and choreographic productions. Especially the combination of a physical artist and media technologies in real time performances describe the core of our work. Hence we are looking for different concepts to make the language of dance accessible to digital worlds, in order to enhance the possibilities of life artistic computer interaction. The beautiful works of Klaus Obermeyer and Frieder Weiss for example, utilize camera systems and projectors to enable dancers interacting with the digital controlled elements of a theatre stage. The projects of those artists and many others have been a huge inspiration to us.

The biggest challenge to create WiiPop, was the development of the Pop detection algorithm. The Pop is a muscular impulse, performed to the percussive events of the music. This impulse is measurable with the accelerometer of the Nintendo Wiimote, mounted to the corresponding limb of the body. The main goal of our algorithm was to detect discrete information, if the Pop was performed and continuous information, about how strong this impulse went through the body. In order to evaluate which ejections of the accelerometer indicate a Pop, and which ejections indicate rubbish shaking of the bones and swings, we need to predefine a couple of assumptions of what a pop is.First of all the Pop can be observed as an outstanding emission from the accelerometer. In my dancing classes I often tend to describe this motion, as something that all of a sudden appears and instantly needs to disappear, before the observer even recognizes the root of this impulse. This feature of the Pop separates it from the shake, which is multiple accelerated in different directions.Additionally our Investigations offered more insight about the nature of this tiny movement. By moving a limb of the body in one direction, we naturally charge it into the other direction. This phenomenon can be easily observed for example at a crouch start of professional sprinters. Before the sprinter starts running into the desired direction, they push themselves even deeper into the other. They use this moment in order to charge the muscles in the opposite direction before they release the power into the direction of the track. This movement which is often described as rebound or counter (Storm), is very economic. Therefore we do so unconsciously every day while lifting a cup of tea or taking a walk. Our investigations indicate that the same process can be observed, while a dancer performs a pop. Before he actually releases the jump of the muscle, he charges the muscles into the opposite direction of the desired force. Describing this method algorithmically enabled the possibility to detect the pop by a smaller error rate. Further we described the strength of the Pop, as the absolute difference between the acceleration value of the charge and the acceleration value of the Pop.

Comments are disabled.