Freedom to Move About
Ever since the concept of STM came about, it had been a goal to eventually be able to experiment with some form of wireless sensor technology in live performance, one that could sense a performer’s motions with a degree of precision and subtlety sufficient to capture and convey the nuances that are essential to the form. For many years, the very high cost and relative scarcity of technology that suited the purpose served to prevent all but a few from participating in this enterprise.
Fortunately, that is no longer true. There are now scores of inexpensive sensors and wireless devices, widely available, that have fostered a virtual golden age of experimentation and innovation in the performing arts.
The Mimn wireless motion sensors, developed over several years by Donald Swearingen and Pamela Z, represent the first two experiments with these new and powerful technologies as applied to the STM concept. (Also of note is that a second major revision of SensorPlay, the STM host software, has been created with special emphasis on STM-5 and STM-6 instruments.)
WiFi Motion Tracking Design
Almost, But Not Quite
Mimn I, the first STM wireless project, was to be a transitional effort. Though it was hoped that it would become a reliable and useful new addition, it turned out, as experimentation often does, to be more valuable as a means of evaluating and refining a concept. In addition, it produced an architectural and design groundwork that later on would be applied to the design of Mimn II.
The images here detail, top to bottom, the components and layout of Mimn I. At top, is a bottom view of the device, with the bottom components removed. On the bottom of the PCB board, left to right, are a 4-pin header for the 6DOF motion sensor, precision resistors for voltage monitoring, and a 6-pin header for both a Bluetooth radio module and firmware programming attachment. Laid out at the bottom of the image are, left-to-right, a 3.6V rechargeable coin cell battery, the 6DOF sensor, and the Bluetooth module.
In the second image we see a bottom/side view with the components in place. The third image shows a top view of the assembled Mimn I, with its Arduino Pro Mini microcontroller. And the final, fourth image shows the unit attached to a stretchable wrist strap.
As can be seen in the video of Pamela Z’s performance (below), Mimn I, despite it’s relatively underpowered processor and early Bluetooth radio, was able, when it worked, to provide a very serviceable level of support to live performance. There were other problems, however, that turned out to place a rather severe limit on its ultimate usefulness. In particular, Mimn I suffered from two major shortcomings: short battery life and unreliable communications.
The relatively compact size of coin cell batteries made them what seemed to be a good choice for powering Minm I. It also made sense that they be rechargeable, a requirement, along with the 3.6V processor, that greatly restricted their availability. It turned out that only one readily available device, the LIR2450, matched the specifications. It would have to be recharged by an external adapter (Mimn I did not have an onboard charger), but, according to the published specifications of all the components, it should have provided well over an hour of operation on a full charge. In practice, the batteries were only able to provide 20-30 minutes of operation, far too short a time for reliable use in live performance, though we nevertheless carried on.
The second problem was the difficulty in establishing reliable communications with the host computer using the Bluetooth protocol. In addition to serious issues in pairing Mimn I with the MacOS host, the Bluetooth link was liable to fail in the middle of a performance, a literal show stopper. A better alternative was needed.
Mimn I Live
Breathing, Pamela Z (2014)
Breathing is a movement from Carbon Song Cycle (a inter-media chamber work by composer/performer Pamela Z and visual artist Christina McPhee). The work was originally written for voice & electronics, bassoon, viola, cello, percussion, and tape. This is a solo version performed by the composer (with just voice and processing) as part of the 2014 ROOM Series at Royce Gallery in San Francisco..
Pamela Z is using Mimn I wireless STM gesture controllers, one worn on the back of her right hand and another under her sleeve on her left forearm, to manipulate the processing on her live voice.
WiFi to the Rescue
The lessons learned from our experience with Mimn I dovetailed nicely with the appearance of a new generation of WiFi SOCs (System on a Chip), one of which had become quite popular in maker communities: the ESP8266, a tiny chip that included not only full WiFi support, but also a 32-bit RISC processor, running at 80 MHz, with lots of user memory. Among the numerous devices that were offered on maker websites was one from SparkFun called the ESP8266 Thing. The small package included not only the ESP8266, but also an onboard charger for a LIPO (Lithium Ion Polymer) battery connected to its power terminal. SparkFun was also selling a 9DOF (3-axis accelerometer, gyroscope, and magnetometer) which, by lucky coincidence rather than design, had drill holes for its signal connections in the exact same order as those of the ESP8266 Thing, meaning that the sensor could be soldered directly without the need for external wiring.
Mimn II puts all this together in a single device, as seen in the series of images displayed here. At top is the ESP8266 Thing, shown in bottom view, with a 6-pin female programming header soldered on top, and one half of a precision resistor network soldered on bottom. The next image shows a top view, with the 9DOF sensor board soldered in place.
The third image shows the assembled Mimn II alongside a compatible LIPO battery. These batteries, the same sort that are use in mobile phones, come in a variety of sizes and current/time ratings. The battery shown here is the one that seems best matched to Mimu II, providing about 4.5 hours of service per charge.
The last image is of the fully assembled Mimn II (with housing hand-made by Pamela Z), ready for use in live performance. Although Mimn, with suitable housing, can be attached in many places on the body, we’ve chosen the back of the hand in initial designs, since it has the most degrees of freedom of motion of any body part (when combined with the additional freedom of the elbow and shoulder joints).
Though there will be always technical issues with any design–WiFi, for example, can experience interference in environments where many signals are competing for bandwidth, though there are ways to deal with it–Mimn II has proved to be an outstanding and exciting new controller, capable of great nuance and power in live performance, especially with the strong support of the SensorPlay host software.
The video of Dohee Lee below provides a convincing view of the capabilities of the new instrument in the hands of a skilled and dynamic performer.
Mimn II Live
Improvisation, Dohee Lee (2015)
Dohee Lee performs an improvisation in voice and dance for the attendees of the Real Future Fair at San Francisco’s Palace of Fine Arts (former site of the Exploratorium) on November 6, 2015.
Dohee is wearing STM Mimn II wireless sensors on each hand (worn inside a black cloth covering) to control the processing her voice and other sounds using custom SensorPlay software by Donald Swearingen.