At interaction 2012, a symposium focused on novel interaction techniques organized by the Information Processing Society of Japan, junior employees at YAMAHA demonstrated a keyboard that allowed real-time synthesized vocal performance using the VOCALOID engine. The interactive demonstration received one of the five interactive paper awards at the symposium and a video describing this keyboard has been uploaded to Nico Nico Douga.
According to a presentation video, this device was constructed by these employees in their spare time, the motivation being that they wanted to perform VOCALOID as an instrument live. Images of the first prototype show a CBX-K1 (a 37-key entry-level MIDI keyboard introduced back in the 90s) attached to an Arduino microcontroller (which presumably also runs the LED display) and a “Vocaloid Board”. The second prototype, the one demonstrated in the video, had a more polished look. VY1 was the VOCALOID sound bank used for the prototype.
As for playing, the left hand selects the consonant and vowel sound for the syllable as well as whether the consonant was voiced; the selected kana is then appended to an LED matrix display above the key bed. The right hand uses the key bed to control the pitch and duration of the note. Furthermore, there are various parameter tweaks possible using the knobs on the keyboard. However, it is not likely that one can perform as much tweaking as usually needed for top-quality VOCALOID songs, as that requires too many parameters to be adjusted in real-time.
A separate video (English version) taken by DigInfo TV at the interactive session shows that at least GEN (gender factor or formant location) can be tweaked. The most obvious reason given for the development of this device was that VOCALOID producers have to currently input the notes and play back the audio instead of doing it real-time. The person demonstrating the device, Shota Kagami, also mentioned that they’ve heard from a lot of female users that it was really hard to make songs after buying music production software; this device would allow anyone who could play a keyboard to perform with VOCALOID. According to the narration from DigInfo, “when several keyboard players evaluated the system, they could enter content in Japanese after about three hours practice, and they became able to perform simple nursery rhymes smoothly.” The video also says that the VOCALOID hardware can be provided to interested companies.
There were a total of 18 oral presentations and 154 interactive presentations at this conference. One best paper award was chosen by the program committee out of all the entries and 19 interactive paper award finalists were also chosen, of which 5 received the award, including the VOCALOID Keyboard’s entry. The best paper award went to interactive song analysis web application Songle, whose front page happens to be loaded with VOCALOID songs at the moment.
Currently there are no plans to productize this prototype, the creators admitted somewhat sheepishly in the video.