YAMAHA Demonstrates Real-time VOCALOID Keyboard

At interaction 2012, a symposium focused on novel interaction techniques organized by the Information Processing Society of Japan, junior employees at YAMAHA demonstrated a keyboard that allowed real-time synthesized vocal performance using the VOCALOID engine. The interactive demonstration received one of the five interactive paper awards at the symposium and a video describing this keyboard has been uploaded to Nico Nico Douga.

According to a presentation video, this device was constructed by these employees in their spare time, the motivation being that they wanted to perform VOCALOID as an instrument live. Images of the first prototype show a CBX-K1 (a 37-key entry-level MIDI keyboard introduced back in the 90s) attached to an Arduino microcontroller (which presumably also runs the LED display) and a “Vocaloid Board”. The second prototype, the one demonstrated in the video, had a more polished look. VY1 was the VOCALOID sound bank used for the prototype.

As for playing, the left hand selects the consonant and vowel sound for the syllable as well as whether the consonant was voiced; the selected kana is then appended to an LED matrix display above the key bed. The right hand uses the key bed to control the pitch and duration of the note. Furthermore, there are various parameter tweaks possible using the knobs on the keyboard. However, it is not likely that one can perform as much tweaking as usually needed for top-quality VOCALOID songs, as that requires too many parameters to be adjusted in real-time.

A separate video (English version) taken by DigInfo TV at the interactive session shows that at least GEN (gender factor or formant location) can be tweaked. The most obvious reason given for the development of this device was that VOCALOID producers have to currently input the notes and play back the audio instead of doing it real-time. The person demonstrating the device, Shota Kagami, also mentioned that they’ve heard from a lot of female users that it was really hard to make songs after buying music production software; this device would allow anyone who could play a keyboard to perform with VOCALOID. According to the narration from DigInfo, “when several keyboard players evaluated the system, they could enter content in Japanese after about three hours practice, and they became able to perform simple nursery rhymes smoothly.” The video also says that the VOCALOID hardware can be provided to interested companies.

There were a total of 18 oral presentations and 154 interactive presentations at this conference. One best paper award was chosen by the program committee out of all the entries and 19 interactive paper award finalists were also chosen, of which 5 received the award, including the VOCALOID Keyboard’s entry. The best paper award went to interactive song analysis web application Songle, whose front page happens to be loaded with VOCALOID songs at the moment.

Currently there are no plans to productize this prototype, the creators admitted somewhat sheepishly in the video.


5 thoughts on “YAMAHA Demonstrates Real-time VOCALOID Keyboard”

  1. TOP 3 on Songle: Kimi no te boku no te, Double Lariat and SPiCA
    wwww
    I think there’s two possible explanations:
    1) Vocaloid is extremely strong on the Internet and technology aware circles, far more than any other kind of music.
    2) Vocaloid is the only music that allows one to upload an MP3 without an asshole suing for copyright infringement (well except SEGA of course 😉 )
    Probably a mix of the two.

    As for the keyboard, nice project but I don’t see any practical use musically speaking. On the other hand the same stuff for Voiceroid RAP could do it 😀

    Edit: There’s something wrong with the comment system, all my carriage returns are removed when doing my first post.

    1. Yeah, it looks really painful to play, although I still think they should hook this up to MikuFlick…  I’m sure that game has trained some users in the way of the flick input system >_>;  What would be also neat would be to hook a speech recognition system into it, so you can have someone talk/sing in the notes and have the VOCALOID board resynthesize it.

  2. When telling people about VOCALOID, I make the joke that the only 2 things it cannot do are Improvise and Perform Live (and then I show them the BluRays of the concerts and brightly remark, “oh, yeah — I guess that they CAN perform live”).

    With this — IMPROVISATION!!!

    Yes, it needs some refining. The left-hand syllable-input keys need a better design. Perhaps a look at the court-reporting ‘Stenograph” machine may help them?

    I couldn’t help but think of the Accordion when watching these videos. The Accordion? Yeah: a keyboard on the right hand, a series of push-buttons on the left (for chords on an accordion, but could be made to call up syllables for Vocaloid), and a bellows to control volume and expression.

  3. This should be difficult 2 let them sing fast words ^^
    honestly i don´t think its worth if they don´t impro the vocal things… so slow… not voclaoid style ya know ^^

Leave a Reply

Your email address will not be published. Required fields are marked *