TOOLS:
– Chrome Experiment MUSIC LAB to generate MIDI and WAV
– Blender 3D
– AddRoutes addon
– iPhone and Face Cap (optional), you can try OpenPose (Windows only)

In this episode of Blender Sushi LIVE, I am giving a demonstration on how you can turn your sentence into a jingle as MIDI and wave, then do an improvisation animation potentially using your own face, and also maybe body motion capture animation.

The whole idea for this started with this browser tool called MUSIC LAB, I realized that it can output both MIDI file and WAVE. This is something that can be used in Blender! I mean it’s one way to get timing and melody at the same time.

I was also inspired actually by a TED talk by this African Lady, where she talks about how sentence in African language, often used as MUSIC then can be translated into dance motion and full performance can be made, it’s all about MUSIC and IMPROVISATION.

I like to do the same thing but using my own tool: Blender 3D! I know the fact that we can capture Face, Body performance, using the tools we have.

For the Melody thing, I am using my made up Jingle:
”I can Blend, a little. Together we Blend more!” — BS Motto
https://musiclab.chromeexperiments.com/Song-Maker/song/6012089459015680

You can use your imagination to turn your own sentence into melody. Just break the sentence, saying it slower or faster.

When ready, the whole data can be brought into Blender using ADD ROUTES addon. MIDI file can be imported into Blender using this addon. Just ensure you have the correct MIDI channel and drivers and record is set properly.

And luckily AddRoutes developer also talked to FaceCap app developer for iPhone and iPad, and then you can import face capture data into Blender, quite easily.

While for body capture, while it’s optional, that’s also something you can do using iPhone with bionical chip, or iPad Pro, more accurate with LIDAR. But if you have Kinect or other method, like using OpenPose, you can do so.

It’s truly quite interesting what you can do and create using your own home computer today! Maybe many people did not realize this yet. Hopefully you find this one interesting and useful.

Maybe in the near future all this can be a LIVE performance and your audience can see it in AR or VR. However, for now, if you could record your face performance and turning it into another character animation, that’s I think quite amazing in our timeline.

NOTE:
One thing that I forgot to mention is that you can also easily capture your mouse motion using Blender playback RECORD function.

ADD ROUTES ADDON:
http://www.jpfep.net/pages/addroutes/