MIDI Controlled Animation in After Effects – Part 1

MIDI Controlled Animation in After Effects – Part 1


– Yo! Joey here at School of Motion and welcome
to day five of 30 Days of After Effects. Today, we’re going to get experimental and
talk about how to get MIDI information into After Effects to create animation driven by
that data. If you don’t know what MIDI is, don’t worry;
I’ll explain that, too. We’re gonna be in and out of After Effects
today; because we’re going to have to use another program, Logic, to be specific, to
actually create the MIDI information. Now hopefully, this is really interesting
and gives you some cool ideas about unique ways to use After Effects. Don’t forget to sign up for a free student
account so you can grab the project files, example MIDI from this lesson, as well as
assets from any other lesson on the site. All right, let’s check this out. So, let’s start out this first video by doing
a quick primer on what MIDI is for those of you that don’t have any experience with it;
and for those of you that do, who knows? Maybe you’ll learn something. This app that I’m in right here is called
Logic and there’s a lot of different programs out there that you can work with audio. You got Pro Tools, Logic, Cube Ace, Sonar,
I mean, there’s a lot. If you have a Mac, it comes with Garage Band,
which can do MIDI. I just like to use Logic. First, let me just show you what MIDI is. If I hit ‘play’… It just sounds like bad 80s music. So, what MIDI is, it’s a way of storing musical
information, okay? This song, and I just downloaded this off
a free MIDI site, it’s got six instruments here and each instrument has it’s own track. If I click on each one, you can see that there
notes assigned to each one. When you go into a recording studio, typically
you have microphones and you record the audio, and once it’s recorded, you can edit it and
do things to it. Typically, that’s what you got. Whatever you play, that’s what you get. With MIDI it doesn’t work that way. With this track selected, I’m just gonna hit
‘play’ and you can hear what it’s doing and you’ll be able to see exactly what all these
notes mean. When you go to a recording studio and you
play a piano in front of a microphone, you’ve recorded an actual audio file; but, when you
record MIDI, all you’re recording is the information about when you hit each key on the keyboard,
how hard you hit the key, and it records just a bunch of data. What’s great about that is then you can change
it. If I take these two notes, I can move them
around, right? I have made the song even worse than it already
is. So, that’s how MIDI works and you can see
that it works on a grid system. This is great if you don’t actually know how
to play an instrument, you can still make music really simply. Let’s get rid of some notes here. You can just program in whatever you want
just by adding notes. There’s a note. So, that’s how MIDI works. If you actually play piano, if you have a
keyboard, you can really easily record your own MIDI information. As an animator, this was intriguing to me
because especially when you’re doing animations that have to be timed out to music, or you
want things to sync up to a beat, or something like that, it can be very tedious. There’s some ways to do it in After Effects. There’s plug-ins that can try to figure out
when the kick drum’s being hit, when the snare drum’s being hit and read the audio file,
but if you could actually get this information into After Effects, that would be super useful. So, first thing I want to do is I want to
show you guys how I got some MIDI information that I thought would be useful. I play drums, so I use an interface called
an Octopad and I’ll link to a little more information about that if any of you guys
play drums and you’re curious about it. What I have here is I have a track and I’ve
got a plug-in that I use when I actually play drums in Logic. It’s called Superior Drummer. It’s this really cool plug-in that works for
a lot of programs and it lets you use MIDI to control this very realistic drum set sound. I have that set up and now I’m gonna walk
over and I’m gonna sit down on Octopad and I’m just gonna hit record. Cool. So, that shows you exactly how MIDI works. As I’m playing the drums, Logic is just recording
on each little track here exactly what I played. The different colors are just showing you
different velocities, right? Okay. You can see that the red hits are the strongest
ones and the green are the weaker ones. What’s really great, especially if you’re
a drummer, is you can select all these notes and you can fix them real quick. Now, everything’s gonna be perfectly in time. The ending’s not in time, but what are you
gonna do? Anyway, there’s our MIDI information, okay? What I wanna do is I wanna be able to use
this in a way that’s useful in After Effects. I’m gonna rerecord something and I’m gonna
make something very, very simple. Then I’m gonna show you how to get that into
After Effects, and I’m gonna show you the basics of how to start working with that information. In part two of this tutorial, I’m gonna get
into some pretty crazy expressions which will let you really, really control stuff. Let me hop back over to the kit and I’ll just
do something really, really simple. Cool. So, that was much simpler and this is gonna
be easier to work with in After Effects. Let’s just make sure that it sounds the way
we want. Great, okay. What I’m looking for here I want a little
bit more variation in the dynamics of the playing. You can see a lot of these hits are red, but
you’ve got a few of them in there that are different colors. What I’m gonna do is I’m just gonna select
some of these and I’m just going to change the velocity of them, which is gonna make
them a little bit different. This will make it a little bit easier to see
what’s going in After Effects. Maybe I’ll make these two a little softer,
too. See, this is what’s great about MIDI. Especially if you’re a musician, you really
can just tweak the crap out of this stuff. Maybe I’ll make these two a little softer. There we go. This one can be a little softer. Cool. And then, we’ll make these a little bit softer,
too, so it’s more of a build up at the end. There we go. There we go. Cool. All right. So, first thing we need to do is export an
audio file of this. And I’ll show you how to do it in Logic. In Garage Band, it’s gonna be a slightly different
set up, but all you need to do is use the internet and figure out how you export an
audio file out of your digital audio workstation program. So, I’m gonna just select this region and
I’m going to bounce this track and we’re gonna call this… Let me make a new folder here and we’ll call
this ‘demo.’ this is gonna be my ‘Toms Audio.’ Cool. All right, now you guys probably saw that
I trimmed this little layer. There was some extra stuff at the beginning
that I didn’t need. I trimmed it so it was just the part of the
audio that I needed. If I just export this as is from Logic, what
I’ve found is that it actually exports even the notes that I thought I erased. What I need to first do is come down here
and in my edit menu just say ‘delete MIDI events outside the region boundaries.’ again, this is how Logic works and I’m on
the newest version of Logic. If you’re using Pro Tools or something, you
just need to look up exactly how to do that. Then, we’re just gonna go ‘file, export, selection
as MIDI file.’ Actually this error message is what I was
hoping wouldn’t happen. It’s saying ‘there are some events before
position one one one one.’ So, it’s actually saying that there are MIDI
events that it’s sensing that are happening before the beginning of the song. Actually, if I zoom in here I think I see
what it’s doing. You see how this first hit is actually a little
bit before the beat? There we go. I just moved it. Let’s see if I try to export this now if it
works. ‘Export, selection as MIDI file.’ There we go. We’re gonna call this ‘Demo, Toms Audio Midi.’ There we go. Now, we have a MIDI file. Now, let’s hop into After Effects and let’s
talk about how to get this information into After Effects. Ha! I lied! We’re not in After Effects. We’re actually in a web browser and why is
that? Well, there’s not a built-in way in After
Effects to get MIDI information in. You have to use a plug in and this is the
only one I’ve found. It works great. I’ve actually mentioned this site in another
one of these 30 Days of After Effects tutorials. This guy has made a bunch of free plug ins
and scripts. In one of them is a MIDI importer, all right? So, you go to his website, and you download
it. It’s a script, so you need to follow the instructions
to actually install it in the right place. Then, once you install it, it shows up in
After Effects in your window menu right here. ‘o-m-underscore-midi’ and here it is, okay? First thing I need to do is import that Tom
Tom audio. So, let’s go into the ‘demo’ folder, let’s
grab the ‘Toms Audio’, let’s throw it down there, and let’s just preview that real quick. Great. There it is. Works perfectly. Your typical workflow to sync things to this
would be open up the the audio wave form, which, if you guys don’t know this, you double
tap ‘l’ on an audio layer- Brings up the wave form- and maybe adding some markers wherever
there’s a hit. Now that we a MIDI, we have much more powerful
tool, so let’s import the MIDI. What we’re gonna do is make sure that our
‘o-m-underscore-MIDI’ script is open, and we’re gonna click on this little tiny icon
there. This is where we tell it which MIDI file. So, I navigate to the folder, there’s the
MIDI file, hit ‘OK.’ Now, this script it’s not a professional thing
that you buy, so sometimes it’s a little bit finicky, okay? I’ve found that what works best is to make
sure you have the correct COMP open, and select some layer in there. It just helps tell this script that this is
the COMP I want you to put the MIDI information into. When you hit apply, it should work. There we go. All right, now here’s what it did: it created
a ‘null’ and I usually just immediately rename that ‘null’ ‘MIDI.’ You can see that it’s put a bunch of effects
on here. Now, the effect that it put on there is actually
my good buddy, the expression slider. It’s renamed them ‘channel zero’, ‘channel
zero’, ‘channel zero’, ‘channel nine’, ‘channel nine’, ‘channel nine.’ If I click on this ‘null’ and I hit ‘u’, let’s
take a look at what we got. We’ve got a bunch of key frames in there,
okay? Now, watch. I’m gonna play the audio and I want you to
watch this play head. isn’t that cool? These key frames line up to the audio. Now, we need to know how to interpret them,
okay? Here’s one of the issues that I have found
with this. I’m not sure if there’s a work around and
if someone figures this out, please tell me. We work in video, we work in frames. This COMP, for example, is 24 frames a second,
but if you listen to the Tom hits at the end, they’re happening pretty quickly and if I
go back to Logic and I zoom out so we can really see all my hits, you can see that there’s,
basically, nine hits here. If we come into After Effects and we look
at the very last little bit here, there’s three, four, five… I can’t tell. That looks like there might be two in there,
kind of snuck in. And what happens is the plug-in actually tries
to put the notes in on individual frames for you. Sometimes, it just gets screwed up. You see how there’s two notes right next to
each other? When you play quickly when notes are very
close together, this script, it doesn’t do a great job of putting things exactly where
they should be. For most cases, close enough and in the way
that I used it in the demo with the most amazing song ever written, Africa, I deliberately
kept my playing pretty simple knowing that this is how it’s gonna get used. Here, let’s just play this one more time. All right, if you listen to the beginning,
there’s three hits right in a row and we’re seeing here’s three key frames, but then here’s
six key frames. Then, over here we’ve got this channel, ‘channel
zero velocity’ that has nothing in it. So, the first thing I like to do is clean
up the MIDI information a little bit and this channel zero has nothing on it. So, I’m gonna delete all three of these channel
zeroes that it’s giving me. Now, all I have is channel nine. Now, in MIDI, there’s a lot of different things
that can be measured. Obviously, the timing when I actually hit
the drum is measured, but also how hard I hit is measured. That’s the velocity. If I click on this and I go into my graph
editor, you can actually see… You can actually see that the velocity matches
up perfectly to my audio. Now, the pitch on this isn’t gonna change
because I’m playing one drum. If you had a piano instrument, though then
you would have the same three channels. You’d have the duration and the duration of
a drum hit is generally just an instantaneous thing. That’s why when I click this, there’s to a
lot going on here. The velocity is changing, but when you’re
playing a piano, you actually might hold a note for a second and then let go. This will actually have more information in
it for a piano than it does for a drum. You’ll also have velocity information and
then, you’ll also have pitch information for the piano, or for any instrument that has
notes. This chart will move up and down and tell
you the pitch of the instrument, which you could do some cool stuff with, too. For now, let’s keep it kind of simple. The drum doesn’t have a pitch, so I’m also
gonna delete that. So now, the way that this works is whenever
I hit a drum and it lands on a frame, I get the key frame on the duration. Here’s our first problem with the duration. You can see there’s four key frames. There’s actually five hits. It sort of didn’t register the first one or
something, but on velocity it did register all of them. What happens with velocity is it actually
puts an extra key frame at the end. That extra key frame levels it back out. You see that? You can delete that or you can just ignore
it. You could go through and just delete each
of these key frames if you didn’t wanna use that. Until you actually start using the data, you’re
not really sure how it’s gonna work. Just for this example, let’s just leave this
as is and let’s start doing something. Here’s a quick and easy thing and just so
you guys know, you’re gonna have to use Expressions to do this. There’s no other way to really make this work. Again, if you’ve watched any of my After Effects
tutorials, you know I love Expressions. It’s kind of my thing. I want you guys to get more comfortable with
them. All I’m gonna do, is I’m gonna make a little
circle here and I’m just gonna tie the scale of the circle to this. Let’s look at the velocity here. The highest it gets is 127, so if I go here,
if I go forward one frame, you can see that the value is 127. The reason that number is 127 is because that’s
just the scale that MIDI works at. If we go back into Logic and take a look at
this… Let me open up this little guy down here. I’m gonna change this graph to ‘Note Velocity.’ So, now you can see with all the notes that
there’s a corresponding dip in this velocity. If I hold my mouse over this note, it says
velocity 127. If I hold it over this note, velocity 80. So, this is the information that’s going back
into After Effects. So, that’s the maximum, 127 and the minimum
is zero, right? If I’m not hitting anything, then the velocity
is zero. So, what I wanna do is I wanna put an Expression
on the scale of this circle. What I want the scale to do is to look at
this velocity slider and I want it to scale from 100% when I’m not hitting the drum, maybe
up to 200% when I’m hitting it as hard as I can. And when I’m hitting it in between, I want
it to be somewhere in the middle. So, the way we do that is we first need to
set up a variable just to make this easier to read. The value I want this Expression to look at
is this slider, channel nine velocity. I’m just gonna make a quick variable: Val
equals, val is just short for value, equals this. We always have to add a semicolon at the end. Then I’m gonna use the linear Expression. Linear Expression in After Effects is awesome. It lets you take one number that’s changing
over time and map it to a different number over time. If you had a value, for example, the velocity
of a MIDI note going from zero to 127, but I wanna map the scale of this circle to go
from 100 to 200, that’s exactly what linear does. The way it works is you type in linear. In parenthesis, you have to give it five arguments. The first one is what value am I looking at. Well, we’re looking at that variable we just
made called Val. Then, the next two arguments are the minimum
and the maximum number that is expected to come out of this thing you’re measuring. So, we know the minimum is zero and the maximum
is 127. Then, the next two numbers are what are we
going to map to these two numbers. So, when this value is zero what should the
result be? Well, when I’m not hitting a drum, I want
the scale to remain at 100, so I’m just gonna type in 100. And when I’m hitting the drum as hard as I
can, I want it to go up to 200. And that’s it. Okay. Oh boy, we’ve got an error. Yes, here we go. The error message was telling me that this
expression won’t work because- and I’m glad you guys saw that because you see, even when
you use expressions all the time, you still screw them up every time. So, scale is expecting an X and a Y number,
and I’m only returning one number here. I’m actually gonna say this: ‘S equals linear.’ Now, I’m setting up one more variable, ‘S’,
which is gonna store the value that comes out of that linear expression. Now, I can return instead of one number, I
can return an X and a Y. The way you do that is with opening bracket
and then the first number, S, comma, the second number, S. close the brackets. So, all I’m doing is I’m telling After Effects
I want this X number and this Y number to be the same. Both of the numbers we’ll get will come from
this expression here. If you are very unfamiliar with expressions,
maybe you should watch the Intro to Expressions in After Effects video. So, now this expression should work. Now, let’s do a quick RAM preview. Let me set this to half so it goes a little
quicker. Great. Okay. Let’s make this number a little bit bigger,
too. Let’s make this 500 and then let’s make this
number 50, so we can get a lot more variation out of it. So, that’s it in a nutshell how to get MIDI
information in to After Effects and how to begin using it. I wanna show you guys one other thing I ran
into while I was preparing for this. I’m not exactly sure why, but it might be
useful for you guys to know about this. I recorded a different piece of audio here. Let me grab it. It’s this piece of audio which is just… Let me hit ‘l-l’ here. This is a snare drum. Kinda just a solo piece here, right? What’s different about this from the Tom thing
I did, is I’m playing a lot faster, right? There’s just a lot more notes to it. This is also longer. This is about 22 seconds here. Here’s what happened when I inputted the MIDI
information. That MIDI information is here. Okay, snare two. Then, let me hit apply. That brings it in. What I noticed was I got MIDI information
that went way out past my actual audio file. I thought that was weird. So, here, let me delete this channel that
has nothing on it and the pitch channel which we don’t need, and let’s look at the velocity
channel. I’m gonna zoom in a little bit here. You notice right away that it doesn’t seem
to be matching up to the audio at all. I thought that’s really strange and I didn’t
really understand why there were so many extra key frames here. So, what I did was I moved this layer over
and I’m just gonna grab… I’m gonna, basically, click on this slider
and then hold shift and click on this just to select every key frame, and then while
I’m holding option, I’m gonna grab the very last key frame and move it to the left. What I’m doing is I’m scaling all of these
key frames and what I wanna do is try to line up this last key frame with this last snare
hit right here. So, I’m basically just going in… If you go into the graph editor, and you select
both of these… So, you go to the graph editor, tap this twice. Then, hold shift and tap this and it will
select everything and if you have this little button checked, the transform box, you can
actually scale them and scale them in between frames so you can really line that last little
hit up. All I’ve done is I’ve taken my MIDI key frames
and scaled them. Now, let’s play this. So, you can see now, it lines up perfectly
and I actually did check it and it lines up all the way through to the end. So, if you have a ton of MIDI notes and it’s
a longer piece just know you may have to scale the key frames, okay? This is actually… This is pretty great. This shows you exactly why using MIDI information
could be so useful, because look how many key frames there are and how much information
is here and if you had to hand key frame that, it would just suck. This is a way better way to do it. Then, just out of curiosity, why don’t I copy
this circle here and I’ll just paste it into this and we can see exactly how it reacts
to this piece of audio. So, I’m just gonna double tap ‘u’ to bring
up everything I’ve changed on my circle layer and this will bring up this expression. Here’s something that’s kind of cool. Because I named my null ‘midi’, in the expression
that’s actually what is was looking for. It was looking for a layer named ‘midi.’ it
just worked and I don’t have to change anything. Let’s do a quick RAM preview here and just
play a little bit of this. You can see that it’s working. Now, obviously it’s very jerky and it’s not
super useful yet. I mean, if you just wanted something really
flickery and cool and maybe you just wanted a layer to turn on and off instantly in time
with music, you can do a very simple expression like this and get that to work. If you wanna have things be triggered and
trigger particles and things like that, then it’s a little more tricky. So, in the next tutorial, that’s what we’re
gonna get into. Hopefully, this was a good foundation, a good
base of knowledge, so when we get into the more advanced stuff, it’ll make a little bit
more sense. Thank you guys, as always and I’ll see you
next time. Thank you so much for watching and make sure
you check out part two of this tutorial where we’ll actually get really deep into expressions
to make this stuff all work the way we want to. If you learned something valuable from this
video, please share it around. It really helps us spread the word and when
you do that, we really appreciate it. Also, don’t forget to sign up for a free student
account to access the project files for the lesson you just watched plus a whole bunch
of other goodies. Thank you so much. I’ll see you next time.

Rooster Teeth Animated Adventures – Twitch Chicken

Yesterday Jon And I streamed separately we were streaming on our own accounts and then people on my chat wereRead More Rooster Teeth Animated Adventures – Twitch Chicken

16 Replies to “MIDI Controlled Animation in After Effects – Part 1”

  1. Thanks for uploading this to YouTube!
    How would you go about a MIDI file created via a piano, imported using om_midi.jsx script, which has chords in a single frame? The graph editor for ch_0_pitch has more than one keyframe when you are playing say a triad…
    An expression for this would be brilliant and a BIG timesaver!

  2. the opening of this video is misleading as fuck, why would you pretend to be playing when you had to go in and manually place each frame in the timeline? don't waste people's time.

  3. I really appreciate your time on these tutorials! 🙂 But can I do this tutorial since I'm not on a Mac? I can't use Logic and therefore don't have the midi file…I only see an aif file. I was thinking of doing the AE Kick start course. Is this going to be a problem since I'm not on a Mac?

  4. I thought you would teach, how to map a midi controller to after affects, i cant believe why after effects doesnt have midi and osc support.

  5. Would there be any drawback to converting MIDI to audio files (individual tracks or stems) and then converting audio files to keyframes? Both are one click functions in Logic and AE. Thanks for your video!

  6. I downloaded the om_midi file and put it in ScriptUI Panels folder, but it is just a text file not a jsx one, so it doesn't show up under the window.
    Does somebody know how to work it well? Please help me.

Leave a Reply

Your email address will not be published. Required fields are marked *