Toonloop: animation for live performances
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Quessy, Alexandre
In this technical talk, the main author of Toonloop will describe how this free software works and why it has been designed with simplicity in mind. It displays image loops, and allows to apply GLSL shader effects on them. The use of the Clutter toolkit makes it possible to create rich user interfaces that are well suited for fullscreen video in the context of live performances and artistic installations. Thanks to GStreamer, Toonloop could grow and upport more operating system and video input types. Also, OSC and MIDI controls make the application open to new possibilities for interactive setups.
 Thanks everyone. Thanks, Alexander. So my name is Alexander Casey and I'll present you Toon Loop. This is a technical talk, so I'm going to go into details. And later this afternoon I have an artsy talk in which I will more rather present like the ideas behind Toon Loop and how it can be used for creative stuff. But now let's get our hands dirty and open the hood of Toon Loop. So Toon Loop is a live animation performance tool. I created it to allow artists to make performances in which they show their creation process as well as the results in order to make live performances like live shows. So the animation is shown. And also we can see the live input of the camera at the same time as the results. And the results is a looping clip. So that's why the name Toon Loop for looping cartoons. So it's a stop motion tool. And well, I guess I'm going to give you a little demo. All right. So this is Toon Loop. Actually I'm going to switch over to another camera. It's going to be more fun. Whoa. That was a clip that I saved previously. I've done that clip in France a couple of months ago. I'm going to start a new project. Project home. Okay, there you go. Fresh start. So what you can see on the left is the live video input of my little webcam. Indeed, this is not a very good camera. It's just a webcam. But it also works with DV cameras. So I'm going to press space bar a couple of times so that you guys can raise your hand or do stuff like that. And unfortunately I can't zoom right now. Okay, there I'm going to rotate this camera so that everyone gets... Oh my God. What happened? Okay, everyone gets dizzy. Okay, so what we can tell right now is the image on the right is looping. And so it's constantly looping. So... Yeah, looping is a word. And well, I can make this loop differently. Like this is a random loop. And this is going to be backward. Or actually if we press I, we're going to have a little bit more information about what's going on. So a little bit more info here. The direction we can see it's yo-yo. So this is going back and forth. You guys are quite funny, isn't it? And I'm going to make this full screen so that's more fun to look at. I think you should show your software far back. Yeah. So there's other directions. This is backward. So if you check it out, it's going backward, remember? Anyway, and there's other directions like drunk. Drunk is like, well, you go back and forth and then one step forward, two step backward, and then randomly like... That's inspired from a pure data object. And this is, yeah, this is back to forward. So, well, I'm going to continue to give you a little tour. You can use Toon Loop in a portrait. If you put your camera like this on the sideways, it's... Well, it works too. And there's other layouts. There's like... Well, there's some layouts in which you can put the two images one on top of each other. So that works also. So this... The info you can make the image faster, of course. This is 60 frames a second. Woo! Okay. And, well, there's some help. You can get some... Well, info about what key to press. Right now Toon Loop is only controlled by the keyboard. Mainly controlled by the keyboard. But there's other inputs and outputs. And I'm going to tell you more about this. Yes, question? Yeah, sorry. I might have missed it. But can you see the previous frame that you took as a reference so that you can... Yeah. Yeah, there's a onion skin, onion skinning feature. Okay, I'm going to create a new clip, actually, because that one was a little bit crazy. So I'm going to make some close shots of my eye, maybe something like that. So then if I press O, I can see an overlay of the previous frame I took so that I can position my eye, in this case, on top of the previous eye. And you can change the opacity of the onion skin. Yeah, this is a lot of fun. And, okay, I'm going to talk a little bit about the inputs, and then I'm going to talk a little bit about the outputs of Toon Loop. So for the inputs, there's video inputs, there's video for Linux. This is what I'm using right now with my little webcam. And it also supports DV cameras. DV cameras are the old cameras with a cassette, but they usually give you a pretty decent picture quality. And so that you can zoom, you can change the shutter speed and stuff, or like the iris, the opening of the iris. So that's a lot better. That's what I use for performances myself, DV camera. It also supports X11 input, which gives you the live screenshots of your desktop. That can be handy for presentations or like screencasts. I don't know. I never used it myself. It was just easy to implement, so I did it. And there's a test, G-streamer test picture, but that's not quite useful. And so that's for the video input. And right now it supports only four to three ratio images. But I'm planning to support more ratios, more image size ratios soon, such as 16 to 9 and stuff. That should be trivial to implement, really. Just that I'm... I don't know. And I'm also planning to support HDV. I'm almost there. HDV gives you like a very large images, 1,400 per 1,080. And also I'm planning to support G-Photo 2 input, but that's for later. That's a little bit of work, so I need help for that if anyone is interested. Yes? I've pointed you to stop motion before, like this other animation software. I presented in a lightning talk, I think last time here in Montreal. When it comes to supported input things, they have an interesting capability in that they actually support using just shell scripts to pull in images so that the user can just configure like a command line command for things like capturing from, say, single frame from HDV or digital camera or some alternate imaging source. It's just kind of an interesting option to have in addition to hard-coded native capabilities. That's very interesting. Thanks for the suggestion. I was actually planning to do this, like at least support like some network messages that would like allow you to import some image from the disk. So that's pretty much what you mean. And yeah, I was planning to do it, but I didn't thought about all those other input types that we would open. So thanks for the suggestion. I'll do it today, like tonight. No, no kidding. It's going to take a few days. And, well, talking of OSC or of network protocols and stuff, ToonLoop can be controlled using OSC protocol. So OSC protocol is a network protocol. It's called, it means open sound control. So that's intended to be used with musical software, like audio software. And it's rather simple. Excuse me a minute. Yeah, this is the OSC interface of ToonLoop. So it supports messages such as grab a frame, change the FPS, move the playhead to a given direction. You can, of course, save the clip. You can also save the project to ToonLoop has state saving. So all your images are saved as JPEG images on your hard drive. And so, yeah, you can add, remove frames, change the playback direction. So pretty much everything you can do with the user interface, You can do it with open sound control messages. And just to, just one thing I didn't mention is the image size For the input can be variable. You can specify, like, whatever image size. So there's actually v4l2 cameras that have, that provide, like, quite large images. So that can be quite useful for large images. For input, ToonLoop also provides a MIDI interface. And I'm going to show you a little, the MIDI script. Yeah, so this is an XML file that specify each, what, each note on a keyboard. Because MIDI stands for musical instrument digital interface. So it's for pretty much a piano or other musical instruments. So you can control ToonLoop with musical instruments. And that's quite handy for live performances. Because I created ToonLoop to make shows. So these are rules, MIDI rules. Each note is, in this case, hooked to selecting a different clip. So I can have, on my keyboard, I have, like, 40 something notes. So each note selects a different clip. And also the controls, you know, those knobs on the keyboard can apply effects to, and change the value of effect given a certain range. Like this guy saturation from zero to two. So two means 200%. And I have a little, try to show you this quickly. I have a little example PD patch. Pure data is a little software for coding with graphs and stuff like this. And just need to select my MIDI input. Okay, so here's my clip. I should be able to change the miscellaneous stuff like saturation. No, it's not working at all. Oh, I need to restart ToonLoop, sorry. Because I didn't use the proper flags. Yeah, right now the interface is only, like, for setting up your configuration options, you need to do it manually on the common line. So that's quite annoying. I'm planning an assistant. With launch ToonLoop with no common line option, should display a little assistant in SVG buttons in all the interfaces in Clutter. And I use Clutter GST to, while display the video in Clutter. So yeah, let's make another clip. What can I do? No, that's not nice. Okay, I'm going to do something like this. Okay. And this. Okay, so now I'm going to simulate sending MIDI notes. And that should change the saturation of the image here you can see. So this is totally grayscale. This is, like, 200% saturation. And that's in the middle. There's also weird effects like additive crossfade. That's quite weird. But yeah. And this is all done through either Clutter opacity stuff or shaders, GLSL shaders and, yeah, I'm changing the contrast here. And you can change the life feed opacity. Well, anyway. And select a different clip. Here I'm choosing clip number two, which is empty. Now back to clip number one. So it's a lot of fun. So I'm planning to add plugins so that you can write more effects. Okay, I have no more time. So I also have an info window, which is a separate window. And there's a lot of path for the future features, but I have no more time. So I guess I'll tell you more about this this afternoon in my other talk. I have one extra talk this morning, so it's getting late. We are delayed so in 20 minutes. I think questions we can keep it for this afternoon. And we have one last, there is one last talk. Alright. Thanks a lot.
 Thanks everyone. Thanks, Alexander.  So my name is Alexander Casey and I'll present you Toon Loop.  This is a technical talk, so I'm going to go into details.  And later this afternoon I have an artsy talk in which I will  more rather present like the ideas behind Toon Loop and how it  can be used for creative stuff. But now let's get our hands dirty  and open the hood of Toon Loop.  So Toon Loop is a live animation performance tool.  I created it to allow artists to make performances in which they  show their creation process as well as the results in order to make  live performances like live shows. So the animation is shown.  And also we can see the live input of the camera at the same time  as the results. And the results is a looping clip.  So that's why the name Toon Loop for looping cartoons.  So it's a stop motion tool.  And well, I guess I'm going to give you a little demo.  All right. So this is Toon Loop.  Actually I'm going to switch over to another camera.  It's going to be more fun.  Whoa. That was a clip that I saved previously.  I've done that clip in France a couple of months ago.  I'm going to start a new project.  Project home.  Okay, there you go.  Fresh start. So what you can see on the left is the live video  input of my little webcam.  Indeed, this is not a very good camera.  It's just a webcam.  But it also works with DV cameras.  So I'm going to press space bar a couple of times so that you guys  can raise your hand or do stuff like that.  And unfortunately I can't zoom right now.  Okay, there I'm going to rotate this camera so that everyone gets...  Oh my God. What happened?  Okay, everyone gets dizzy.  Okay, so what we can tell right now is the image on the right is looping.  And so it's constantly looping.  So...  Yeah, looping is a word.  And well, I can make this loop differently.  Like this is a random loop.  And this is going to be backward.  Or actually if we press I, we're going to have a little bit more  information about what's going on.  So a little bit more info here.  The direction we can see it's yo-yo.  So this is going back and forth.  You guys are quite funny, isn't it?  And I'm going to make this full screen so that's more fun to look at.  I think you should show your software far back.  Yeah.  So there's other directions.  This is backward.  So if you check it out, it's going backward, remember?  Anyway, and there's other directions like drunk.  Drunk is like, well, you go back and forth and then one step forward,  two step backward, and then randomly like...  That's inspired from a pure data object.  And this is, yeah, this is back to forward.  So, well, I'm going to continue to give you a little tour.  You can use Toon Loop in a portrait.  If you put your camera like this on the sideways, it's...  Well, it works too.  And there's other layouts.  There's like...  Well, there's some layouts in which you can put the two images one on top of each other.  So that works also.  So this...  The info you can make the image faster, of course.  This is 60 frames a second.  Woo!  Okay.  And, well, there's some help.  You can get some...  Well, info about what key to press.  Right now Toon Loop is only controlled by the keyboard.  Mainly controlled by the keyboard.  But there's other inputs and outputs.  And I'm going to tell you more about this.  Yes, question?  Yeah, sorry. I might have missed it.  But can you see the previous frame that you took as a reference so that you can...  Yeah.  Yeah, there's a onion skin, onion skinning feature.  Okay, I'm going to create a new clip, actually,  because that one was a little bit crazy.  So I'm going to make some close shots of my eye, maybe something like that.  So then if I press O, I can see an overlay of the previous frame I took  so that I can position my eye, in this case, on top of the previous eye.  And you can change the opacity of the onion skin.  Yeah, this is a lot of fun.  And, okay, I'm going to talk a little bit about the inputs,  and then I'm going to talk a little bit about the outputs of Toon Loop.  So for the inputs, there's video inputs, there's video for Linux.  This is what I'm using right now with my little webcam.  And it also supports DV cameras.  DV cameras are the old cameras with a cassette,  but they usually give you a pretty decent picture quality.  And so that you can zoom, you can change the shutter speed and stuff,  or like the iris, the opening of the iris.  So that's a lot better.  That's what I use for performances myself, DV camera.  It also supports X11 input, which gives you the live screenshots of your desktop.  That can be handy for presentations or like screencasts.  I don't know. I never used it myself.  It was just easy to implement, so I did it.  And there's a test, G-streamer test picture, but that's not quite useful.  And so that's for the video input.  And right now it supports only four to three ratio images.  But I'm planning to support more ratios, more image size ratios soon,  such as 16 to 9 and stuff.  That should be trivial to implement, really.  Just that I'm... I don't know.  And I'm also planning to support HDV. I'm almost there.  HDV gives you like a very large images, 1,400 per 1,080.  And also I'm planning to support G-Photo 2 input, but that's for later.  That's a little bit of work, so I need help for that if anyone is interested.  Yes?  I've pointed you to stop motion before, like this other animation software.  I presented in a lightning talk, I think last time here in Montreal.  When it comes to supported input things, they have an interesting capability  in that they actually support using just shell scripts to pull in images  so that the user can just configure like a command line command  for things like capturing from, say, single frame from HDV or digital camera  or some alternate imaging source.  It's just kind of an interesting option to have in addition to hard-coded native capabilities.  That's very interesting. Thanks for the suggestion.  I was actually planning to do this, like at least support like some network messages  that would like allow you to import some image from the disk.  So that's pretty much what you mean.  And yeah, I was planning to do it, but I didn't thought about all those other input types  that we would open.  So thanks for the suggestion. I'll do it today, like tonight.  No, no kidding. It's going to take a few days.  And, well, talking of OSC or of network protocols and stuff,  ToonLoop can be controlled using OSC protocol.  So OSC protocol is a network protocol.  It's called, it means open sound control.  So that's intended to be used with musical software, like audio software.  And it's rather simple. Excuse me a minute.  Yeah, this is the OSC interface of ToonLoop.  So it supports messages such as grab a frame, change the FPS,  move the playhead to a given direction.  You can, of course, save the clip.  You can also save the project to ToonLoop has state saving.  So all your images are saved as JPEG images on your hard drive.  And so, yeah, you can add, remove frames, change the playback direction.  So pretty much everything you can do with the user interface,  You can do it with open sound control messages.  And just to, just one thing I didn't mention is the image size  For the input can be variable. You can specify, like, whatever image size.  So there's actually v4l2 cameras that have, that provide, like,  quite large images. So that can be quite useful for large images.  For input, ToonLoop also provides a MIDI interface.  And I'm going to show you a little, the MIDI script.  Yeah, so this is an XML file that specify each, what, each note on a keyboard.  Because MIDI stands for musical instrument digital interface.  So it's for pretty much a piano or other musical instruments.  So you can control ToonLoop with musical instruments.  And that's quite handy for live performances.  Because I created ToonLoop to make shows.  So these are rules, MIDI rules. Each note is, in this case,  hooked to selecting a different clip. So I can have, on my keyboard,  I have, like, 40 something notes. So each note selects a different clip.  And also the controls, you know, those knobs on the keyboard can  apply effects to, and change the value of effect given a certain range.  Like this guy saturation from zero to two. So two means 200%.  And I have a little, try to show you this quickly.  I have a little example PD patch.  Pure data is a little software for coding with graphs and stuff like this.  And just need to select my MIDI input.  Okay, so here's my clip. I should be able to change the  miscellaneous stuff like saturation. No, it's not working at all.  Oh, I need to restart ToonLoop, sorry.  Because I didn't use the proper flags.  Yeah, right now the interface is only, like, for setting up your  configuration options, you need to do it manually on the common line.  So that's quite annoying. I'm planning an assistant.  With launch ToonLoop with no common line option, should display a little  assistant in SVG buttons in all the interfaces in Clutter.  And I use Clutter GST to, while display the video in Clutter.  So yeah, let's make another clip. What can I do?  No, that's not nice. Okay, I'm going to do something like this.  Okay.  And this.  Okay, so now I'm going to simulate sending MIDI notes.  And that should change the saturation of the image here you can see.  So this is totally grayscale. This is, like, 200% saturation.  And that's in the middle. There's also weird effects like additive  crossfade. That's quite weird.  But yeah. And this is all done through either Clutter  opacity stuff or shaders, GLSL shaders and, yeah, I'm changing the contrast here.  And you can change the life feed opacity.  Well, anyway. And select a different clip.  Here I'm choosing clip number two, which is empty.  Now back to clip number one.  So it's a lot of fun.  So I'm planning to add plugins so that you can write more effects.  Okay, I have no more time. So I also have an info window, which is  a separate window. And there's a lot of path for the future  features, but I have no more time. So I guess I'll tell you more about this  this afternoon in my other talk.  I have one extra talk this morning, so it's getting late.  We are delayed so in 20 minutes.  I think questions we can keep it for this afternoon.  And we have one last, there is one last talk.  Alright. Thanks a lot.
Toonloop__animation_for_live_performances_0_2171.png Toonloop__animation_for_live_performances_2172_3275.png Toonloop__animation_for_live_performances_3276_3527.png Toonloop__animation_for_live_performances_3528_4247.png Toonloop__animation_for_live_performances_4248_4379.png Toonloop__animation_for_live_performances_4380_4679.png Toonloop__animation_for_live_performances_4680_4823.png Toonloop__animation_for_live_performances_4824_4967.png Toonloop__animation_for_live_performances_4968_5651.png Toonloop__animation_for_live_performances_5652_6035.png Toonloop__animation_for_live_performances_6036_6527.png Toonloop__animation_for_live_performances_6528_6599.png Toonloop__animation_for_live_performances_6600_7415.png Toonloop__animation_for_live_performances_7416_7823.png Toonloop__animation_for_live_performances_7824_8399.png Toonloop__animation_for_live_performances_8400_8627.png Toonloop__animation_for_live_performances_8628_8831.png Toonloop__animation_for_live_performances_8832_9695.png Toonloop__animation_for_live_performances_9696_10607.png Toonloop__animation_for_live_performances_10608_11063.png Toonloop__animation_for_live_performances_11064_11327.png Toonloop__animation_for_live_performances_11328_14063.png Toonloop__animation_for_live_performances_14064_24507.png