Python Profiling with Intel® VTune™ Amplifier
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Sobhee, Shailen
Python Profiling with Intel® VTune™ Amplifier Python has grown in both significance and popularity in the last years, especially in the field of high performance computing and machine learning. When it comes to performance, there are numerous ways of profiling and measuring code performance—with each analysis tool having its own strengths and weaknesses. In this talk, we will introduce a rich GUI application (Intel® VTune™ Amplifier) which can be used to analyze the runtime performance of one’s Python application, and fully understand where the performance bottlenecks are in one’s code. With this application, one may also analyze the call-stacks and get quick visual clues where one’s Python application is spending time or wasting CPU cycles
 I'm Shailen, I am a technical consulting engineer at Intel in the developer products division team and we are based in Munich, Germany. And today's focus will be performance analysis of Python applications. And yeah, we have to say it's of no denial that Python is getting a lot of traction, a lot of importance these days. And if you look at what our friends at CodeEval have published, indeed, Python has grown in popularity over the last years and in 2016, Python remains the number one most used language. And also what is more surprising is that Python remains the number one programming language in hiring demand. So it's a great skill to have in this decade to be proficient in Python. And when it comes to performance analysis, there are certain fields that are kind of driving the technologies of the future and technologies that are kind of really important right now. And these fields, I would say, would be mathematics and data science. And to get my facts straight, to get the numbers correct, I went to Stack Overflow, our favorite website where we have problems. And Stack Overflow shows me that indeed, Python is the most used language in the fields of mathematics and data science. Now you may think the math doesn't make sense. If you add all the percentage, it doesn't make up to 100. That's because out of those approximately 50,000 people who responded to the survey, they chose several languages. But most of them chose Python. Over 50% of this. So that's quite impressive. So math and data science, these fields actually drive high performance computing, or HPC, and other fields like artificial intelligence, machine learning, or deep learning. And Intel realizes that these fields are going to define the future. And so we have worked really hard to release a distribution of software, which we call the Intel distribution for Python. And it comes up out of the box with highly optimized sublibraries to allow you to develop high performance applications with Python. We made it super easy to use, super easy to install. Packages can be easily downloaded through Anaconda or even YAM, providing the RPMs. And our distribution of Python comes with highly optimized libraries like NumPy, SciPy, Techitlearn, which actually are the base leverages Intel Mkl, which is short for MathKernel library. Now in itself, Mkl, if you've never heard about it, if you've heard about it, it's made in assembly. It's super optimized. Mathematical routines have been designed to make the most out of the Intel architecture, how many cores you have, make use of the latest instruction set architecture, for instance, AVX, AVX 512, whatever you have out of the box, make the most of vectorization. So you don't have to worry about this by using the Intel distribution for Python. Now performance is really important. So how do we actually measure performance of a Python application? Enter Intel V2Namplifier. V2Namplifier is a code profiler. It is a profiler that allows you to know where are the performance problems in your software. It has been developed over many years, over 15 years, and it's still in development. We're getting a lot of improvements every day. Our engineers are working hard, and over the last four years, we have worked on profiling Python. And what is great is that it comes with its low overhead sampling technology, which is unrivaled. No other profiler is able to get performance data as good as Intel V2Namplifier. So there are some techniques how we are able to get performance data with low overhead. So basically when Big Brother is watching, there is no big impact on the performance of your real application. With Intel V2Namplifier, we are able to get precise... We are able to get precise line level information. Some profilers allow you to do that, but others you may use, give you data at the function level. So basically you have to kind of guess where the performance is if you have a big function. But with V2Namplifier, you get right to the source line where there are bottlenecks. Now a bottleneck is basically like the bottle in the neck. This is where the performance is kind of capped. And our goal is to find those areas in your code and optimize on them in order to eventually increase the performance of your application. And what is also great is that we can not only analyze the Python performance, but also cite in language. And if applicable, any C code that your Python code is calling. Basically you can analyze your whole system and get data about not just the Python binary and the Python files being called, but other modules that can be built in C or C++. So in the coming 10 to 15 minutes, I'll be talking about why Python optimization is important. How do we find those bottlenecks? And a very short overview of the various profilers available on the market. And then a very quick demo of how the GUI looks like, what you see in the tool, and a few words about mixed-word profiling. So why do we need Python optimization? Well, it's no denial. Python is everywhere. Python is being used in a lot of applications that today need a lot of performance. So if you look at web frameworks, Django, Turbogears, Flask, so all these require that stuff be done really, really, really fast. There are built systems like Scones, build bot. I don't know if you use it in your company, but we use build bot for instance actually to build the package for IntelliVtion amplifier and other tools across Intel. Scientific calculations. There are tools like FreeCAD, it's a 3D modeling software that has large sections built in Python. So these require high performance. There are also other tools if you know. Gamepan Linux made out of Python. Games, there are games like Civilization 4, The Sims 4, these are Python based games. Obviously, you want your game to be efficient and run fast, right? So how do we measure the performance? There are a couple of techniques. There is code examination. You can open the editor and check the code. This can be very tedious if you don't own the code, you haven't coded it, or if the code is super large, how would you check everything? But that's one way. There is another way, logging. You basically enter pieces of code in your Python script and say, okay, print this time step here and let me know at the end of my function how much time that function run. This is also tedious, manual work. Then there is profiling. Profiling is basically the code how IntelliVtion amplifier works. In essence, what we are going to do is gather metrics from the system as the application is running. And then at the end of the run, we are going to analyze all those metrics and make sense out of all the data that we get. And we are going to focus on CPU hotspot profiling and find places in your code where your code is spending a lot of time on the CPU or wasting a lot of time off. You have a threaded application where the one thread is waiting on a lock and not doing anything or essentially stalling. Finding those issues and removing them is the way to go. Now profiling. There are a couple of types of profiling. There is event-based profiling, which is essentially collecting data when certain events happen. For instance, entering a function or exiting a function or loading a class and loading a class. So things like that. So at those certain events, we get performance data. There is also instrumentation where the target application is modified and basically the application profiles itself. And then there is sampling. Statistical profiling. Now this is how VTUN works. VTUN is a statistical performance profiler. There are some caveats. To bear in mind, obviously, as a statistical method, the larger the data, the larger the time that your application is running, the more accurate it is. So this is why I have underlined approximate there, but I've also put in bold much less intrusive. So with this statistical method that we employ in order to measure performance of Python applications, we're able to get low overhead performance profiles. And the longer your application runs, the better the results. This is a short overview of the various profilers you may have seen or not. There may be others, but these are the most common ones. Intel VTUN amplifier, what is great with it is that it comes with a rich, highly advanced, highly customizable GUI viewer in order to see quickly and visually where are the problems. Works on Linux windows. And what is also nice is this line level profiling. Not at the function level, but right at the source line where your problems are. And overhead, very important in Python interpreted world, only 1.1x performance. And that's a really low number. Compared to other line profilers like line profiler itself, which has a 10x performance hit. So basically when you use line profilers, unusable, you get bogus data. C profile gets you data at the function level with a relatively low overhead, but then again, the granularity is very coarse. And there are also other Python tools that come bundled in IDs like Visual Studio. Line function level, 2x performance hit. Our tool works with basically every Python distribution you may be using. Even the Python distribution supplied by Ubuntu or whatever system you're using, or our own, obviously, the Intel distribution for Python, which is built with ICC. For 2.7x, Python free, and remote collection over SSH. So you can be using a Windows machine, and then you can remote profile a Linux machine where your Python code is running. So that's really great. You can attach to a running process if your Python code cannot be stopped. You can just attach to the PID and get performance data. And analyzing performance is actually really simple. Some free basic steps, create a project in our tool, configure the various settings, run, interpret. Essentially, I did a small test just to show you how it works. So I have actually a code in Python. It's doing something very, very simple. I will show you this piece of code. I hope it's not too small. Can you see it? Is it good enough? Yeah. Okay, I'll get the thumbs up, so it's good. So this code is very simple. Not a lot of lines of code. That's it only one script, but it does some computation, some heavy computation. So imagine seeing this in some high-performance kernels. What it does is there's a small main script. There are two parts. One is going to use multiprocessing and create two subprocesses, and then call multiply, which is essentially going to multiply as it says, two matrices, A times B and store it in C. So we're going to create two subprocesses and do this highly, well, quite badly made free nested loop computation here. So if you guys do this, don't do it. It's really bad implementation. And then there is another method, which is out of the box using numpy. So this is the blast multiply. Okay, so basically in algebra. And then we're going to run the code. I've already run it in my Linux virtual machine. I collected the results in order to save time and opened it in VTune here on Windows. So this is how it looks like. I have in my summary page an overview of my other time that the application has run. There is also the CPU time, which is basically the time per CPU core. Here I see 130, which looks good because I have a dual core system. And the elapsed time, my wall clock time was 57 times 2 approximately 100. So my code was actually quite parallelized. And you can also see in the CPU usage histogram, my CPU concurrency was 2. And that's great. And some collection platform. But although I opened two multiprocessors because I have a two core system, that doesn't mean that it was great because we're free nested loop. It's not so nice. That's why also in this script, I'm also profiling the performance of this blast numpy code. If I go in the bottom up, oh, actually one more thing. In the top hotspot, it has already listed where you need to spend time to optimize your code. So if I go into the bottom up, it has sorted all the various methods called in your Python script. And you can see that multiply, the aggregation of those two multiplies contributed to most of the time. And because I've also collected the call stack, I can go and drill down to how my method was called in my Python script. I can double click on it. And it will open the source file and tag at the source line where most time was spent. So this is what I've done. I've double clicked on that line, on that call stack line, and it has automatically opened the source script. I can move that line a bit here. So we can see that most of the time was spent in doing this matrix multiplication. 26% of CPU usage. And going back to the bottom up, we can see the timeline, how active was my CPU over the whole runtime of my application. You can see that for the two multiprocesses that the package multiprocess has created, my CPU was active. Both processes were busy computing the matrix multiplication. And then at the end of my stupid multiplication, I had the blast one. And this can be seen at the very tiny end here. I can zoom in and filter in by selection. This is the zoomed in timeline. There's a very tiny little piece on that main thread, which is thread ID 3, 3, 4, 5. And that was the blast version using NumPy. We can even zoom in further, filter in, zoom in and filter in. So what this does is it will get that timeline. I'm zooming in and then filter on the timeline. You tell me during that timeline which methods were being called. So even more control and more power on what you see. So I can see that for this little part here, for instance, array matrix product was called. It is a shared object. So NumPy built with C. So it's a shared object. And the call stack for NumPy. So going back to my slides. You are able to also run mixed mode analysis. So basically get performance information about your Python code and also siten or native code being called in your application. BTC, C++. And you get all these, for instance here. Shared object. So that's a native library. And the other one is Py. So Python script. So the summary. Tuning your application, obviously, is a good thing. Everybody has to do it. There are ways to do it. VTune is a tool for it. Because I've been asked earlier by Thomas who's sitting in front. Maybe that's interesting for you. Intel VTune amplifier is a commercial product, but there are ways also to get it for free. It's for free in the beta program. So if you sign up for the beta 2008 and 18 that comes up with more advanced profiling capabilities for VTune, for instance, getting detailed information about credit applications and also memory consumption. It's available in the 18 version beta. It's for free for testing evaluation for a long period of time. It's also for free for all people in academics, students, professors, universities, anybody from academia, it's for free. But only for companies that work on real projects and generate money, you require a license. Just a small word about it. I'm an engineer. I don't talk about business, but I think that might be relevant for you. You may get more information in two talks conducted by my colleague David Lui. There is infrastructure design patterns with Python on Wednesday, but what is more relevant to this talk would be probably the workshop on Thursday, which is all about the hands-on on how to tune your application with our tools. On this, thank you very much for your attention. Hi, and thanks for your talk. If I understand well, you can annotate the source of Python and also C to see line by line the time of execution. Will it be possible also to annotate directly the Python source and not the C++ or C source that this Python generated? What do you mean by annotate? How else? Because there is instrumentation. But tell me more about annotation in your case. I mean, just as we saw in the diagram that you can see actually the source lines and the time that they took to execute the cumulative time, this kind of profiling part. If instead of showing the C source that was generated from the Python, if we can see that directly with the lines of Python. Yeah, actually, you will start directly from the line of Python. How does it work? Because the question was said without microphone. So the application is already running. It has a process ID. How do we actually attach to it? Well, there are mechanisms. So you already know the PID, right? If it's running. Also if it has a PID, you can also know the name of the application. And then in the GUI, you can either provide the name or the process ID. And Vichun will attach to it. And one other question, when you have C extension modules, you also need that module to be compiled with the debug flag so that you can sample from it. Yeah. And if you don't have access to that, like if it's just the binary that came in your distribution. Yeah, that's a very good question. Yeah, in this case, you will basically see Fank at dead beef, which is basically a hex code for functions that you don't know the name. Our Python binary provided by the distribution is built with ICC with a debug flag. So essentially, you can see deep down in Python itself, the method names being used. That's for Python. And in the next library, obviously, you would like to have minus G to get the debug information for your code. And your Python distribution comes with an Anaconda distribution, or you have other channels. This is just one of the way. You can actually just do, you just add the repository. And then you can also do, yeah, install. But Anaconda is a preferred way. There is Anaconda, and some of this. Thank you. Oh, hello. So you mentioned that VTUNE is a statistical type profiler. And we've seen some results of some of the code that you are running. So the metric multiplication. I was wondering if the results that we've seen are actually the results of running the code maybe like a number of times, 10,000 times, and taking some statistics over those, or was that just one of run, and we just displayed the results of them? Also that's an excellent question. In this case, it was run once. So this is what you get right away. But in order for yourself to confirm that the data that I got actually makes sense and is true, you run it yourself many times. You can have actually another Python script that runs your script many times. And also, our tool, VTUNE, comes with a command line interface as well. So you can have this one line that does the profiling for you, saves the results and everything. So you can have your script and automate the running of your program many times and have VTUNE wrapping your application in its command line interface. And this is how you can have your own build system or regression testing system and get data. And if that's the case, how do you find the time? Is it quite slow to run this kind of analysis, like multiple times? This one I didn't get. There was an echo. So I was just wondering what time, how much time do you have to spend to have, to say, run your code 10,000 times and draw statistics from it? Do you have any type of metrics on that? Okay. This depends on the resolution of your analysis. So in my case, I did an analysis with a resolution of 10 milliseconds, which is quite big actually. So if you want more data, more resolution, you can lower this time. And how many times? Obviously, the lower the time duration to get the samples, the larger the data, the larger potentially the overhead, and less accurate would be your result. So it's playing around. In general, anything longer than two free seconds is good enough. Hi. A couple of questions. Can you attach the profiler to a running process? And does that process has to be built in a special way for that? Yeah. So you can do some profiling in production kind of thing? I think that question was asked already. Sorry. Okay. So the answer is yes. You can attach to a running process. So the second question was, you had an early slide where you showed the presentation of the time taken on a particular line of code. That line of code had two calls. It was like logging.info, brackets, template.format. So there's two function calls in there. Can you decompose that in the browser to those two function calls and the process time that each one took? Because you're just showing the sum of the total for that line. And when you have these multi... The multi-process. So the question is, you have created two multi-processes? No, no, no. I'm making two function calls in one line, two method calls in one line. It's something like logging.info, bracket, template.format. So you're calling info and format in the whole line. Can you decompose that in the browser? Well, in this case, it will aggregate the time and show you on that one source line the whole time for that. But I think it's a bad practice to do this for code readability. In my opinion. I don't know how you do it. But wait, hold on, hold on. I will add one more thing, by the way. In this case, you will see the source line because we're actually associating time to a source line in your source code. But in the bottom view, you will see different functions, two functions. But the thing is, when you click on both functions, you will go to the same source line. But you will know the time for each function. I would like to ask, what interpreter do you use in your distribution? And if you have applied modifications to the interpreter to make it faster? Well, the acoustics are not so great. What I got? What I got is that how is alignment done? Memory alignment? No, no, no. What interpreter do you use? And have you made any changes to the interpreter to optimize it? Can anybody rephrase this for me? Please. What's the difference between the interpreter and the interpreter? Oh, okay. Thank you, Adam. Yeah. Well, our interpreter has been made from scratch and compiled with ICC. There were some changes. I don't know in detail what has changed, but there were minor changes in the interpreter. However, all the libraries making use of heavy mathematics, these have been redesigned completely, making use of MKL. So this is the benefit we're bringing with our Intel distribution of Python so that you guys, when you do HPC-based applications made in Python or machine learning, deep learning, or even using SDKs or frameworks like TensorFlow, Cafe, or even the autonomous driving SDK or the computer vision SDK from Intel that leverages the Python distribution, you get the performance out of the box. So you don't have to be like a math genius to code properly or a super software engineer with great skills and code optimization to create high performance. It's done out of the box for you. Thanks. Welcome. It may be already lunchtime. Just one thing, if you have really interesting questions that you really want to get answered, our workshop on it just on this topic could be very useful for you. It's on Thursday. Check it out. I have a question for cluster users because I see that on my machine it can connect to the process but if I have a cluster, how can measure the performance of all the workers' machine or is it possible? Great question. Yes, it is possible. So you're probably using MPI, right? Yeah. No, I'm not using MPI but I'm using just... Okay. Let me take MPI as an example. You have a cluster, several nodes. Your Python code is being running on all. You will have each an amplifier, the sampling driver on all those guys. And with MPI G tool, for instance, you just pass MPI run, G tool, amplifier XECL, which is the command line interface tool, and then your Python script and it will do the job out of for you and get you the results. It's magic. It's really nice. Very interesting. Thanks. Yeah. Okay, thank you.
 I'm Shailen, I am a technical consulting engineer at Intel in the developer products division  team and we are based in Munich, Germany.  And today's focus will be performance analysis of Python applications.  And yeah, we have to say it's of no denial that Python is getting a lot of traction,  a lot of importance these days.  And if you look at what our friends at CodeEval have published, indeed, Python has grown  in popularity over the last years and in 2016, Python remains the number one most used language.  And also what is more surprising is that Python remains the number one programming language  in hiring demand.  So it's a great skill to have in this decade to be proficient in Python.  And when it comes to performance analysis, there are certain fields that are kind of  driving the technologies of the future and technologies that are kind of really important  right now.  And these fields, I would say, would be mathematics and data science.  And to get my facts straight, to get the numbers correct, I went to Stack Overflow, our favorite  website where we have problems.  And Stack Overflow shows me that indeed, Python is the most used language in the fields of  mathematics and data science.  Now you may think the math doesn't make sense.  If you add all the percentage, it doesn't make up to 100.  That's because out of those approximately 50,000 people who responded to the survey, they chose  several languages.  But most of them chose Python.  Over 50% of this.  So that's quite impressive.  So math and data science, these fields actually drive high performance computing, or HPC,  and other fields like artificial intelligence, machine learning, or deep learning.  And Intel realizes that these fields are going to define the future.  And so we have worked really hard to release a distribution of software, which we call  the Intel distribution for Python.  And it comes up out of the box with highly optimized sublibraries to allow you to develop  high performance applications with Python.  We made it super easy to use, super easy to install.  Packages can be easily downloaded through Anaconda or even YAM, providing the RPMs.  And our distribution of Python comes with highly optimized libraries like NumPy, SciPy,  Techitlearn, which actually are the base leverages Intel Mkl, which is short for MathKernel library.  Now in itself, Mkl, if you've never heard about it, if you've heard about it, it's  made in assembly.  It's super optimized.  Mathematical routines have been designed to make the most out of the Intel architecture,  how many cores you have, make use of the latest instruction set architecture, for instance,  AVX, AVX 512, whatever you have out of the box, make the most of vectorization.  So you don't have to worry about this by using the Intel distribution for Python.  Now performance is really important.  So how do we actually measure performance of a Python application?  Enter Intel V2Namplifier.  V2Namplifier is a code profiler.  It is a profiler that allows you to know where are the performance problems in your  software.  It has been developed over many years, over 15 years, and it's still in development.  We're getting a lot of improvements every day.  Our engineers are working hard, and over the last four years, we have worked on profiling  Python.  And what is great is that it comes with its low overhead sampling technology, which is  unrivaled.  No other profiler is able to get performance data as good as Intel V2Namplifier.  So there are some techniques how we are able to get performance data with low overhead.  So basically when Big Brother is watching, there is no big impact on the performance  of your real application.  With Intel V2Namplifier, we are able to get precise...  We are able to get precise line level information.  Some profilers allow you to do that, but others you may use, give you data at the function  level.  So basically you have to kind of guess where the performance is if you have a big function.  But with V2Namplifier, you get right to the source line where there are bottlenecks.  Now a bottleneck is basically like the bottle in the neck.  This is where the performance is kind of capped.  And our goal is to find those areas in your code and optimize on them in order to eventually  increase the performance of your application.  And what is also great is that we can not only analyze the Python performance, but also  cite in language.  And if applicable, any C code that your Python code is calling.  Basically you can analyze your whole system and get data about not just the Python binary  and the Python files being called, but other modules that can be built in C or C++.  So in the coming 10 to 15 minutes, I'll be talking about why Python optimization is important.  How do we find those bottlenecks?  And a very short overview of the various profilers available on the market.  And then a very quick demo of how the GUI looks like, what you see in the tool, and a few words  about mixed-word profiling.  So why do we need Python optimization?  Well, it's no denial.  Python is everywhere.  Python is being used in a lot of applications that today need a lot of performance.  So if you look at web frameworks, Django, Turbogears, Flask, so all these require that  stuff be done really, really, really fast.  There are built systems like Scones, build bot.  I don't know if you use it in your company, but we use build bot for instance actually  to build the package for IntelliVtion amplifier and other tools across Intel.  Scientific calculations.  There are tools like FreeCAD, it's a 3D modeling software that has large sections built in Python.  So these require high performance.  There are also other tools if you know.  Gamepan Linux made out of Python.  Games, there are games like Civilization 4, The Sims 4, these are Python based games.  Obviously, you want your game to be efficient and run fast, right?  So how do we measure the performance?  There are a couple of techniques.  There is code examination.  You can open the editor and check the code.  This can be very tedious if you don't own the code, you haven't coded it, or if the  code is super large, how would you check everything?  But that's one way.  There is another way, logging.  You basically enter pieces of code in your Python script and say, okay, print this time  step here and let me know at the end of my function how much time that function run.  This is also tedious, manual work.  Then there is profiling.  Profiling is basically the code how IntelliVtion amplifier works.  In essence, what we are going to do is gather metrics from the system as the application  is running.  And then at the end of the run, we are going to analyze all those metrics and make sense  out of all the data that we get.  And we are going to focus on CPU hotspot profiling and find places in your code where your code  is spending a lot of time on the CPU or wasting a lot of time off.  You have a threaded application where the one thread is waiting on a lock and not doing  anything or essentially stalling.  Finding those issues and removing them is the way to go.  Now profiling.  There are a couple of types of profiling.  There is event-based profiling, which is essentially collecting data when certain events happen.  For instance, entering a function or exiting a function or loading a class and loading  a class.  So things like that.  So at those certain events, we get performance data.  There is also instrumentation where the target application is modified and basically the  application profiles itself.  And then there is sampling.  Statistical profiling.  Now this is how VTUN works.  VTUN is a statistical performance profiler.  There are some caveats.  To bear in mind, obviously, as a statistical method, the larger the data, the larger the  time that your application is running, the more accurate it is.  So this is why I have underlined approximate there, but I've also put in bold much less  intrusive.  So with this statistical method that we employ in order to measure performance of Python  applications, we're able to get low overhead performance profiles.  And the longer your application runs, the better the results.  This is a short overview of the various profilers you may have seen or not.  There may be others, but these are the most common ones.  Intel VTUN amplifier, what is great with it is that it comes with a rich, highly advanced,  highly customizable GUI viewer in order to see quickly and visually where are the problems.  Works on Linux windows.  And what is also nice is this line level profiling.  Not at the function level, but right at the source line where your problems are.  And overhead, very important in Python interpreted world, only 1.1x performance.  And that's a really low number.  Compared to other line profilers like line profiler itself, which has a 10x performance  hit.  So basically when you use line profilers, unusable, you get bogus data.  C profile gets you data at the function level with a relatively low overhead, but then again,  the granularity is very coarse.  And there are also other Python tools that come bundled in IDs like Visual Studio.  Line function level, 2x performance hit.  Our tool works with basically every Python distribution you may be using.  Even the Python distribution supplied by Ubuntu or whatever system you're using, or  our own, obviously, the Intel distribution for Python, which is built with ICC.  For 2.7x, Python free, and remote collection over SSH.  So you can be using a Windows machine, and then you can remote profile a Linux machine  where your Python code is running.  So that's really great.  You can attach to a running process if your Python code cannot be stopped.  You can just attach to the PID and get performance data.  And analyzing performance is actually really simple.  Some free basic steps, create a project in our tool, configure the various settings,  run, interpret.  Essentially, I did a small test just to show you how it works.  So I have actually a code in Python.  It's doing something very, very simple.  I will show you this piece of code.  I hope it's not too small.  Can you see it?  Is it good enough?  Yeah.  Okay, I'll get the thumbs up, so it's good.  So this code is very simple.  Not a lot of lines of code.  That's it only one script, but it does some computation, some heavy computation.  So imagine seeing this in some high-performance kernels.  What it does is there's a small main script.  There are two parts.  One is going to use multiprocessing and create two subprocesses, and then call multiply,  which is essentially going to multiply as it says, two matrices, A times B and store  it in C. So we're going to create two subprocesses and do this highly, well, quite badly made  free nested loop computation here.  So if you guys do this, don't do it.  It's really bad implementation.  And then there is another method, which is out of the box using numpy.  So this is the blast multiply.  Okay, so basically in algebra.  And then we're going to run the code.  I've already run it in my Linux virtual machine.  I collected the results in order to save time and opened it in VTune here on Windows.  So this is how it looks like.  I have in my summary page an overview of my other time that the application has run.  There is also the CPU time, which is basically the time per CPU core.  Here I see 130, which looks good because I have a dual core system.  And the elapsed time, my wall clock time was 57 times 2 approximately 100.  So my code was actually quite parallelized.  And you can also see in the CPU usage histogram, my CPU concurrency was 2.  And that's great.  And some collection platform.  But although I opened two multiprocessors because I have a two core system, that doesn't  mean that it was great because we're free nested loop.  It's not so nice.  That's why also in this script, I'm also profiling the performance of this blast numpy code.  If I go in the bottom up, oh, actually one more thing.  In the top hotspot, it has already listed where you need to spend time to optimize your code.  So if I go into the bottom up, it has sorted all the various methods called in your Python  script.  And you can see that multiply, the aggregation of those two multiplies contributed to most  of the time.  And because I've also collected the call stack, I can go and drill down to how my method was  called in my Python script.  I can double click on it.  And it will open the source file and tag at the source line where most time was spent.  So this is what I've done.  I've double clicked on that line, on that call stack line, and it has automatically opened  the source script.  I can move that line a bit here.  So we can see that most of the time was spent in doing this matrix multiplication.  26% of CPU usage.  And going back to the bottom up, we can see the timeline, how active was my CPU over the  whole runtime of my application.  You can see that for the two multiprocesses that the package multiprocess has created,  my CPU was active.  Both processes were busy computing the matrix multiplication.  And then at the end of my stupid multiplication, I had the blast one.  And this can be seen at the very tiny end here.  I can zoom in and filter in by selection.  This is the zoomed in timeline.  There's a very tiny little piece on that main thread, which is thread ID 3, 3, 4, 5.  And that was the blast version using NumPy.  We can even zoom in further, filter in, zoom in and filter in.  So what this does is it will get that timeline.  I'm zooming in and then filter on the timeline.  You tell me during that timeline which methods were being called.  So even more control and more power on what you see.  So I can see that for this little part here, for instance, array matrix product was called.  It is a shared object.  So NumPy built with C.  So it's a shared object.  And the call stack for NumPy.  So going back to my slides.  You are able to also run mixed mode analysis.  So basically get performance information about your Python code and also siten or native  code being called in your application.  BTC, C++.  And you get all these, for instance here.  Shared object.  So that's a native library.  And the other one is Py.  So Python script.  So the summary.  Tuning your application, obviously, is a good thing.  Everybody has to do it.  There are ways to do it.  VTune is a tool for it.  Because I've been asked earlier by Thomas who's sitting in front.  Maybe that's interesting for you.  Intel VTune amplifier is a commercial product, but there are ways also to get it for free.  It's for free in the beta program.  So if you sign up for the beta 2008 and 18 that comes up with more advanced profiling  capabilities for VTune, for instance, getting detailed information about credit applications  and also memory consumption.  It's available in the 18 version beta.  It's for free for testing evaluation for a long period of time.  It's also for free for all people in academics, students, professors, universities, anybody  from academia, it's for free.  But only for companies that work on real projects and generate money, you require a license.  Just a small word about it.  I'm an engineer.  I don't talk about business, but I think that might be relevant for you.  You may get more information in two talks conducted by my colleague David Lui.  There is infrastructure design patterns with Python on Wednesday, but what is more relevant  to this talk would be probably the workshop on Thursday, which is all about the hands-on  on how to tune your application with our tools.  On this, thank you very much for your attention.  Hi, and thanks for your talk.  If I understand well, you can annotate the source of Python and also C to see line by  line the time of execution.  Will it be possible also to annotate directly the Python source and not the C++ or C source  that this Python generated?  What do you mean by annotate?  How else?  Because there is instrumentation.  But tell me more about annotation in your case.  I mean, just as we saw in the diagram that you can see actually the source lines and  the time that they took to execute the cumulative time, this kind of profiling part.  If instead of showing the C source that was generated from the Python, if we can see that  directly with the lines of Python.  Yeah, actually, you will start directly from the line of Python.  How does it work?  Because the question was said without microphone.  So the application is already running.  It has a process ID.  How do we actually attach to it?  Well, there are mechanisms.  So you already know the PID, right?  If it's running.  Also if it has a PID, you can also know the name of the application.  And then in the GUI, you can either provide the name or the process ID.  And Vichun will attach to it.  And one other question, when you have C extension modules, you also need that module to be compiled  with the debug flag so that you can sample from it.  Yeah.  And if you don't have access to that, like if it's just the binary that came in your  distribution.  Yeah, that's a very good question.  Yeah, in this case, you will basically see Fank at dead beef, which is basically a hex  code for functions that you don't know the name.  Our Python binary provided by the distribution is built with ICC with a debug flag.  So essentially, you can see deep down in Python itself, the method names being used.  That's for Python.  And in the next library, obviously, you would like to have minus G to get the debug information  for your code.  And your Python distribution comes with an Anaconda distribution, or you have other channels.  This is just one of the way.  You can actually just do, you just add the repository.  And then you can also do, yeah, install.  But Anaconda is a preferred way.  There is Anaconda, and some of this.  Thank you.  Oh, hello.  So you mentioned that VTUNE is a statistical type profiler.  And we've seen some results of some of the code that you are running.  So the metric multiplication.  I was wondering if the results that we've seen are actually the results of running the  code maybe like a number of times, 10,000 times, and taking some statistics over those,  or was that just one of run, and we just displayed the results of them?  Also that's an excellent question.  In this case, it was run once.  So this is what you get right away.  But in order for yourself to confirm that the data that I got actually makes sense and  is true, you run it yourself many times.  You can have actually another Python script that runs your script many times.  And also, our tool, VTUNE, comes with a command line interface as well.  So you can have this one line that does the profiling for you, saves the results and everything.  So you can have your script and automate the running of your program many times and have  VTUNE wrapping your application in its command line interface.  And this is how you can have your own build system or regression testing system and get  data.  And if that's the case, how do you find the time?  Is it quite slow to run this kind of analysis, like multiple times?  This one I didn't get.  There was an echo.  So I was just wondering what time, how much time do you have to spend to have, to say,  run your code 10,000 times and draw statistics from it?  Do you have any type of metrics on that?  Okay.  This depends on the resolution of your analysis.  So in my case, I did an analysis with a resolution of 10 milliseconds, which is quite big actually.  So if you want more data, more resolution, you can lower this time.  And how many times?  Obviously, the lower the time duration to get the samples, the larger the data, the larger  potentially the overhead, and less accurate would be your result.  So it's playing around.  In general, anything longer than two free seconds is good enough.  Hi.  A couple of questions.  Can you attach the profiler to a running process?  And does that process has to be built in a special way for that?  Yeah.  So you can do some profiling in production kind of thing?  I think that question was asked already.  Sorry.  Okay.  So the answer is yes.  You can attach to a running process.  So the second question was, you had an early slide where you showed the presentation of  the time taken on a particular line of code.  That line of code had two calls.  It was like logging.info, brackets, template.format.  So there's two function calls in there.  Can you decompose that in the browser to those two function calls and the process time that  each one took?  Because you're just showing the sum of the total for that line.  And when you have these multi...  The multi-process.  So the question is, you have created two multi-processes?  No, no, no.  I'm making two function calls in one line, two method calls in one line.  It's something like logging.info, bracket, template.format.  So you're calling info and format in the whole line.  Can you decompose that in the browser?  Well, in this case, it will aggregate the time and show you on that one source line the  whole time for that.  But I think it's a bad practice to do this for code readability.  In my opinion.  I don't know how you do it.  But wait, hold on, hold on.  I will add one more thing, by the way.  In this case, you will see the source line because we're actually associating time to  a source line in your source code.  But in the bottom view, you will see different functions, two functions.  But the thing is, when you click on both functions, you will go to the same source line.  But you will know the time for each function.  I would like to ask, what interpreter do you use in your distribution?  And if you have applied modifications to the interpreter to make it faster?  Well, the acoustics are not so great.  What I got?  What I got is that how is alignment done?  Memory alignment?  No, no, no.  What interpreter do you use?  And have you made any changes to the interpreter to optimize it?  Can anybody rephrase this for me?  Please.  What's the difference between the interpreter and the interpreter?  Oh, okay.  Thank you, Adam.  Yeah.  Well, our interpreter has been made from scratch and compiled with ICC.  There were some changes.  I don't know in detail what has changed, but there were minor changes in the interpreter.  However, all the libraries making use of heavy mathematics, these have been redesigned completely,  making use of MKL.  So this is the benefit we're bringing with our Intel distribution of Python so that you  guys, when you do HPC-based applications made in Python or machine learning, deep learning,  or even using SDKs or frameworks like TensorFlow, Cafe, or even the autonomous driving SDK or  the computer vision SDK from Intel that leverages the Python distribution, you get the performance  out of the box.  So you don't have to be like a math genius to code properly or a super software engineer  with great skills and code optimization to create high performance.  It's done out of the box for you.  Thanks.  Welcome.  It may be already lunchtime.  Just one thing, if you have really interesting questions that you really want to get answered,  our workshop on it just on this topic could be very useful for you.  It's on Thursday.  Check it out.  I have a question for cluster users because I see that on my machine it can connect to  the process but if I have a cluster, how can measure the performance of all the workers'  machine or is it possible?  Great question.  Yes, it is possible.  So you're probably using MPI, right?  Yeah.  No, I'm not using MPI but I'm using just...  Okay.  Let me take MPI as an example.  You have a cluster, several nodes.  Your Python code is being running on all.  You will have each an amplifier, the sampling driver on all those guys.  And with MPI G tool, for instance, you just pass MPI run, G tool, amplifier XECL, which  is the command line interface tool, and then your Python script and it will do the job  out of for you and get you the results.  It's magic.  It's really nice.  Very interesting.  Thanks.  Yeah.  Okay, thank you.
Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_0_875.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_876_7799.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_7800_8015.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_8016_9995.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_9996_19127.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_19128_19859.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_19860_23711.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_23712_34139.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_34140_35807.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_35808_36347.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_36348_37319.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_37320_38483.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_38484_38831.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_38832_39455.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_39456_40703.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_40704_41267.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_41268_42347.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_42348_43775.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_43776_45047.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_45048_45143.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_45144_47927.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_47928_48695.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_48696_48815.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_48816_49703.png Shailen_Sobhee_-_Python_Profiling_with_Intel__VTune__Amplifier_G31yUYwvGgc_49704_49849.png