<jupyter_start><jupyter_text>This example is prepared by [<NAME>](https://github.com/qingkaikong/) (<EMAIL>) from [Berkeley Seismological Lab](http://seismo.berkeley.edu/) for the lightning talk at [The Hacker Within](http://www.thehackerwithin.org/berkeley/) at BIDS on April 6th 2016. > The purpose of this script is to show how I usually do to speedup my python script. And hope this is useful to you.   Line_profilerMany times, you find your python script is slow, but you just don't know which part drags the whole performance down. To get an idea, I usually use the [line_profiler](https://github.com/rkern/line_profiler) from <NAME>, this is really good if you want to identify which line uses more time than you expceted, and how often each line executed. Here's a blog talking about profile python [<NAME>ini](http://marcobonzanini.com/2015/01/05/my-python-code-is-slow-tips-for-profiling/)You can use line_profiler either in command line or in ipython notebook.  1 Command lineIn a typical workflow, one only cares about line timings of a few functions because wading through the results of timing every single line of code would be overwhelming. However, LineProfiler does need to be explicitly told what functions to profile. The easiest way to get started is to use the kernprof script.Steps:> 1. in your script, you decorate the functions you want to profile with @profile. For example:```python@profiledef function_to_profile(a, b, c):    ...```2. Run the following command in the terminal:```shkernprof -v -l profile_test.py``` 2 Run in Ipython notebookTo run line_profiler in the notebook, you need load the extension first, and then use the magic commands %lprun to profile the script.<jupyter_code>%load_ext line_profiler
def example_function(myRange):
    # directly convert range to string list
    str_list = []
    for i in myRange:
        str_list.append(str(i))
        
def example_function2(myRange):
    # use list comprehension to convert range to string list
    str_list = [str(i) for i in myRange] 
        
%lprun -f example_function example_function(range(1000000))
%lprun -f example_function2 example_function2(range(1000000))<jupyter_output><empty_output><jupyter_text>Using f2pyf2py - Fortran to Python interface generator, is used to call Fortran 77/90/95 external subroutines and Fortran 90/95 module subroutines as well as C functions. It is part of the Numpy now. You can find more details [here](http://docs.scipy.org/doc/numpy-dev/f2py/). Let's grab the example from the above link, and compare a python version and a fortran version for speed.   Python version<jupyter_code>import numpy as np

def fib(A):
    '''
    CALCULATE FIRST N FIBONACCI NUMBERS
    '''
    n = len(A)
    
    for i in range(n):
        if i == 0:
            A[i] = 0.
        elif i == 1:
            A[i] = 1.
        else:
            A[i] = A[i-1] + A[i-2]
            
    return A
dat_in = np.zeros(10) 
dat_out = fib(dat_in)
dat_out
dat_in = np.zeros(1000) 
%lprun -f fib fib(dat_in)<jupyter_output><empty_output><jupyter_text>Fortran versionlet's first write a simple fortran subroutine<jupyter_code>!ls
%%writefile fib1.f
C FILE: FIB1.F
      SUBROUTINE FIB(A,N)
C
C     CALCULATE FIRST N FIBONACCI NUMBERS
C
      INTEGER N
      REAL*8 A(N)
      DO I=1,N
         IF (I.EQ.1) THEN
            A(I) = 0.0D0
         ELSEIF (I.EQ.2) THEN
            A(I) = 1.0D0
         ELSE 
            A(I) = A(I-1) + A(I-2)
         ENDIF
      ENDDO
      END
C END FILE FIB1.F
!f2py -c fib1.f -m fib1
!ls<jupyter_output>Profile python script and f2py.ipynb fib1.f
README.md                            [31mfib1.so[m[m
example.py                           [34mfib1.so.dSYM[m[m<jupyter_text>You can see it created an python interface fib1.so. Now you can import the function into python.<jupyter_code>import fib1
import numpy as np
print fib1.fib.__doc__
a = np.zeros(9)
fib1.fib(a)
a<jupyter_output><empty_output><jupyter_text>Let's compare the time of both function, we can see for this exmaple, the fortran version is about 100 times faster than the python version.<jupyter_code>a = np.zeros(1000)
%timeit fib1.fib(a)
%timeit fib(a)<jupyter_output>1000 loops, best of 3: 363 µs per loop
{'fraction_non_alphanumeric': 0.08892577661844914, 'fraction_numerical': 0.02513635285748162, 'mean_word_length': 3.3484536082474228, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 25, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '304776', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1480, 'n_tokens_neox': 1383, 'n_words': 504, 'path': 'python_line_profiler_f2py/Profile python script and f2py.ipynb', 'repo_name': 'rainwoodman/thehackerwithin-berkeley', 'size': '4249'}