<jupyter_start><jupyter_text>Python Metaprogramming (the sometimes useful but often interesting) Why?1. Repetative code is an excuse to waste time?2. Might allow you to control functions, classes behavior at a higher level than just writing code3. Using decorators that are available in the Python standard library might save you time/preserve functionality of your code https://docs.python.org/3/library/functools.html What are they? 1. Decorators apply operations to functions/classes that are generally applicable to many objects.2. They are functions that take function/classes as arguments and change their behavior3. Special syntax @decorator Decorators example: Debugging with the print statementLet's start with a simple function that does something:```pythondef sum(x,y):    print x+y;    return x+y;```Ok, but you probably want to remove the print before you let someone see your code... and maybe changes in the future will also need this debugging step. Our first 'decorator'```pythondef debug(val):    print val;    return val;def sum(x,y):    return x+y;x,y=2,3; 'explicit decorator'debug(sum(x,y))```Ok but we moved the print statement -out- of the function. Version controling between the master and debugging branch might still be weird. What we really want is a modified version of the function sum. What if a function could modify the call to sum:```pythondef debug_on(func):    def debugging(*args,**kwargs):        print 'Scope: debugging',func(*args,**kwargs);        return func(*args,**kwargs);    print 'Scope: debug_on',debugging    return debugging    ```Read what is happening here carefully. ```pythondef debug_on(func):    def debugging(*args,**kwargs):    ...    return debugging;```We take in a function, and return a function. What happens for example:```python>> a = debug_on(sum);  a->debugging                       Note that debugging calls our original function sumScope: debug_on ```We pass all positional (\*args) and keyword (\*\*kwargs) through debugging to sum and return the result. ```python>> print(a(3,4));Scope: debugging, 77```<jupyter_code>from functools import wraps
def debug_on(func):
    @wraps(func) #preserving the metadata for func
    def debugging(*args,**kwargs):
        retval = func(*args,**kwargs);
        print('Scope: debugging %s:%s:%s'%(func,func.__name__,retval));
        return retval;
    print('Scope: debug_on',debugging)
    return debugging
# the @ syntax is equivalent to function nesting https://www.python.org/dev/peps/pep-0318/
# debug_on(sum)

@debug_on
def sum(x,y):        # sum->debugging
    '''Return the sum'''
    return x+y;
@debug_on
def sum2(x,y):       #sum2->debugging, but seperate instance
    '''Return the square of the sum'''
    return (x+y)**2.;

print('Scope: main sum %s, sum2 %s'%(sum,sum2))
print('-'*10)
print('Q:What does %s do? A:%s'%(sum2.__name__,sum2.__doc__))<jupyter_output>Scope: debug_on <function sum at 0xf4ab2104>
Scope: debug_on <function sum2 at 0xf4ab2f5c>
Scope: main sum <function sum at 0xf4ab2104>, sum2 <function sum2 at 0xf4ab2f5c>
----------
Q:What does sum2 do? A:Return the square of the sum<jupyter_text>NOTE! decorated sum and debugging point to the same function!<jupyter_code>sum(3,4)
sum2(3,4)<jupyter_output>Scope: debugging <function sum at 0xf4ab214c>:sum:7
Scope: debugging <function sum2 at 0xf4ab2194>:sum2:49.0<jupyter_text>What about class or a class functions?An example class:```pythonclass point():    def __init__(self,x,y):        self.x=x;        self.y=y;    def dist(self,x,y):        return self.x**2.+self.y**2;```<jupyter_code>@debug_on
class point():
    @debug_on
    def __init__(self,x,y):
        self.x=x;
        self.y=y;
    @debug_on
    def dist(self):
        return self.x**2.+self.y**2;
a = point(1,2);
a.dist()<jupyter_output>Scope: debugging <function point.__init__ at 0xf4ab265c>:__init__:None
Scope: debugging <class '__main__.point'>:point:<__main__.point object at 0xf4aaa58c>
Scope: debugging <function point.dist at 0xf4ab280c>:dist:5.0<jupyter_text>So it works on classes too. Why?(Left for the reader) Ok, so the syntax for debugging the class is kind of annoying... can we debug all the methods with a single @?```pythondef debug_on_classes(cls):    def debugging(*args,**kwargs):        cdict = cls.__dict__;        for item in cdict:            func = getattr(cls,item);            if( hasattr(func,'__call__') ):                setattr(cls,item,debug_on(func))        return cls(*args,**kwargs);    return debugging;```Note the careful syntax. Here we simply ask for all the items in the class dictionary and, if they are callable, we wrap then with the debug_on decorator. Test it out!<jupyter_code>def debug_on_classes(cls):
    @wraps(cls) #preserving the cls metadata
    def debugging(*args,**kwargs):
        cdict = cls.__dict__;
        for item in cdict:
            func = getattr(cls,item);
            if( hasattr(func,'__call__') ):
                setattr(cls,item,debug_on(func))
        return cls(*args,**kwargs);
    return debugging;
@debug_on_classes
class point():
    def __init__(self,x,y):
        self.x=x;
        self.y=y;
    def dist(self):
        return self.x**2.+self.y**2;
a = point(1,2);
a.dist();<jupyter_output>Scope: debug_on <function point.dist at 0xf4ab2bfc>
Scope: debug_on <function point.__init__ at 0xf4a8ac8c>
Scope: debugging <function point.__init__ at 0xf4a8ae84>:__init__:None
Scope: debugging <function point.dist at 0xf4a8a194>:dist:5.0<jupyter_text>What is the deal with the parameter? @(*args,**kwargs)Decorators can take arguments. Basically this involves another layer on top of the decorator```pythonfrom timeit import default_timer as timerdef time_execution(symb='*'*10):    def time_decorator(func):        def time_function(*args,**kwargs):            start = timer();            result=func(*args,**kwargs);            end = timer()-start;            print('Function call %s took %.2f seconds'%(func.__name__,end));            return result;        return time_function;    return time_decorator;```Note that this is just another function pointer on top of the time_decorator function. It manages the \*args and \*\*kwargs passed to the decorator<jupyter_code>from timeit import default_timer as timer
from time import sleep;

def time_execution(*args,**kwargs):
    symb = kwargs.pop('symb','*'*10)
    def time_decorator(func):
        def time_function(*args,**kwargs):
            start = timer();
            result=func(*args,**kwargs);
            end = timer()-start;
            print('%s Function call %s took %.2f seconds'%(symb,func.__name__,end));
            return result;
        return time_function;
    return time_decorator;

@time_execution(symb='='*10)               # note that @profile is provided by https://mg.pov.lt/profilehooks/
def recursive_counter(n=0):
    while(n<10):
        sleep(0.1);
        n = recursive_counter(n+1);
    return n;

print( recursive_counter(0) )<jupyter_output>========== Function call recursive_counter took 0.00 seconds
========== Function call recursive_counter took 0.10 seconds
========== Function call recursive_counter took 0.20 seconds
========== Function call recursive_counter took 0.30 seconds
========== Function call recursive_counter took 0.40 seconds
========== Function call recursive_counter took 0.50 seconds
========== Function call recursive_counter took 0.60 seconds
========== Function call recursive_counter took 0.70 seconds
========== Function call recursive_counter took 0.80 seconds
========== Function call recursive_counter took 0.90 seconds
========== Function call recursive_counter took 1.00 seconds
10
{'fraction_non_alphanumeric': 0.12493452069146149, 'fraction_numerical': 0.022001047668936617, 'mean_word_length': 4.064323607427056, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 37, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '334406', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2386, 'n_tokens_neox': 2227, 'n_words': 759, 'path': 'python_metaprogramming/DecoratorsMetaclasses.ipynb', 'repo_name': 'rainwoodman/thehackerwithin-berkeley', 'size': '7636'}