<jupyter_start><jupyter_text>HierarchyPython code can be ordered using the following hierarchy.Note that some hierarchy levels are not applicable for Jupyter Notebooks but only for "normal" Python code. Packages [optional]A directory conaining .py files (i.e. Modules) and/ or subpackages. A directory becomes a Python Package if the file     __init__.py     (which is usually empty) is present. ModulesA file containing Python code. It may be top level or inside a Package. A module may contain functions, classes, variables (usually constants) and/ or code.    Code inside a Module is automatically executed if the module is imported. Code which should be only executed if the module itself is executed (not by imports from other modules) is protected using the following check:        if __name__ == '__main__':        code        Note that it is perfectly fine in Python to have constants and functions directly in Modules, outside classes (unlike e.g. in Java, where everything must be inside a class). The module-level namespaces avoid naming conflicts.  Imports<jupyter_code>%cat ../code/my_module.py # example module to be imported<jupyter_output>"""example file for module imports"""

__version__ = '0.0.1'

my_secret = 'you_will_never_guess'
my_list = [1, 2, 3, 4]
my_tuple = ('a', 'b', 'c')

def double_me(x):
    """doubles input value"""
    return 2*x

def power_me(x, i: int, times_two=False):
    """useless example function"""
    mult = 2 if times_two else 1
    return mult*x**i

class Multiplier:
    """class that creates multiplier objects with a given factor"""
    def __init__(self, i):
        self.i = i
    def get_result(self, x):
        """returns factor specified with initialization multiplied by input value"""
        return self.i*x

mult2 = Multiplier(2)
mult3 = Multiplier(3)

print('my_module imported')

if __name__ == '__main__':
    # this code is not executed when importing this module, but only if the module is executed
    print('executing main block')<jupyter_text>Pythonpath<jupyter_code>import sys
import os
new_path = os.path.abspath('../code')
if new_path not in sys.path:
    sys.path.append(new_path)<jupyter_output><empty_output><jupyter_text>Python searches for modules in import statements in the following places:* Current directory* Python builtins* Directories in PythonpathThe code above checks if the given directory is included in Pythonpath and adds it if not.This pattern is also useful e.g. for unit tests if the test cases are kept in a directory different from the code.  Import Specific Objects<jupyter_code>from my_module import double_me
from my_module import my_list as my_constants # rename object with importings<jupyter_output>my_module imported<jupyter_text>The     from module import object [as alias]    pattern imports specific objects of the module in local namespace.Note that the print statement at the bottom of the module is executed, although it is not in one of the imported objects. Python runs the complete module script the first time it is used in any import.<jupyter_code>double_me(42)
my_constants<jupyter_output><empty_output><jupyter_text>Antipatterns<jupyter_code>my_constants.append(5)
my_constants<jupyter_output><empty_output><jupyter_text>Extremely dangerous: changing (mutable) data in an other module. All imports referring to this data will be affected.<jupyter_code>my_tuple = ('this', 'is', 'very', 'important')
my_tuple
from my_module import * # Imports all members of the module into local namespace
my_tuple<jupyter_output><empty_output><jupyter_text>The usage of the pattern    from module import *has the following drawbacks and is therefore not recommended:* There could be naming conflicts between local and imported objects* There is no information or control in the module where the import statement has been executed which object originates from which module.  Import Whole Module as Separate Namespace<jupyter_code>import my_module<jupyter_output><empty_output><jupyter_text>This is the recommended method to import modules, especially if multiple objects of the module are required. If the number of imported objects is small, the *from module import object* syntax is also fine.The code in the imported module is only executed once during the first import. Thus the print statement is not shown again.<jupyter_code>my_module.double_me(42)
mult7 = my_module.Multiplier(7)
mult7.get_result(7)
my_module.mult2.get_result(3)
my_module.my_list # note the effect of my_constants.append(5)
my_module.my_list is my_constants # both are pointers to the same list object<jupyter_output><empty_output><jupyter_text>Note that the modification of the imported list changes this list anywhere it is imported (actually, all imports refer to the same object). Therefore it is very dangerous to import and modify mutable data structures from other modules.  Import from Packages<jupyter_code>import mypackage # note that this does not import submodules in this package
try:
    mypackage.my_module2.triple_me(3)
except AttributeError as e:
    print(e)
import mypackage.my_module2 # explicit import of modules in package into package namespace
mypackage.my_module2.triple_me(3)
from mypackage import my_module2 # import submodules into current namespace
my_module2.triple_me(3)
from mypackage.my_module2 import triple_me as tripler
# import objects from submodules into current namespace
tripler(4)
from mypackage import * # imports only content/imports of __init__.py and modules
# listed in the __all__ variable in the init file into current namespace
%cat ../example_files/mypackage/__init__.py # note that my_module4 is included here, but not my_module3
try:
    my_module3.square_me(4) # not imported with from package import *
except NameError as e:
    print(e)
my_module4.divide_me_by_2(8) # imported with from package import * because module name
# is in __all__ of __init__.py<jupyter_output><empty_output><jupyter_text>Main BlockIn some cases, the same Python module id used both in imports and executed directly.In this case, a main block can be defined which is not executed during import. There is no pre-defined method to do this in Python (like the main class in Java), but the following code pattern is used:    if __name__ == '__main__':        do_something<jupyter_code>import my_module # note that the print command inside the main block is not executed 
exec(open('../code/my_module.py').read()) # antipattern: never use this as part of a program workflow
# use import instead<jupyter_output>my_module imported
executing main block<jupyter_text>Inspection  Docstrings<jupyter_code>my_module.__doc__
my_module.double_me.__doc__
my_module.Multiplier.__doc__
my_module.Multiplier.get_result.__doc__<jupyter_output><empty_output><jupyter_text>Docstrings, i.e. strings that are defined immediately after start of modules, functions, classes or methods, are not only helpful for reading code. They can be accessed on runtime with the doc dunder property.  Help Function<jupyter_code>help(my_module)
help(my_module.Multiplier)<jupyter_output>Help on class Multiplier in module my_module:

class Multiplier(builtins.object)
 |  Multiplier(i)
 |  
 |  class that creates multiplier objects with a given factor
 |  
 |  Methods defined here:
 |  
 |  __init__(self, i)
 |      Initialize self.  See help(type(self)) for accurate signature.
 |  
 |  get_result(self, x)
 |      returns factor specified with initialization multiplied by input value
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)<jupyter_text>The builtin help function automatically generates information about the objects using the signatures and docstrings.  Inspect Module<jupyter_code>import inspect
inspect.getmembers(my_module.double_me)<jupyter_output><empty_output><jupyter_text>Gives (very long) list of all available objects inside the stated object.<jupyter_code>inspect.getdoc(my_module.double_me)<jupyter_output><empty_output><jupyter_text>Docstring - equivalent to     my_module.double_me.__doc__<jupyter_code>inspect.getsource(my_module.double_me)<jupyter_output><empty_output><jupyter_text>shows source code of object<jupyter_code>module = inspect.getmodule(my_module.double_me)
module<jupyter_output><empty_output><jupyter_text>Yields the module of the stated object. Note that this is a reference to the actual module, not just its name and path as text.<jupyter_code>sig = inspect.signature(my_module.power_me)
sig.parameters<jupyter_output><empty_output><jupyter_text>Shows arguments (including type hints and standard values) of stated function / method<jupyter_code>'i' in sig.parameters<jupyter_output><empty_output><jupyter_text>Check if function has a specific parameter<jupyter_code>inspect.getfullargspec(my_module.power_me)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07340164847404768, 'fraction_numerical': 0.00478948540877701, 'mean_word_length': 5.133196721311475, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 73, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '604633', 'license': 'MIT', 'n_tokens_mistral': 2577, 'n_tokens_neox': 2539, 'n_words': 985, 'path': 'notebooks/modules_and_imports.ipynb', 'repo_name': 'lungben/python_tutorial', 'size': '8978'}