<jupyter_start><jupyter_text>Modules and Packages__Modules__: Normally Module is a single Python source file. Its a basic tool for organizing code. Represented by module object.It is loaded with import statement. __Packages__: Packages in Python is just a special type of module.It can contain other modules including other packages.So packages are a way to define hierarchies of modules in Python. This allows to group modules with similar functionality together in a ways that express their cohesiveness.Packages provide high level structure to Python code.Packages are generally represented by directories in the file system while modules are represented by single files.<jupyter_code>import urllib
import urllib.request
type(urllib)
type(urllib.request)<jupyter_output><empty_output><jupyter_text>In the above case, request is nested inside urllib. Hence urllib is a package and request is a module.Here only urllib is bound to a name, so it is not possible to access request sub-module directly.When imported this way access to the sub-module via a fully qualified hierarchical module name.<jupyter_code># import a sub-module directly using the from import syntax.
# This only binds request to a name in the local namespace. The parent urllib package will have been imported
# but won't be directly accessible via the urllib name.
from urllib import request
request # The sub-module knows its own hierarchical module name
# __path__ attribute is a list of file system paths indicating where urllib searches to find nested modules.
# This hints at the nature of the distinction between packages and modules.
# In Python3 prior to version 3.3, __path__ was just a single string. Not a list.
urllib.__path__
urllib.request.__path__<jupyter_output><empty_output><jupyter_text>Locating ModulesWhen python encounter import statement, it looks on the file system for the corresponding Python source fileand loads that code. But how does Python knows where to look? Python checks the path attribute of the standardsys module, commonly referred to as sys.path.This is a list of directories.Python checks each entry in sys.pathobject in order until it finds the appropriate Python source file i.e module.In case there is no match in alldirectories in sys.path, ImportError is raised.This can be modified by adding new path to sys.path using append() method as sys.path merely a list object.Sometimes this is the best way to make code available to Python.There is an another way to add entries in sys.path, that does not require direct manipulation of sys.path.The PYTHONPATH environment variable is a list of paths that are added to sys.path when Python starts.The format ofPYTHONPATH is same as PATH on the platform. In windows its a ; separated list of directories and in linux/macits a : separated list of directories.<jupyter_code>import sys
sys.path<jupyter_output><empty_output><jupyter_text>Creating packagesTo create a module, create a python source file in a directory contained in sys.path. Creating package is not much different.Package can be created by following below ways -1. First create a package's root directory. This root directory needs to be in some directory on sys.path.2. Create a file called __init__.py in that root directory.This is called package init file.This makes a package a module.__init__.py can be and often is empty.Its presence alone suffices to establish the package.Since PEP420 which was introduced in Python 3.3, __init__.py file is not technically required for packages any more.This is actually required for earlier version of python 3.3.Furthermore __init__.py provides powerful tool for package initialization.But it is recommended to include __init__.py file when possible, because explicit is better than implicit.The existance of package initializationfile is an unambiguous signal that a Python developer intend for a direcory to be a package.<jupyter_code>%%bash
mkdir -p demo_reader
touch demo_reader/__init__.py
%%writefile demo_reader/__init__.py
print("demo_reader is being imported!")
import demo_reader # Successfully imported and executed __init__.py file
# demo_reader is a module even though on the file system the name demo_reader refers to a directory.
# A package is a directory containing __init__.py file
demo_reader.__file__<jupyter_output><empty_output><jupyter_text>Working with package and subpackageLets create a python source file called multireader.py inside demo_reader.change the content of demo_reader/__init__.py file as " demo_reader/__init__.py".<jupyter_code>%%writefile demo_reader/__init__.py
# demo_reader/__init__.py
import demo_reader.multireader
rd = demo_reader.multireader.MultiReader('demo_reader/__init__.py')
print(rd.read())
rd.close()
%%bash
mkdir -p demo_reader/compressed
touch demo_reader/compressed/__init__.py<jupyter_output><empty_output><jupyter_text>Added two more python source files gzipped.py and bzipped.py inside demo_reader/compressed subpackage.Lets import them.<jupyter_code>import demo_reader
import demo_reader.multireader
import demo_reader.compressed
import demo_reader.compressed.gzipped
import demo_reader.compressed.bzipped
%%bash
# Creating one sample file with bz2 compression
python -m demo_reader.compressed.bzipped sample.bz2 data compressed with bz2
%%bash
# Creating one sample file with gz compression
python -m demo_reader.compressed.gzipped sample.gz data compressed with gz
# Let's read the files
from demo_reader.multireader import MultiReader
bz = MultiReader('sample.bz2')
gz = MultiReader('sample.gz')
print(bz.read())
print(gz.read())
bz.close()
gz.close()<jupyter_output>data compressed with bz2
data compressed with gz<jupyter_text>Relative importsThere are two ways the modules can be imported.1. Absolute import - specify all of the ancestor modules of any module you want to import  Example - import demo_reader.compressed.gzipped  from demo_reader.compressed import gzipped1. Relative import - mention equal number of dots(.) with the number of ancestor packages before the module namethat is doing the import.Instead of specifying imports with absolute paths from the root of package tree, it is specified relative to the importing module.Hence relative imports.It can only be used with the "from module import names" form of import.Relative imports also works only to import modules within the current top level package.Below are some of the benefits of relative imports.- Reduces typing in deeply nested package structure- Promote a certain form of modifiability since they allow you a principle, to rename top-level and subpackages insome cases.  Example - from . import gzipped  import .gzipped - Will raise ImportErrorIt is advisable to use absolute import.<jupyter_code>%%bash
mkdir -p demo_reader/util
touch demo_reader/util/__init__.py<jupyter_output><empty_output><jupyter_text>Created one more subpackage util inside demo_reader and added __init__.py file.  \__all\__ attributeThis is a module level attribute.This lets you control which attributes are imported when from module import * form of import is being used.If \__all\__ is not specified then from module import * imports all public names those without leading underscores from the imported module.This module attribute must be a list of strings and each string indicates a name which will be imported when the * syntax is used.<jupyter_code>from pprint import pprint
pprint(locals()) # baseline display of the local variables<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.04652098136479421, 'fraction_numerical': 0.0025472583456227376, 'mean_word_length': 5.40893470790378, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '28706', 'license': 'MIT', 'n_tokens_mistral': 1934, 'n_tokens_neox': 1863, 'n_words': 953, 'path': 'OrganizingProgramms/modules_packages.ipynb', 'repo_name': 'mithunpal989/pystart', 'size': '7459'}