<jupyter_start><jupyter_text>Python Application BuildingAY 250, Spring 2013, Fall 2016; <NAME><jupyter_code>!pip install simplejson --target=/tmp/
!ls /tmp/simple*<jupyter_output>/tmp/simplejson:
__init__.py                     encoder.py
[34m__pycache__[m[m                     ordered_dict.py
[31m_speedups.cpython-35m-darwin.so[m[m scanner.py
compat.py                       [34mtests[m[m
decoder.py                      tool.py

/tmp/simplejson-3.10.0.dist-info:
DESCRIPTION.rst METADATA        WHEEL           top_level.txt
INSTALLER       RECORD          metadata.json<jupyter_text>`...dist-info` directory is part of the "standard infrastructure to manage project distributions installed on a system, so all tools that are installing or removing projects are interoperable." [PEP-376](https://www.python.org/dev/peps/pep-0376/id19)[Note the .whl extension...see ["Wheel vs EGG"](https://packaging.python.org/wheel_egg/) and [Wheel info](https://wheel.readthedocs.io/en/latest/)]Now, you can have Python know about your special installation directory by modifying your `PYTHONPATH` environment variable in your `.bashrc`, `.cshrc`, or `.tcshrc` file:```bashBASH Style: export PYTHONPATH=/tmp/simplejson:$PYTHONPATHCSH Style:setenv PYTHONPATH /path/to/my_choice:$PYTHONPATH```  Getting and Installing Packages with `setup.py` Sometimes `conda` and `pip` cannot find a codebase you're trying to install. In this case you'll need to do it yourself using a tarball and a `setup.py` file. This is the most straightforward way to get packages: download them from the developer’s website and hope that they’ve followed the standard conventions.There is a standard Python package distribution scheme using `distutils2` and `setup.py` files...more on that later.Basic workflow of installing a package with `setup.py`:```bash$ cd [folder with package and setup.py file]$ sudo python setup.py install    [ progress report ... ]$ Finished processing dependencies for [package]    [if you want more info, there are several options to modify]$ python setup.py --help install```To do a custom installation directory (if you dont have sudo, e.g.):```bash {-- on unix --}$ python setup.py install --home  {-- on windows --}$ python setup.py install --prefix “my_choice”```  Managing Packages - `virtualenv`/`conda` environments * Open Source software is constantly changing - how do you protect working code against future updates?* Or, what if there is a beta release of a package you want to try, but you don’t want to fully commit yet?* `virtualenv` and `conda -n` creates a local, self-contained, and totally separate python installation.* Use it to create a local Python ecosystem, separate from your computer’s main system, so that you can do what you want in one without affecting the other.  `virtualenv` installing:<jupyter_code>!pip install --upgrade virtualenv<jupyter_output>Requirement already up-to-date: virtualenv in /Users/jbloom/anaconda/envs/seminar/lib/python3.5/site-packages<jupyter_text>Creating a new environment: [![asciicast](https://asciinema.org/a/cxhbq03rtd39ez8sy5scwqk1w.png)](https://asciinema.org/a/cxhbq03rtd39ez8sy5scwqk1w) (https://asciinema.org/)During a shell session, you can source this environment so that it runs as the default:```bash$ source LocalPython/bin/activate(LocalPython)$[ pip and python commands now point to new environment ](LocalPython)$ which pythonLocalPython/bin/python```We can get out of the enviroment:```bash deactivate LocalPython```Just delete to remove environment:```bashrm -r LocalPython```  conda -n http://www.continuum.io/blog/conda<jupyter_code>!conda info
!conda info -e
!conda search simplejson
!conda create -n sj381-test simplejson=3.8.1 -y
!conda info -e
!ls /Users/jbloom/anaconda/envs/sj381-test/bin<jupyter_output>[31m2to3[m[m             [35measy_install-2.7[m[m [35mpython-config[m[m    [31msqlite3[m[m
[35mactivate[m[m         [31midle[m[m             [35mpython2[m[m          [31mtclsh8.5[m[m
[31mc_rehash[m[m         [31mopenssl[m[m          [35mpython2-config[m[m   [31mwheel[m[m
[35mconda[m[m            [31mpip[m[m              [31mpython2.7[m[m        [31mwish8.5[m[m
[35mdeactivate[m[m       [31mpydoc[m[m            [31mpython2.7-config[m[m
[31measy_install[m[m     [35mpython[m[m           [31msmtpd.py[m[m<jupyter_text>We could make this environment the default if we want to:```bashexport PATH=~/anaconda/envs/sj381-test/bin:$PATH```And if we want to remove that environment:```bashconda remove -n sj381-test --all```<jupyter_code>!conda remove -n sj381-test --all -y
!conda info -e<jupyter_output>Warning: could not import binstar_client ('args' object has no attribute 'site')# conda environments:
#
_test                    /Users/jbloom/anaconda/envs/_test
bootcamp                 /Users/jbloom/anaconda/envs/bootcamp
dd                       /Users/jbloom/anaconda/envs/dd
i13                      /Users/jbloom/anaconda/envs/i13
ipykernel_py2            /Users/jbloom/anaconda/envs/ipykernel_py2
numpy15                  /Users/jbloom/anaconda/envs/numpy15
py3k                     /Users/jbloom/anaconda/envs/py3k
seminar               *  /Users/jbloom/anaconda/envs/seminar
root                     /Users/jbloom/anaconda<jupyter_text>Command Line Parsing`python myawesomeprogram.py -o option1 -p parameter2 -Q -R` **Goal**: build a command-line 'standalone' codebase in Python, w/ CL options & keywords  **Solution**: `argparse`, which has been built in to Python 2.7 & above (if you don’t have it, you can get it with `pip argparse`) * Allows for  user-friendly command line interfaces, and leaves it up to the code to determine what it was the user wanted.* Also automatically generates help & usage messages and issues errors when invalid arguments are provided.(Note on `optparse`: being replaced in favor of `argparse`)<jupyter_code>import argparse<jupyter_output><empty_output><jupyter_text>Setting up a parser * First step for `argparse`: create parser object & tell it what arguments to expect. * It can then be used to process the command line arguments on runtime* Parser class: `ArgumentParser`. Takes several arguments to set up the description used in the help text for the program & other global behaviors     See  http://www.doughellmann.com/PyMOTW/argparse/<jupyter_code>%%writefile myfile.py
#!/usr/bin/env python
import argparse
parser = argparse.ArgumentParser(description='Sample Application')
print("hi")
%run myfile.py<jupyter_output>hi<jupyter_text>Defining Arguments & Parsing* Arguments can trigger different actions, specified by the action argument to `add_argument()`. * Several supported actions.* Once all of the arguments are defined, you can parse the command line by passing a sequence of argument strings to `parse_args()`. * By default, arguments are taken from `sys.argv[1:]`, but you can also pass your own list.<jupyter_code>%%file argparse_action.py
import argparse
parser = argparse.ArgumentParser(description='Sample Application')
parser.add_argument('required_arg_1', help='This positional argument is required')
parser.add_argument('required_arg_2', help='This positional argument is also required')
parser.add_argument('-s', action='store', dest='simple_value',
                    help='Store a simple value')
parser.add_argument('-c', action='store_const', dest='constant_value',
                    const='value-to-store',
                    help='Store a constant value')
parser.add_argument('-t', action='store_true', default=False,
                    dest='boolean_switch',
                    help='Set a switch to true')
parser.add_argument('-a', action='append', dest='collection',
                    default=[],
                    help='Add repeated values to a list',
                    )
parser.add_argument('-A', action='append_const', dest='const_collection',
                    const='value-1-to-append',
                    default=[],
                    help='Add different values to list')
parser.add_argument('-B', action='append_const', dest='const_collection',
                    const='value-2-to-append',
                    help='Add different values to list')
parser.add_argument('--version', action='version', version='%(prog)s 1.0')

results = parser.parse_args()
print('required_args    =', results.required_arg_1, results.required_arg_2)
print('simple_value     =', results.simple_value)
print('constant_value   =', results.constant_value)
print('boolean_switch   =', results.boolean_switch)
print('collection       =', results.collection)
print('const_collection =', results.const_collection)
%run argparse_action.py --help<jupyter_output>usage: argparse_action.py [-h] [-s SIMPLE_VALUE] [-c] [-t] [-a COLLECTION]
                          [-A] [-B] [--version]
                          required_arg_1 required_arg_2

Sample Application

positional arguments:
  required_arg_1   This positional argument is required
  required_arg_2   This positional argument is also required

optional arguments:
  -h, --help       show this help message and exit
  -s SIMPLE_VALUE  Store a simple value
  -c               Store a constant value
  -t               Set a switch to true
  -a COLLECTION    Add repeated values to a list
  -A               Add different values to list
  -B               Add different values to list
  --version        show program's version number and exit<jupyter_text>* `store`: Save the value, after optionally converting it to a different type (default)* `store_const`: Save the value as defined as part of the argument specification, rather than a value that comes from the arguments being parsed* `store_true`/`store_false`: Save the appropriate boolean value* `append`: Save the value to a list.  Multiple values are saved if the argument is repeated* `append_const`: Save a value defined in the argument specification to a list* `version`: Prints version details about the program and then exits There's also some other ways of doing arguement parsing that you might want to explore: * Click: http://click.pocoo.org/5/quickstart/basic-concepts<jupyter_code>!pip install click
%%writefile hello.py
import click
@click.command()
@click.option('--count', default=1, help='number of greetings')
@click.argument('name')
def hello(count, name):
    for x in range(count):
        click.echo('Hello %s!' % name)

if __name__ == '__main__':
    hello()
!python hello.py --help
!python hello.py --count=4 Cal<jupyter_output>Hello Cal!
Hello Cal!
Hello Cal!
Hello Cal!<jupyter_text>Breakout! * Go to the breakout folder in: `../Breakouts/01_Versioning_Application_Building/`* Work on the file `breakout1.py`.  Do not move or modify the other files, in the other folders, but you will need to use them.  (You may add files to these directories, if necessary)* Build up a command line parser which allows the user to specify: - how many datapoints to generate - whether to plot with a filled in histogram or an outlined one - the title of the plot - And then have the plot be generated.* We want to be able to run a command like:```bashpython breakout1.py -t -n 200 -T "My Awesome Title"```<jupyter_code>cd /Users/jbloom/Classes/python-seminar/Breakouts/01_Versioning_Application_Building/sol
!cat plotting/histOutline.py
%matplotlib inline
%run breakout1_solution.py -t
%run breakout1_solution.py -n 200 -T "My Awesome Title"
%run breakout1_solution.py --help<jupyter_output>usage: breakout1_solution.py [-h] [-t] [-T TITLE] [-n NUM] [--version]

Sample Application

optional arguments:
  -h, --help  show this help message and exit
  -t          Fill the histogram?
  -T TITLE    Title
  -n NUM      Number of data points
  --version   show program's version number and exit
{'fraction_non_alphanumeric': 0.10826671202585474, 'fraction_numerical': 0.014798435108011567, 'mean_word_length': 3.677406523468576, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 6, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '630821', 'license': 'MIT', 'n_tokens_mistral': 3731, 'n_tokens_neox': 3432, 'n_words': 1157, 'path': '01_Versioning_Application_Building/00_Application Building.ipynb', 'repo_name': 'thomascarey/python-ay250-lectures', 'size': '11762'}