<jupyter_start><jupyter_text>Python for EpisAll these files are here: https://github.com/kialio/py4Epis  You should be able to install python and run them after this.  Feel free to ask me questions now or later.* I'm going to give some background and then some high level examples as fast as I can...* There are many examples on the web.      * There are even some for SAS Users. Here's a good one: https://github.com/RandyBetancourt/PythonForSASUsers  Objectives* Introduce you to the Python language* Show its utility in your research life* (I'm not going to show how to install python or get it going on your machine, if you want to get going quickly, check out conda: https://docs.conda.io/en/latest/) Credits* Borrowed heavily from https://github.com/profjsb/python-bootcamp   Who I Am <NAME>[@oldmanperkins](https://twitter.com/oldmanperkins)https://github.com/kialioI work at NASA/GSFC (here as a private citizen) and work on developing next generation gamma-ray instrumentation ([AMEGO](https://asd.gsfc.nasa.gov/amego/), [BurstCube](https://asd.gsfc.nasa.gov/burstcube/)).  I use python to analyze data, control hardware, figure out budgets (I try to get data out of the excel spreadsheets my financial people give me as fast as possible), make pretty plots...  Introduction* What is Python?* Why Python?* Getting Started...  What is Python?>Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. Python's simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance. Python supports modules and packages, which encourages program modularity and code reuse. The Python interpreter and the extensive standard library are available in source or binary form without charge for all major platforms, and can be freely distributed.https://www.python.org/doc/essays/blurb/  What is Python?            interpreted        no need for a compiling stage                object-oriented        programming paradigm that uses objects (complex data structures with methods)                high level        abstraction from the way machine interprets & executes                dynamic semantics        can change meaning on-the-fly                built in        core language (not external)                data structures        ways of storing/manipulating data                script/glue        programs that control other programs                typing        the sort of variable (int, string)                syntax        grammar which defines the language                library        reusable collection of code                binary        a file that you can run/execute      Development History* Started over the Christmas break 1989, by <NAME>* Developed in the early 1990s* Name comes from Monty Python’s Flying Circus* Guido is the Benevolent Dictator for Life (BDFL), meaning that he continues to oversee Python’s development.  Development History* Open-sourced development from the start (BSD licensed now)    * http://www.opensource.org/licenses/bsd-license.php* Relies on large community input (bugs, patches) and 3rd party add-on software* Version 2.0 (2000), 2.6 (2008), 2.7 (2010). * Version 2.7.X is reaching end of life this year.* Version 3.X (2008) is not backward compatible with 1.X & 2.X. If you're starting now, use 3.X.  Why Python  Some of the AlternativesI've used almost all of these at some point C, C++, Fortran*Pros: great performance, backbone of legacy scientific computing codes*`Cons: syntax not optimized for causal programming, no interactive facilities, difficult visualization, text processing, etc. ` Mathmatica, Maple, Matlab, IDL (and I guess SAS, SPSS,...)*Pros: interactive, great visuals, extensive libraries*`Cons: costly, proprietary, unpleasant for large-scale programs and non-mathematical tasks.` Perlhttp://strombergers.com/python/  Why Python* **Free** (BSD license), highly portable (Linux, OSX, Windows, lots...)* **Interactive** interpreter provided.* Extremely readable syntax (**“executable pseudo-code”**). * **Simple**: non-professional programmers can use it effectively    * great documentation    * total abstraction of memory management * Clean object-oriented model, but **not mandatory**.* Rich built-in types: lists, sets, dictionaries (hash tables), strings, ... * Very comprehensive standard library (**batteries included**) * Standard libraries for IDL/Matlab-like arrays (NumPy)* Easy to wrap existing C, C++ and FORTRAN codes.  Why Python Amazingly Scalable* Interactive experimentation * build small, self-contained scripts or million-lines projects. * From occasional/novice to full-time use (try that with C++).* Large community of open source packages The Kitchen Sink (in a good way)* really can do anything you want, with impressive simplicity Performance, if you need it* As an interpreted language, Python is slow.* But...if you need speed you can do the heavy lifting in C or FORTRAN ...or you can use a Python compiler (e.g., Cython)   My Group Uses Python For Providing a comprehensive analysis framework for Fermi LAT data(I was forced into using python...)* Interface to the low-level (c++) code - Interactive data analysis* Scripting* Developing new analysis techniques* Adding features to static code quickly* Providing high-level analysis tools (data selection, statistical testing, simulation development, plot making, and so on and so forth)* Validation and Testing What I Use Python For* Data reduction & Analysis    * processing FITS images quickly    * wrapping around 3rd party software* A Handy & Quick Calculator* Prototyping new algorithms/ideas* Making plots for papers* Notebooking (i.e. making me remember stuff)    * see the iPython sessions later* Writing Presentations (these slides)* Controling hardware  Python is everywherehttps://wiki.python.org/moin/OrganizationsUsingPython Applications are Numerous* Scripting and Programing* GUI's* Web Development* Interactive Notebooks (see later)* Visualization* Parralelization* Animation* And so on...  Firing up the interpreter in OSX Go to Utilities->Terminal***`[pyuser@pymac ~]$ python``Python 3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:07:37)``[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin``Type "help", "copyright", "credits" or "license" for more information.``>>>`***The details might be different (different version, different compiler).  You could also use iPython:***`[pyuser@pymac ~]$ ipython ``Python 3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:07:37)``Type 'copyright', 'credits' or 'license' for more information``IPython 7.8.0 -- An enhanced Interactive Python. Type '?' for help.``In [1]:`***  Firing it up in other OS's like WindowsInstall python via Conda and follow the directions.    Creating Python Programs and Scripts* Basically, any raw text editor will do    * Lot's of the basic ones will do syntax highlighting (reccomended)* You create a python program or script file in the text editor and usually save it with a *.py extension* There are lots of programs out there that can do this and have fancy markup.    * I'm still using emacs    * List: https://wiki.python.org/moin/PythonEditors    * Make sure it saves as raw text (and not rich text or something else)  Last Thing: The Notebook* The jupyter Notebook is a powerful tool* You **will** want to use it.* To start it up from the terminal type`jupyter notebook`and a browser window should open that looks like this![notebook](iPython_Notebook.png)<jupyter_code>%matplotlib inline
from matplotlib import pyplot as plt
import numpy as np
plt.xkcd()
plt.figure(figsize=(16,8))
x = np.arange(10)
plt.plot(x,x+0.5*x*x)
plt.xlabel('Years Since Release')
plt.ylabel('Interest in Python')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08059701492537313, 'fraction_numerical': 0.01082089552238806, 'mean_word_length': 4.161103979460847, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 12, 'https://': 11, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '513296', 'license': 'MIT', 'n_tokens_mistral': 2164, 'n_tokens_neox': 1974, 'n_words': 1012, 'path': '00_Intro.ipynb', 'repo_name': 'kialio/py4Epis', 'size': '8058'}