<jupyter_start><jupyter_text>A Gentle Introduction to Programming Concepts - Using Python  Introduction  Play along at homeYou can follow along and through the notebooks that we will be working through by going to the GitHub repository that we manage our content in.* Repository: https://github.com/unmrds/cc-python* Introduction/Concepts (this notebook):     * https://github.com/unmrds/cc-python/blob/master/1.1-Programming%20Concepts.ipynb* Variables and Data Types:     * https://github.com/unmrds/cc-python/blob/master/1.2-The%20Basics.ipynb* Functions (Part 2 of the Python series):     * https://github.com/unmrds/cc-python/blob/master/2.1-Functions.ipynb    You can practice and play with code in Pangeo binder environment: [![Binder](https://binder.pangeo.io/badge_logo.svg)](https://binder.pangeo.io/v2/gh/unmrds/cc-python.git/master) Why learn the basic principles of programming?* Thinking algorithmically (a key element in the process used in developing programming solutions) is a powerful problem solving skill that is reinforeced with practice. Practicing programming is great practice.     * Defining a problem with sufficient specificity that a solution can be effectively developed    * Defining what the end-product of the process should be    * Breaking a problem down into smaller components that interact with each other    * Identifying the objects/data and actions that are needed to meet the requirements of each component    * Linking components together to solve the defined problem    * Identifying potential expansion points to reuse the developed capacity for solving related problems![Problem decomposition illustration](problemDecomposition.png)* Capabilities to streamline and automate routine processes through scripting are ubiquitous    * Query languages built into existing tools (e.g. Excel, ArcGIS, Word)    * Specialized languages for specific tasks (e.g. R, Pandoc template language, PHP)    * General purpose languages for solving many problems (e.g. Bash shell, Perl, Python, C)* Repeatabilty with documentation* Scalability* Portability  Why Python?* It is available as a free and [Open Source](https://opensource.org/osd-annotated) programming language that can be installed on numerous computer systems, including Windows, Linux and the Mac OS. It can even be editited and run through a web interface such as this Jupyter Notebook. * It is a modern programming language that includes many features that make it a very efficient languageto both learn programming with and write programs in.* It is readable and expressive. * It supports a variety of development models including object-oriented, procedural and functional capabilities. * It includes a standard library of functions that support significant programming capabilities including:    * Handling email    * Interacting with and publishing web and other online resources    * Connecting with a wide variety of databases    * Executing operating system commands    * Developing graphical user interfaces* It is relatively easy to start to become productive in Python, though it still takes time and practice to becomean expert (as is the case with any programming language).The primary downside that is mentioned when discussing the choice of Python as a programming language is that asan interpreted language it can execute more slowly than traditional compiled languages such as C or C++.   Can I Play at Home?There are a variety of ways to run Python on your computer:* You may already have a version of Python installed. Many operating systems have a version of Python installed that is used for routine processes within the operating system. You can easily check to see what version of Python might already be on your computer by typing `python` at the `Command Prompt` (Windows) or in the `Terminal` (Mac OS) and seeing what response you get. If Python is installed you will typically see information about the currently installed version and then be taken to the Python command prompt where you can start typing commands. * You can install one of the available versions directly from the Python project site: https://www.python.org/downloads/. Following this installation you will be able to execute commands from the *interactive command prompt* or you can start the *IDLE* integrated development environment (IDE). * You can install a pre-packaged python system such as the Anaconda release of Python (https://www.anaconda.com/products/individual) that has both Python 2.x and 3.x versions available for download. I prefer this method as it installs a copy of Python that is separate from any previous ones on your system, and allows you to execute the (enhanced) interactive Python command prompt, **and** run the Jupyter Notebook web-based environment for writing and executing Python code. The examples that we will go through today will be executed in the Jupyter Notebook environment.   Running a Python EnvironmentOnce Python is installed on your computer you have a number of options for how you start up an environment where you can execute Python commands/code. 1. The most simple method is to just type `python` at the *Command Prompt* (Windows) or *Terminal* (Mac OS and Linux). If your installation was successful you will be taken to the interactive prompt. For example:            UL0100MAC:~ kbene$ python        Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, May 28 2015, 17:04:42)        [GCC 4.2.1 (Apple Inc. build 5577)] on darwin        Type "help", "copyright", "credits" or "license" for more information.        Anaconda is brought to you by Continuum Analytics.        Please check out: http://continuum.io/thanks and https://binstar.org        >>>2. If you would like to run the IDLE IDE you should be able to find the executable file in the folder where the Python executable installed on your system.3. If you installed the Anaconda release of Python you can type `ipython` at the *Command Prompt* (Windows) or *Terminal* (Mac OS and Linux). If you installation was successful you will be taken to an enhanced (compared with the basic Python prompt) interactive prompt. For example:        (base) karl cc-python (master) >> python        Python 3.8.8 (default, Apr 13 2021, 12:59:45)        [Clang 10.0.0 ] :: Anaconda, Inc. on darwin        Type "help", "copyright", "credits" or "license" for more information.        >>>4. If you installed the Anaconda release of Python you can type `jupyter notebook` at the *Command Prompt* (Windows) or *Terminal* (Mac OS and Linux). If your installation was successful you should see some startup messages in the terminal window and your browser should open up and display the *Jupyter Notebook* interface from where you can navigate through your system's folder structure (limited to the folder that you ran the `jupyter notebook` command from and its children), and load existing notebooks or create new ones in which you can enter and execute Python commands.  In more recent releases of the Anaconda Python distribution you can run the *Anaconda Navigator* from which you can run *Jupyter Notebooks* and other applications. **This is the interface that we are using for today's workshop**. ![Sample Jupyter Notebook Interface](images/jupyterNotebook.png)If, for some reason you can't run the jupyter notebook on your system, you can run the workshop content through a cloud service called Binder hosted by PanGeo:  [![Binder](https://binder.pangeo.io/badge_logo.svg)](https://binder.pangeo.io/v2/gh/unmrds/cc-python.git/master)  Getting HelpThere are a number of strategies that you can use for getting help with specific Python commands and syntax. First and foremost you can access the Python [documentation](https://docs.python.org/3/index.html) which will default to the most recent Python 3.x version that is in production, but from which (in the upper left corner of the page) you can select other Python versions if you are not using the version referenced by the page. Looking at and working through some of the materials in the Python [tutorial](https://docs.python.org/3/tutorial/) is also a great way to see the core Python capabilities in action. In some cases you can find quite a few useful and interesting resources through a resonably crafted Google search: e.g. for [`python create list`](https://www.google.com/search?client=safari&rls=en&q=python+create+list&ie=UTF-8&oe=UTF-8). You can also get targeted help for specific commands or objects from the command prompt by just using the `help()` function. Where you put the name of the command or object between the parentheses `()`. For example:    >>>help(print) and    >>>help(str)and    >>>myVar = [1,2,3,4,5]    >>>help(myVar)<jupyter_code>help(print)
help(str)
myVar = [1,2,3,4,5]
help(myVar)<jupyter_output>Help on list object:

class list(object)
 |  list(iterable=(), /)
 |  
 |  Built-in mutable sequence.
 |  
 |  If no argument is given, the constructor creates a new empty list.
 |  The argument must be an iterable if specified.
 |  
 |  Methods defined here:
 |  
 |  __add__(self, value, /)
 |      Return self+value.
 |  
 |  __contains__(self, key, /)
 |      Return key in self.
 |  
 |  __delitem__(self, key, /)
 |      Delete self[key].
 |  
 |  __eq__(self, value, /)
 |      Return self==value.
 |  
 |  __ge__(self, value, /)
 |      Return self>=value.
 |  
 |  __getattribute__(self, name, /)
 |      Return getattr(self, name).
 |  
 |  __getitem__(...)
 |      x.__getitem__(y) <==> x[y]
 |  
 |  __gt__(self, value, /)
 |      Return self>value.
 |  
 |  __iadd__(self, value, /)
 |      Implement self+=value.
 |  
 |  __imul__(self, value, /)
 |      Implement self*=value.
 |  
 |  __init__(self, /, *args, **kwargs)
 |      Initialize self.  See help(type(self)) for accurate sign[...]<jupyter_text>Try It YourselfType in the help command in a code box in Jupyter Notebook for a few of the following commands/objects and take a look at the information you get:* `dict` - e.g. `help(dict)`* `print`* `sorted`* `float`For some commands/functions you need to import the module that that command belongs to. For example:        import os        help(os.path)Try this pair of commands in a code window in your Jupyter Notebook or interactive terminal.<jupyter_code># type your help commands in the box and 
# execute the code in the box by typing shift-enter 
# (hold down the shift key while hitting the enter/return key)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07185284537267676, 'fraction_numerical': 0.008334930063230503, 'mean_word_length': 4.16271018793274, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 31, 'https://': 15, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '190891', 'license': 'Apache-2.0', 'n_tokens_mistral': 2739, 'n_tokens_neox': 2633, 'n_words': 1371, 'path': '1.1-Programming Concepts.ipynb', 'repo_name': 'unmrds/cc-python', 'size': '10438'}