<jupyter_start><jupyter_text>Workshop 4: cartopy and best practices Part II: Best PracticesHere I dump my entire accumulated wisdom upon you, not so much hoping that you know it all by the end, but that you know of the concepts and know what to search for. I realize that many lessons will be learned the hard way.  1. Technical tips jupyter lab tips||||--- |--- || `Option/Alt` + drag | multi-line selection || `Ctrl/Cmd` + `X`/`C`/`V` | cut/copy/paste lines|| `Cmd` + `?`| comment out line | Linux and SSHThe science faculty has its own cluster: `gemini.science.uu.nl`. There you can perform heavier, longer computations.You can tunnel into the cluster by typing```ssh (your_solis_id)@gemini.science.uu.nl```you are then prompted for your password.Note that working `ssh` is not a very stable connection: `broken pipe` error. All active commands are interrupted if your connection is severed. One way around this is to use the `screen` functionality.Many advanced text editors let you work remotely. For example, Visual Studio Code has a `Remote-SSH` extension that lets you work on the remote machine as if it were local.Some useful commands when navigating the terminal and submitting jobs on gemini.| command              | effect || ---                  | --- || `pwd`                | print working directory, where  are you in the file system|| `cd`                 | change directory, without: to home directory; `..` for level up|| `ls`                 | list content of folder || `cp`/`mv`            | copy/move file || `rm`                 | remove file (CAUTION: this is permanent) || `grep (word) (file)` | search for `word` in `file` || `touch (file)`       | create empty `file` || `top`                | monitor processes and system resources || `Up(arrow)`          | previous command| `Ctrl + C`           | cancel || `Ctrl + R`           | search command history || `qsub (your_job.sh)` | submits `your_job.sh` bash script to queue* || `qstat`              | check on your jobs in the batch queue || `qdel (job-id)`      | deletes job with id `job-id` in queue |\* There are 48 job slots (12 nodes with 4 cores) with 4Gb of memory per core Project organizationOnce your projects contain more than a few results (e.g. for SOAC and your thesis), it is worthwhile organizing. A common structure has proven useful for most cases:```| project_name  README            (markdown or simple text file describing your project and its structure)  | data            (when working with big data this may be external to the project folder)    | raw_data      (never touch these files)      | processed     (derived files)  | doc             (includes `requirements.txt` with python environment description)  | src             (all your [well-documented] code: .py, .ipynb, ... files)  | results         (all figures, maybe in subfolders)```A well organized project helps you current and future self as well as anyone else looking at the results. It is thus an important step for reproducibility. colormapsBe conscious about the colors and colormaps you use! Colors can hide or emphasize data, which can be used to improve your presentation. Read this [short blog post](https://jakevdp.github.io/blog/2014/10/16/how-bad-is-your-colormap/) and learn why the `jet`colormap is bad.A main consideration for accessibility of your results must be the color blindness which afflicts quite a few people. See [ColorBrewer](https://colorbrewer2.org) for colors that work well together.The `cmocean` package adds some [beautiful, well-designed colormaps for oceanography](https://matplotlib.org/cmocean/) to the standard matplotlib colormaps.  2. Fundemental programming guidelines Understanding Python error messagesUnderstanding Python errors can be daunting at first, especially if they are very long. But don;t despair, after some practive you become better at interpreting them and will find aathem helpful in pinning down the problem. The general idea is that Python shows you the steps from the line where you called the offending line all the way down to the line in the file that raised an error. Often, the most important part of the error message is located at the end. DRY: Don't repeat yourselfIt is almost always a sign of bad programming if you have to repeat a line several times. It clutters the code and makes the code harder to maintain. simplify codeInstead of writing one huge function, __break your functions down into logical component functions__. This will save you many headaches when hunting for bugs. coding stylePython is very forgiving towards your code writing style. Just because it runs wihtout errors does not mean is well written, though.How to write good, readable python code is laid out in the __[PEP8 Style Guide for Python Code](https://pep8.org/)__. Read it and try to adhere to it. reuse codeOnce you have iterated to stable code (and you want to share it across jupyter notebooks), you should put it in a separate `.py` file. You can __import functions from `.py` files__ simply as `from file import function`. Defensive programmingDefensive programming is a programming philosophy that tries to guard against errors and minimize time spent on solving bugs. The fundamental idea is that of __unit testing__: you break the code into the small into the small steps (functions) and then test whether they give the expected (known) results for simple test cases. One would write a function with known input and output. Unit testing can be automated and this is known as "continuous integration (CI)" (integrated in GitHub,for example).This approach works well for traditional software development with fixed goals, but it is not always suited to scientific programming as the goals shift with new knowledge.However, the principle of defensive programming is still very valuable. A __simple and easy-to-implement version of this defensive philosophy__ can be implemented by using the `assert` statement often (this is not exactly unit testing). It checks whether a statement is true and can raise a custom error.<jupyter_code>import numpy as np

def calc_circumference(radius):
    """ simple example function to calculate circle cirumference """
    return 2*np.pi*radius

def calc_circumference2(radius):
    """ simple example function to calculate circle cirumference """
    assert type(radius) in [float, int], 'radius must be a number'
    return 2*np.pi*radius
# this works as expected
calc_circumference(1)
# this does not work and Python tells us why with its own error message
calc_circumference('hello')
# this does not work and our message tells us why
calc_circumference2('hello')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06744779930899805, 'fraction_numerical': 0.003755445395823945, 'mean_word_length': 3.8421818181818184, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '22675', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1661, 'n_tokens_neox': 1586, 'n_words': 924, 'path': '4b_best_practices.ipynb', 'repo_name': 'AJueling/python_climate_physics', 'size': '6657'}