<jupyter_start><jupyter_text>Notebook by <NAME> (<EMAIL>)  Which figure would you choose to include in a paper on the Air Temperature in the Arctic? ![Airtemps.png](Airtemps.png)  Using color in visualizing ocean and atmosphere data  Introduction to colors, visualizationIn meteorology and oceanography we often try to understand processes that are hard to see with the naked eye. In order to find patterns and explore the (model)data we have collected, we almost always use artificial colors. In this notebook we are going to explore what options we have in **matplotlib** to use color correctly and creatively.To create effective figures, it is paramount to realise what your data are and what your question is: In research, we formulate some sort of **question**, gather **data** and try to find what **story** we can tell or what answers we can give by interpreting the data. A good **visualization** helps us and our audience in the correct interpretation of the patterns and distributions in the data. The spark for this talk is the ongoing online anger in the form of ranting blogposts on the internet directed against scientists using the **'jet'** or **'rainbow'** colormap. So, why are people claiming it misrepresents the data?<NAME> lays out the whole problem in a very nice blog at Visually (https://visual.ly/blog/rainbow-color-scales/). I have adapted his explanation slightly and added some concepts from another nice series by <NAME> from NASA (https://earthobservatory.nasa.gov/blogs/elegantfigures/2013/08/05/subtleties-of-color-part-1-of-6/):- **"Divisions between hues create edges in the visualization"**Jet uses the most diverse range of colors or hues possible to make it easier to differentiate between different colors. However, because not all colors are perceived the same through human eyes, we may see discrete edges where there is a continuum. These figures by Jake Vanderplas show how especially cyan and yellow can be perceived as banded colors. As seen in the grayscale version of the colormap, the brightness of the colors show a complex pattern that may change the way we see the data distributed. This also makes it impossible to assess when printed in black and white.![jakevdp_jet.jpg](jakevdp_jet.jpg)![jet_grayscale.png](jet_grayscale.png)Source: https://jakevdp.github.io/blog/2014/10/16/how-bad-is-your-colormap/- **"Yellow is special"**![cone_sensitivity.png](cone_sensitivity.png)Source: https://en.wikipedia.org/wiki/File:Cone-response.svg- **"The spectral order of hues has no inherent meaning"**![spectralorder.png](spectralorder.png)Source: https://visual.ly/blog/rainbow-color-scales/- **"Detail is actually harder to see in a rainbow"**If we look at subplot A of the figure showing the air temperature, we see that details, especially in the cyan/green/yellow regions (for example in Siberia) are very hard to see. This is because the brightness of these colours is very similar and humans can distinguish shades of black and white better than it can different hues. This is because we have way more rods than cones and the cones do not absorb a single frequency.![RainbowVsGray.png](RainbowVsGray.png)Source: https://visual.ly/blog/rainbow-color-scales/- **"Colorblind people cannot use them."**![colorblind.png](colorblind.png)Source: https://visual.ly/blog/rainbow-color-scales/  Data types and colormaps Okay, so now it is clear why the 'rainbow' colormaps are not ideal. But what kind of different options do we have? Cynthia Brewer has an entire website devoted to this (http://www.personal.psu.edu/cab38/ColorSch/SchHome.html). There are three main different types of colormaps, each suitable for different types of data:- **Qualitative** -> different **hues** for different **nominal groups** - **Sequential** -> **quantitative** data represented by ordered/sequential **lightness steps**, multiple hues possible - **Diverging** -> essentially two sequential colormaps combined, very useful for **quantitative** data **diverging** from a **critical midpoint/median** ![data_types.png](data_types.png)Source: http://www.personal.psu.edu/cab38/ColorSch/Schemes.html You will also see this separation of different types of colormaps in matplotlib. Take a good look at your data and decide what you think is suitable. What do you think is the most suitable type of colormap for air temperature?**A**: Qualitative**B**: Sequential**C**: Diverging Now let's return to criticism that the rainbow colormap is bad at separating details. The conclusion was that differences over very **small distances** are better seen with varying brightness than varying hue. However, comparing differences over **longer distances** is actually harder with monotonic colormaps that change in brightness. This is caused by a phenomenon known as simultaneous contrast. The same shade of gray looks lighter surrounded by a dark background and darker surrounded by a light background. This makes it hard to compare purely sequential colormaps over longer distances. To help tackle this, you can use **sequential** colormaps with **multiple hues** such as in subplot **C** for the air temperature data.![simultaneous_contrast.png](simultaneous_contrast.png)Source: http://ccom.unh.edu/sites/default/files/publications/Ware_1988_CGA_Color_sequences_univariate_maps.pdf  Using packages and functions to customize your colorsWhen mapping data, matplotlib has several options that handle colormaps and colors very differently, e.g. **imshow**, **contourf** and **pcolormesh**. I think contourf is the most straightforward to use, but I prefer pcolormesh because it tends to show the data at the correct resolution. Here, we will therefore focus on **pcolormesh**. As covered in the earlier PFL sessions on matplotlib, there are several ways to access matplotlib objects, which is no different for colormaps. Most of the basic options are covered in `matplotlib.pyplot`. Also we are going to look at the cmocean package (https://matplotlib.org/cmocean/), which has beautiful and perceptually uniform colormaps that are tailored for different types of oceanographic data.<jupyter_code>import matplotlib.pyplot as plt
import numpy as np
import xarray as xr
import cmocean<jupyter_output><empty_output><jupyter_text>Matplotlib colormap objects are used to map a scalar dataset to a colorscheme. There are different ways to do this, but the standard matplotlib colormaps contains a discrete number of RGBA colours onto which the data is mapped by a matplotlib normalization function. There are many colormaps available. To see which ones are available, go to https://matplotlib.org/tutorials/colors/colormaps.html. We can show all options by calling `plt.colormaps()`.<jupyter_code>plt.colormaps()<jupyter_output><empty_output><jupyter_text>We are going to choose the perceptually uniform **sequential** colormap called `'magma'`. The names can be used directly inside plotting functions such as `plt.contourf`, `plt.pcolormesh` or `plt.scatter` with the keyword `cmap`. If we want more control over the colormap object, we can also create it beforehand by using `plt.get_cmap`. Since the standard colormaps return a discrete number of colours, we can specify this number with the second argument.<jupyter_code>my_cmap = plt.get_cmap('magma',10)
print(my_cmap)
print(my_cmap.colors)                       # Show all RGBA colors in my_cmap
print('my_cmap color 0 = '+str(my_cmap(0))) # Individual colors are callable<jupyter_output>my_cmap color 0 = (0.001462, 0.000466, 0.013866, 1.0)<jupyter_text>A colormap maps scalar data onto different colors. To see how it does this we are going to look at how a dataset with values between 0 and 100 are mapped onto the ten colors.<jupyter_code>im = np.outer(np.ones(10), np.arange(100))  # Create a grid with data from 0 to 99 homogeneous in the vertical
print(im[0])

fig = plt.figure(figsize=(6, 1))
ax = fig.add_subplot(111)
ax.pcolormesh(im,cmap=my_cmap)              # I prefer to use pcolormesh over contourf. Here you can also directly call the colormap with cmap='magma'
plt.show()<jupyter_output>[ 0.  1.  2.  3.  4.  5.  6.  7.  8.  9. 10. 11. 12. 13. 14. 15. 16. 17.
 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35.
 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. 52. 53.
 54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71.
 72. 73. 74. 75. 76. 77. 78. 79. 80. 81. 82. 83. 84. 85. 86. 87. 88. 89.
 90. 91. 92. 93. 94. 95. 96. 97. 98. 99.]<jupyter_text>Now we are going to look at the default setting of the colormap.<jupyter_code>my_cmap = plt.get_cmap('magma')

print('The number of colors by default = '+str(len(my_cmap.colors)))


im = np.outer(np.ones(10), np.arange(100))

fig = plt.figure(figsize=(6, 1))
ax = fig.add_subplot(111)
ax.pcolormesh(im,cmap=my_cmap)
plt.show()<jupyter_output>The number of colors by default = 256<jupyter_text>If we want to **reverse** the colormap, simply add `'_r'` to the name:<jupyter_code>my_cmap = plt.get_cmap('magma_r',256)


im = np.outer(np.ones(10), np.arange(100))

fig = plt.figure(figsize=(6, 1))
ax = fig.add_subplot(111)
ax.pcolormesh(im,cmap=my_cmap)
plt.show()<jupyter_output><empty_output><jupyter_text>The cmocean package contains additional colormaps and some user-friendly tools. The colormaps are also registered in the standard matplotlib library, identifiable by the `cmo.` prefix<jupyter_code>my_cmap = plt.get_cmap('cmo.curl')
im = np.outer(np.ones(10), np.arange(100))  # Create a grid with data from 0 to 99 homogeneous in the vertical

fig = plt.figure(figsize=(6, 1))
ax = fig.add_subplot(111)
ax.pcolormesh(im,cmap=my_cmap)              # I prefer to use pcolormesh over contourf
plt.show()<jupyter_output><empty_output><jupyter_text>The tool `crop_by_percent` clips your colormap so that darker colors that may be hard to differentiate between when close together are taken out.<jupyter_code>newcmap = cmocean.tools.crop_by_percent(my_cmap, 30, which='both', N=None)

im = np.outer(np.ones(10), np.arange(100))  # Create a grid with data from 0 to 99 homogeneous in the vertical

fig = plt.figure(figsize=(6, 1))
ax = fig.add_subplot(111)
ax.pcolormesh(im,cmap=newcmap)              # I prefer to use pcolormesh over contourf
plt.show()<jupyter_output><empty_output><jupyter_text>Now let's see how the figure from the start of the presentation is created. The dataset used is from the **xarray** tutorials. It contains air temperature data over the Arctic.<jupyter_code>ds = xr.tutorial.load_dataset('rasm')
ds
### Defining figure and subplots ###
fig = plt.figure(figsize=(15,10))
fig.suptitle('Daily average Air Temperature in the Arctic', fontsize=24)

ax1 = fig.add_subplot(221)
ax1.get_yaxis().set_ticklabels([])
ax1.get_xaxis().set_ticklabels([])
ax1.text(0.05,0.05,'A',transform=ax1.transAxes, fontsize=20,weight='bold')


### Interesting settings for plot A ###                                       
ax1.set_facecolor('gray')                             # Defining a suitable facecolor can be very relevant for the effect of your colormap

cmap1 = plt.get_cmap('jet')                           # The 'despised' jet colormap with 256 colors

pcm1 = ax1.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)  # define the grid and the 2D data to be plotted
                    ,cmap=cmap1                       # Use the jet colormap
                    , vmin=-20, vmax=20)              # Set the upper and lower bound of data mapped onto the colormap
cbar1 = fig.colorbar(pcm1,extend='both')              # By default the colormap colors values above and under the bounds with the last color and pcolormesh shows these values. Show this in the colorbar with extend = 'both'
cbar1.set_label('$T_{air}$ [$\circ$C]',size=16)

### Defining subplot B ###
ax2 = fig.add_subplot(222)
ax2.get_yaxis().set_ticklabels([])
ax2.get_xaxis().set_ticklabels([])
ax2.text(0.05,0.05,'B',transform=ax2.transAxes, fontsize=20,weight='bold')


### Interesting settings for plot B ###
ax2.set_facecolor('gray')

cmap2 = plt.get_cmap('Blues_r')                       # Reverse the 'Blues' colormap
cmap2.set_over('w')                                   # Add the color white for values above the bounds in pcolormesh

pcm2 = ax2.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap2
                   , vmin=-20, vmax=20)
cbar2 = fig.colorbar(pcm2,extend='both')
cbar2.set_label('$T_{air}$ [$\circ$C]',size=16)

### Defining subplot C ###
ax3 = fig.add_subplot(223)
ax3.get_yaxis().set_ticklabels([])
ax3.get_xaxis().set_ticklabels([])
ax3.text(0.05,0.05,'C',transform=ax3.transAxes, fontsize=20,weight='bold')


### Interesting settings for plot C ###
ax3.set_facecolor('gray')

cmap3 = plt.get_cmap('cmo.thermal')                    # Use the cmocean package

pcm3 = ax3.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap3
                   , vmin=-20, vmax=20)               
cbar3 = fig.colorbar(pcm3,extend='both')               # Now there are no values below the lower bound, so only the positive side of the colorbar is extended with extend = 'max'
cbar3.set_label('$T_{air}$ [$\circ$C]',size=16)

### Defining subplot D ###
ax4 = fig.add_subplot(224)
ax4.get_yaxis().set_ticklabels([])
ax4.get_xaxis().set_ticklabels([])
ax4.text(0.05,0.05,'D',transform=ax4.transAxes, fontsize=20,weight='bold')


### Interesting settings for plot C ###
ax4.set_facecolor('gray')

cmap4 = plt.get_cmap('bwr')                          # Use a diverging colormap

pcm4 = ax4.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap4
                   , vmin=-20, vmax=20)
cbar4 = fig.colorbar(pcm4,extend='both')
cbar4.set_label('$T_{air}$ [$\circ$C]',size=16)

plt.savefig('Airtemps')<jupyter_output><empty_output><jupyter_text>Here, let's look at some more alternatives<jupyter_code>fig = plt.figure(figsize=(15,10))
fig.suptitle('Daily average Air Temperature in the Arctic', fontsize=24)

ax1 = fig.add_subplot(221)
ax1.get_yaxis().set_ticklabels([])
ax1.get_xaxis().set_ticklabels([])
ax1.set_facecolor('gray')
ax1.text(0.05,0.05,'A',transform=ax1.transAxes, fontsize=20,weight='bold')


cmap1 = plt.get_cmap('magma',8)
cmap1.set_over('w')

pcm1 = ax1.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap1
                   , vmin=-30, vmax=10)
cbar1 = fig.colorbar(pcm1,extend='max')
cbar1.set_label('$T_{air}$ [$\circ$C]',size=16)

ax2 = fig.add_subplot(222)
ax2.get_yaxis().set_ticklabels([])
ax2.get_xaxis().set_ticklabels([])
ax2.set_facecolor('gray')
ax2.text(0.05,0.05,'B',transform=ax2.transAxes, fontsize=20,weight='bold')


cmap2 = plt.get_cmap('bwr')

pcm2 = ax2.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap2
                   , vmin=-20, vmax=20)
cbar2 = fig.colorbar(pcm2,extend='both')
cbar2.set_label('$T_{air}$ [$\circ$C]',size=16)

ax3 = fig.add_subplot(223)
ax3.get_yaxis().set_ticklabels([])
ax3.get_xaxis().set_ticklabels([])
ax3.set_facecolor('gray')
ax3.text(0.05,0.05,'C',transform=ax3.transAxes, fontsize=20,weight='bold')


cmap3 = plt.get_cmap('seismic')

pcm3 = ax3.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap3
                   , vmin=-30, vmax=10)
cbar3 = fig.colorbar(pcm3,extend='max')
cbar3.set_label('$T_{air}$ [$\circ$C]',size=16)

ax4 = fig.add_subplot(224)
ax4.get_yaxis().set_ticklabels([])
ax4.get_xaxis().set_ticklabels([])
ax4.set_facecolor('gray')
ax4.text(0.05,0.05,'D',transform=ax4.transAxes, fontsize=20,weight='bold')


cmap4 = plt.get_cmap('seismic')

pcm4 = ax4.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap4
                   , vmin=-20, vmax=20)
cbar4 = fig.colorbar(pcm4,extend='both')
cbar4.set_label('$T_{air}$ [$\circ$C]',size=16)

plt.savefig('AirtempsAlt')<jupyter_output><empty_output><jupyter_text>The next step could be to change the seismic colormap itself, so the blues span a larger range, showing more detail in Greenland.  ExerciseTo finish up, an exercise for those who want some practice: I have made a map using the methods we've treated. You can try to recreate it by changing the colormap and pcolormesh in the code below. Tips: look at the colorbar and check the cmocean page!![Exercise.png](Exercise.png)<jupyter_code>### leave these unchanged ###
fig = plt.figure(figsize=(15,10))
ax = fig.add_subplot(111)
ax.set_title('Daily average Air Temperature in the Arctic',fontsize=24)
ax.get_yaxis().set_ticklabels([])
ax.get_xaxis().set_ticklabels([])
ax.set_facecolor('gray')


### CHANGE THINGS HERE ###

cmap = plt.get_cmap('viridis')

pcm = ax.pcolormesh(ds.x,ds.y,ds.Tair.isel(time=0)
                    ,cmap=cmap
                   , vmin=-20, vmax=5)


### leave these unchanged ###
cbar = fig.colorbar(pcm,extend='both')
cbar.set_label('$T_{air}$ [$\circ$C]',size=16)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11708390319319736, 'fraction_numerical': 0.03555925551525242, 'mean_word_length': 4.255625, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 49, 'https://': 9, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '238302', 'license': 'MIT', 'n_tokens_mistral': 5998, 'n_tokens_neox': 5465, 'n_words': 1769, 'path': 'projects/PFL_17_matplotlib_colors/.ipynb_checkpoints/Using color in ocean and atmosphere data-checkpoint.ipynb', 'repo_name': 'reint-fischer/reint-fischer.github.io', 'size': '16846'}