<jupyter_start><jupyter_text>Numpy [![colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/davemlz/spyndex/blob/main/docs/tutorials/numpy.ipynb)![level2](https://raw.githubusercontent.com/davemlz/spyndex/main/docs/_static/level2.png) Welcome to the second level: `spyndex + numpy`!Remember to install `spyndex`!<jupyter_code>!pip install -U spyndex<jupyter_output><empty_output><jupyter_text>Now, let's start!First, import `spyndex` and `numpy`:<jupyter_code>import spyndex
import numpy as np<jupyter_output><empty_output><jupyter_text>`numpy.int*`We all know the amazing world of `numpy`, right? Let's simulate some `integer` values!<jupyter_code>NIR = np.int16(6578)
RED = np.int16(1435)<jupyter_output><empty_output><jupyter_text>If we check the data types for each one of our variables, we will see that they are `numpy.int16`:<jupyter_code>print(f"NIR type: {type(NIR)}")
print(f"RED type: {type(RED)}")<jupyter_output>NIR type: <class 'numpy.int16'>
RED type: <class 'numpy.int16'><jupyter_text>And let's check what we need to compute the `DVI`:<jupyter_code>spyndex.indices.DVI<jupyter_output><empty_output><jupyter_text>Let's see the standard names for the bands:<jupyter_code>spyndex.indices.DVI.bands<jupyter_output><empty_output><jupyter_text>With this we can create our `dict`!<jupyter_code>parameters = {"N": NIR, "R": RED}<jupyter_output><empty_output><jupyter_text>Now, let's compute the index!<jupyter_code>idx = spyndex.computeIndex("DVI",parameters)<jupyter_output><empty_output><jupyter_text>If we check the type of the result, we'll see that it is also a `numpy.int16`!<jupyter_code>print(f"idx type: {type(idx)}")<jupyter_output>idx type: <class 'numpy.int16'><jupyter_text>And the value of the index:<jupyter_code>idx<jupyter_output><empty_output><jupyter_text>That's great! Now, let's exagerate a little bit... Imagine we have, for some reason, very big values and we need a different data type: `numpy.int64`> Note that using this kind of data type is not recommended for most spectral indices!<jupyter_code>NIR = np.int64(657812517257537)
RED = np.int64(143532873927932)<jupyter_output><empty_output><jupyter_text>If we check the types:<jupyter_code>print(f"NIR type: {type(NIR)}")
print(f"RED type: {type(RED)}")<jupyter_output>NIR type: <class 'numpy.int64'>
RED type: <class 'numpy.int64'><jupyter_text>And if we compute the `DVI`:<jupyter_code>idx = spyndex.computeIndex("DVI",{"N": NIR, "R": RED})<jupyter_output><empty_output><jupyter_text>We'll see that the data type of the result is now `numpy.int64`!<jupyter_code>print(f"idx type: {type(idx)}")<jupyter_output>idx type: <class 'numpy.int64'><jupyter_text>Here the resulting index:<jupyter_code>idx<jupyter_output><empty_output><jupyter_text>`numpy.float*`Now, let's do things as they are supposed to be: Using `floats`!<jupyter_code>NIR = np.float32(0.63241)
RED = np.float32(0.13641)<jupyter_output><empty_output><jupyter_text>If we check the data types, we'll see the `numpy.float32`:<jupyter_code>print(f"NIR type: {type(NIR)}")
print(f"RED type: {type(RED)}")<jupyter_output>NIR type: <class 'numpy.float32'>
RED type: <class 'numpy.float32'><jupyter_text>Now, let's compute the `DVI` again:<jupyter_code>idx = spyndex.computeIndex("DVI",{"N": NIR, "R": RED})<jupyter_output><empty_output><jupyter_text>And the result is a beautiful `numpy.float32`!<jupyter_code>print(f"idx type: {type(idx)}")<jupyter_output>idx type: <class 'numpy.float32'><jupyter_text>Here our `DVI` value:<jupyter_code>idx<jupyter_output><empty_output><jupyter_text>If required, different `float` datatypes, such as `numpy.float64`, can be used too!  `numpy.ndarray`Let's get spicy! We are going to work with the main data type of `numpy`: `arrays`.Let's simulate some arrays for the visible spectrum!<jupyter_code>BLUE = np.random.normal(0.13,0.05,1000)
GREEN = np.random.normal(0.23,0.06,1000)
RED = np.random.normal(0.14,0.07,1000)<jupyter_output><empty_output><jupyter_text>If we check the data types for each one of our variables, we will see that they are `numpy.ndarray`:<jupyter_code>print(f"BLUE type: {type(BLUE)}, shape: {BLUE.shape}")
print(f"GREEN type: {type(GREEN)}, shape: {GREEN.shape}")
print(f"RED type: {type(RED)}, shape: {RED.shape}")<jupyter_output>BLUE type: <class 'numpy.ndarray'>, shape: (1000,)
GREEN type: <class 'numpy.ndarray'>, shape: (1000,)
RED type: <class 'numpy.ndarray'>, shape: (1000,)<jupyter_text>Amazing! Let's compute some spectral indices with the visible spectrum: `GLI` and `VARI`!<jupyter_code>spyndex.indices.GLI
spyndex.indices.VARI<jupyter_output><empty_output><jupyter_text>Let's check the standard names!<jupyter_code>spyndex.indices.GLI.bands
spyndex.indices.VARI.bands<jupyter_output><empty_output><jupyter_text>Since both use the visible spectrum, we just need to create a `dict` with the `B`, `G` and `R` parameters!<jupyter_code>parameters = {"B": BLUE, "G": GREEN, "R": RED}<jupyter_output><empty_output><jupyter_text>Now we just have to compute the indices! It can be easily done with `spyndex.computeIndex()`:<jupyter_code>idx = spyndex.computeIndex(["GLI","VARI"], parameters)<jupyter_output><empty_output><jupyter_text>Let's check our result!<jupyter_code>print(f"idx type: {type(idx)}, shape: {idx.shape}")<jupyter_output>idx type: <class 'numpy.ndarray'>, shape: (2, 1000)<jupyter_text>Yes! It is a `numpy.ndarray`! And check the shape of it: `(2, 1000)`, why is that? Well, our initial shapes were `(1000, )` and `(1000, )`, and `spyndex` concatenates the resulting indices along the first axis, so, since we computed 2 indices, the result has the shape `(2, 1000)`. Let's try other shapes!<jupyter_code>BLUE = np.random.normal(0.13,0.05,(100,100,100))
GREEN = np.random.normal(0.23,0.06,(100,100,100))
RED = np.random.normal(0.14,0.07,(100,100,100))<jupyter_output><empty_output><jupyter_text>We have created 3 cubes of spectral data! Let's confirm it:<jupyter_code>print(f"BLUE type: {type(BLUE)}, shape: {BLUE.shape}")
print(f"GREEN type: {type(GREEN)}, shape: {GREEN.shape}")
print(f"RED type: {type(RED)}, shape: {RED.shape}")<jupyter_output>BLUE type: <class 'numpy.ndarray'>, shape: (100, 100, 100)
GREEN type: <class 'numpy.ndarray'>, shape: (100, 100, 100)
RED type: <class 'numpy.ndarray'>, shape: (100, 100, 100)<jupyter_text>Nice! Now we can define our parameters!<jupyter_code>parameters = {"B": BLUE, "G": GREEN, "R": RED}<jupyter_output><empty_output><jupyter_text>And let's compute the indices again!<jupyter_code>idx = spyndex.computeIndex(["GLI","VARI"], parameters)<jupyter_output><empty_output><jupyter_text>What shape has the result?...<jupyter_code>print(f"idx type: {type(idx)}, shape: {idx.shape}")<jupyter_output>idx type: <class 'numpy.ndarray'>, shape: (2, 100, 100, 100)<jupyter_text>That's right! Now we have two cubes! `(2, 100, 100, 100)`, one for the `GLI` and another one for the `VARI`!Everything is beautiful, and so are shapes... pay attention! if your shapes are not equal, you're going to be in trouble!<jupyter_code>BLUE = np.random.normal(0.13,0.05,(200,100,100))
GREEN = np.random.normal(0.23,0.06,(100,300,100))
RED = np.random.normal(0.14,0.07,(100,100,400))<jupyter_output><empty_output><jupyter_text>What kind of monster creates these cubes with different shapes?<jupyter_code>print(f"BLUE type: {type(BLUE)}, shape: {BLUE.shape}")
print(f"GREEN type: {type(GREEN)}, shape: {GREEN.shape}")
print(f"RED type: {type(RED)}, shape: {RED.shape}")<jupyter_output>BLUE type: <class 'numpy.ndarray'>, shape: (200, 100, 100)
GREEN type: <class 'numpy.ndarray'>, shape: (100, 300, 100)
RED type: <class 'numpy.ndarray'>, shape: (100, 100, 400)<jupyter_text>Be careful!<jupyter_code>parameters = {"B": BLUE, "G": GREEN, "R": RED}<jupyter_output><empty_output><jupyter_text>Oh God, I'm scared...<jupyter_code>idx = spyndex.computeIndex(["GLI","VARI"], parameters)<jupyter_output><empty_output><jupyter_text>![youdied](https://raw.githubusercontent.com/davemlz/spyndex/main/docs/_static/youdied.png)Of course you died! Never use parameters with different shapes! :) Unless you want to use some of the parameters as constants!<jupyter_code>BLUE = np.random.normal(0.13,0.05,(100,100,100))
GREEN = 0.23
RED = np.random.normal(0.14,0.07,(100,100,100))<jupyter_output><empty_output><jupyter_text>Is this going to work?<jupyter_code>parameters = {"B": BLUE, "G": GREEN, "R": RED}<jupyter_output><empty_output><jupyter_text>I hope so...<jupyter_code>idx = spyndex.computeIndex(["GLI","VARI"], parameters)<jupyter_output><empty_output><jupyter_text>There you go! It did! That's because you can use constants in combination with `numpy.ndarray` objects!<jupyter_code>print(f"idx type: {type(idx)}, shape: {idx.shape}")<jupyter_output>idx type: <class 'numpy.ndarray'>, shape: (2, 100, 100, 100)
{'fraction_non_alphanumeric': 0.15897727272727272, 'fraction_numerical': 0.042045454545454546, 'mean_word_length': 8.186847599164928, 'pattern_counts': {'":': 18, '<': 170, '<?xml version=': 0, '>': 171, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '40', 'dir': 'jupyter-structured-clean-dedup', 'id': '341263', 'license': 'MIT', 'n_tokens_mistral': 3688, 'n_tokens_neox': 3371, 'n_words': 738, 'path': 'docs/tutorials/numpy.ipynb', 'repo_name': 'davemlz/spyndex', 'size': '8800'}