<jupyter_start><jupyter_text>*mocalum* tutorial 1  Getting started with mocalum A notebook by <NAME>      Introduction This is an interactive tutorial that will introduce you to the basic of Monte Carlo uncertainty modeling using `mocalum` package. After completing the tutorial you will be familiar with basic `mocalum` functionality such as:  - Adding lidars to the `mocalum` object  - Configuring lidar measurement scenario  - Configuring and generating lidar uncertainty contributors  - Configuring and generating flow field  - Calculating radial velocity  - Reconstructing wind speed  - Accessing generated `mocalum` specific `xarray` datasets         On *mocalum*    `mocalum` is a python package for Monte Carlo based lidar uncertainty modeling. It has following features: - Slick and super fast Monte Carlo uncertainty modeling (thouhsands of simulations in matter of seconds) - Simulation of single or multi lidar configuration - Configuration of arbitrary trajectories for single and multi lidars - Configuration of `IVAP` (sector-scan) trajectory for single lidar - 3D or 4D / uniform or turbulent flow field generation - Sampling of correlated or uncorrelated uncertainty terms - Built-in 2nd order kinematic model for calculation of trajectory timing - 3D or 4D interpolation/projection of flow on lidar(s) line-of-sight(s) - Results given as [xarray](http://xarray.pydata.org/en/stable/) datasets enriched with metadata (no more confusion what is what) *mocalum* workflow    Typical `mocalum` workflow includes following steps (depict in the figure on right): 1. Create mocalum object    1. Adding lidar(s) and configuring uncertainty contributors to the object 2. Setting up measurement scenario for previously added lidar(s) 3. Generating flow field entailing measurement points 4. Project flow field on line-of-sight(s) 5. Reconstruct wind vector(s) 6. Perform statistical analysis of reconstructed wind vectorsWe will follow the above workflow in this tutorial.     Import packagesWe will import standard and external packages that we will need in tutorial, specifically: - `numpy` is python's main numerical array package https://www.numpy.org/  - `matplotlib` is python's main plotting package https://matplotlib.org/ - `xarray` is a powerful high level package for labelled multi-dimensional arrays http://xarray.pydata.org - `mocalum` - `mocalum_tutorial` contains a plotting function that we will use in this tutorial    > **tutorial note**: some basic understanding of `xarray` is prerequisit to follow this tutorial. Therefore, get familiar with `xarray` first, particularly accessing and working with [dataset](http://xarray.pydata.org/en/stable/data-structures.htmldataset)<jupyter_code>import numpy as np           
import matplotlib.pyplot as plt
import xarray as xr

import mocalum as mc

from mocalum_tutorial import plot_sd_scan_setup, plot_bbox<jupyter_output><empty_output><jupyter_text>Create and explore mocalum object`mocalum` is developed considering object-oriented paradigm. Therefore, all calculations and access to generated data are through an instance of `mocalum` object on which various methods are applied. Once an instance of `mocalum` object is created users have access to following methods:- `add_lidar`- `generate_complex_trajectory`- `generate_PPI_scan`- `generate_uncertainties`- `generate_flow_field`- `project_to_los`- `reconstruct_wind`as well to `data` that will be created as a result of applying the above listed methods.> **notebook note**: you can view the documentation for a function in jupyter notebooks by placing a `?` in front of the function, and you can get the entire function by using `??`.<jupyter_code>mc_test = mc.Mocalum()<jupyter_output><empty_output><jupyter_text>Adding lidar and configuring uncertantyChecking `docstring` of `add_lidar` method we can see that following parameters should be provided:- `id` of lidar- `lidar position` as `numpy` array of triplets (x, y, z)- `uncertanty dictionary` containing configuration of uncertainty contributorsLets consider a lidar which we will call 'koshava', located at position of (0,0,0) having following values for uncertainty contributors (standard uncertainty):- estimation uncertainty of radial velocity of 0.1 m/s - ranging uncertainty of 5 m- azimuth uncertainty of 0.1 deg- elevation uncertainty of 0.1 degWe will consider that uncertainty values are not correlated from one measurement point to another, thus correlation coeficient will be equal to 0Accordingly we add the above lidar to the `mocalum` object `mc_test`:<jupyter_code>lidar_pos = np.array([0,0,0])

unc_cfg = {'unc_az'   : 0.1,
           'unc_el'   : 0.1,
           'unc_rng'  : 5,
           'unc_est'  : 0.1,
           'corr_coef':0}

mc_test.add_lidar('koshava', lidar_pos, unc_cfg)<jupyter_output><empty_output><jupyter_text>The lidar information are added to the measurement configuration dictionary stored inside `data` subobject:<jupyter_code>mc_test.data.meas_cfg['koshava']<jupyter_output><empty_output><jupyter_text>Setting up measurement scenario for previously added lidarFollowing the lidar placement and uncertainty configuration we will configure the measurement scenario which lidar will 'virtually' perform. In this example we will configure a `PPI` scan, thus we will use the built-in method `generate_PPI_scan`, which requires following inputs: - lidar id for which the PPI scan is configured - PPI scan dictionary containing following required parameters:    - `sector_size` : size of the scanned sector in degrees    - `azimuth_mid` : central azimuth angle of the PPI scanned arc    - `angular_step` : Incremental step performed to complete a PPI scan    - `acq_time` : acquisition time to complete an angular step, thus acquire LOS measurements    - `elevation` : elevation angle of PPI scan    - `range` : range at which measurements should take place    - `no_scans` : number of PPI scans must be equal or bigger than 1       while following parameters are optional:     - `max_speed` : maximum permitted angular speed    - `max_acc` : maximum permitted angular acceleration    > **mocalum note**: your PPI dict keys must have names identical to names listed above. > **mocalum note**: If `max_speed` or `max_acc` are not provided `mocalum` considers that there are no kinematic restrictions, thus values for these parameters are consider to be infinit. This practically means that in case of a PPI scan the time it takes to return from the end of the PPI scan to its begining is 0 s. Let say we want to peform PPI scan of 30 deg centered at azimuth angle of 90 deg at range of 1 km over 100 m above ground level. We will do 10000 PPI scans and we will consider kinematic limits of the long-range WindScanner:<jupyter_code>PPI_cfg = {
    'no_scans' : 10000,
    'range' : 1000,
    'meas_height' : 100,
    'elevation' : np.degrees(np.arcsin(100 / 1000)), # to assure measurements at 100 m agl
    'angular_step' : 1,  # degreee
    'acq_time' : 1,      # s
    'azimuth_mid' : 90,  # central azimuth angle
    'sector_size' : 30,  # degree
}

mc_test.generate_PPI_scan('koshava', PPI_cfg)<jupyter_output><empty_output><jupyter_text>Now if we again inspeact measurement configuration dictionary for `koshava` we will notice that it became updated with PPI scan configuration:<jupyter_code>mc_test.data.meas_cfg['koshava']<jupyter_output><empty_output><jupyter_text>Besides the update of the measurement configuration dictionary, a probing `xarray` dataset was created in this step in the background which fused all information together. To access it we simply execture following command:<jupyter_code>mc_test.data.probing['koshava']<jupyter_output><empty_output><jupyter_text>For convinience a plotting function `plot_sd_scan_setup` has been implemented in `mocalum_tutorial` that helps visually representing the configure measurement scenario:<jupyter_code>plot_sd_scan_setup('koshava', mc_test)<jupyter_output><empty_output><jupyter_text>If we closely inspect the above dataset we will notice that all our uncertainty variables have value equal to 0. The reason for this is that we still did not sample uncertainty contributors. In fact we only setup expected standard deviation for each uncertainty contributor. To actually sample uncertainties we need to execute the method `generate_uncertainties`, which only requires lidar id as a parameter. We will call this method and then once again access the probing dataset to verify that it became updated with uncertainty values:<jupyter_code>mc_test.generate_uncertainties('koshava')
mc_test.data.probing['koshava']

plot_sd_scan_setup('koshava', mc_test)<jupyter_output><empty_output><jupyter_text>Now as we can see that even the plot measurement scenario changed, since we introduced uncertainty in laser beam pointing and ranging. If we execute command `mc_test.data.probing['koshava'].unc_el.std()` we will get exactly the value we set initial (in this case 0.1):<jupyter_code>mc_test.data.probing['koshava'].unc_el.std()
mc_test.data.bbox_meas_pts['koshava']<jupyter_output><empty_output><jupyter_text>Generating flow field entailing measurement pointsCurrently, `mocalum`, through the method `generate_flow_field`, provides means to generate:- `uniform` flow field assuming that wind is only changing with height according to the [power law](https://en.wikipedia.org/wiki/Wind_profile_power_law)- `turbulent` flow field which is generated using a wrapper around [pyconturb](https://gitlab.windenergy.dtu.dk/pyconturb/pyconturb)Based on the measurement scenario `mocalum` calculates necessary dimensions and resolution of flow field box (3D for uniform or 4D for turbulent), generates the flow field data and saves them as `xarray` dataset. For this tutorial we will generate `uniform` flow field. The method `generate_flow_field` requires following input parameters: - `lidar_id` : string or list of string corresponding to lidar ids - `atmo_cfg` : dictionary describing mean flow parameters, containing following keys:     - `wind_speed` : mean horizontal wind speed in m/s     - `upward_velocity` : vertical wind speed in m/s     - `wind_from_direction` : mean wind direction in deg     - `reference_height` : height at which wind speed is given in m     - `shear_exponent` : vertical wind shear exponent       - `flow_type` : flow field type to be generated, it can be `uniform` or `turbulent`Let's define and generate flow field for 'koshava', which will be stored under `data.ffield`:<jupyter_code>atmo_cfg={'wind_speed':5,
          'upward_velocity':0,
          'wind_from_direction':45,
          'reference_height':100, # we set the reference height same as the scanned height 
          'shear_exponent':0.2}

mc_test.generate_flow_field('koshava', atmo_cfg, 'uniform')

plot_bbox(mc_test)<jupyter_output><empty_output><jupyter_text>We can plot flow field directly:<jupyter_code>mc_test.data.ffield.isel(y=0).v.plot()<jupyter_output><empty_output><jupyter_text>Project flow field on line-of-sight(s)Now we have all elements to calculate what wind lidar would measure, thus to project the flow field on the lidar LOSs. We do this using method `project_to_los`, which we supply with id of the lidar. This results in creation of yet another `xarray` dataset stored under `data.los['lidar_id']`.<jupyter_code>mc_test.project_to_los('koshava')
mc_test.data.los['koshava']<jupyter_output><empty_output><jupyter_text>Reconstruct wind vectorTo reconstruct wind vector from radial wind speeds we use method `reconstruct_wind` which requires following inputs:- `lidar_id` : for which reconstruction is performed, that is `koshava`- `rc_method`: wind vector reconstruction method, in this tutorial set to `IVAP`- `no_scans_avg`: number of scans to average prior reconstruciton (optional)The execution of the method will result in creation of `xarray` dataset containing the reconstructed wind vector (`data.rc_win`).> **mocalum note**: `no_scans_avg` must have such value that when total number of scans is divided by it remainder is 0. When this is not the case `mocalum` will raise `ValueError`.<jupyter_code>mc_test.reconstruct_wind('koshava', 'IVAP')
mc_test.data.rc_wind<jupyter_output><empty_output><jupyter_text>Statistical analysis of reconstructed windSince we know exactly what the wind speed is suppose to be at the scanned height of 100 m, that is 5 m/s, to assess the uncertainty we simply calculate mean and standard deviation of the reconstructed wind speed:<jupyter_code>mc_test.data.rc_wind.ws.mean()
mc_test.data.rc_wind.ws.std()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07110438729198185, 'fraction_numerical': 0.007803169042121188, 'mean_word_length': 5.242544731610338, 'pattern_counts': {'":': 0, '<': 62, '<?xml version=': 0, '>': 67, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '354219', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3609, 'n_tokens_neox': 3392, 'n_words': 1461, 'path': 'examples/tutorial-01/tutorial-01-getting-started.ipynb', 'repo_name': 'niva83/mocalum', 'size': '12570'}