<jupyter_start><jupyter_text>GSTools: The Python toolbox for your geo-statistical project!  <NAME>, <NAME>, <NAME>, <NAME> and <NAME>*To navigate through the presentation, use the arrows in the lower right corner or your arrow-keys (*__*right*__ *&rarr; next section;* __*down*__ *&rarr; go through current section)* **GSTools** is embedded in the GitHub Organiztaion **GeoStat-Framework**: https://github.com/GeoStat-Framework/GSTools Purpose<imgalign="right"width="300" src="https://raw.githubusercontent.com/GeoStat-Framework/GeoStat-Framework.github.io/master/img/GS_globe.png" alt=""/>GeoStatTools provides geostatistical tools for various purposes in *arbitrary dimensions*:- simple, ordinary, universal and external drift kriging- random field generation- conditioned field generation- incompressible random vector field generation- (automatted) variogram estimation and fitting- directional variogram estimation and modelling- data normalization and transformation- many readily provided and even user-defined covariance models- metric spatio-temporal modelling- plotting and exporting routines <imgalign="right"width="300" src="https://raw.githubusercontent.com/GeoStat-Framework/GSTools/master/docs/source/pics/demonstrator.png" alt=""/> InstallationYou can install GSTools via `pip` *or* `conda`:```bashpip install gstools  /  conda install gstools```<jupyter_code>import warnings
warnings.filterwarnings('ignore')<jupyter_output><empty_output><jupyter_text>1. Random Field GenerationThe original main feature of GSTools is the spatial random field generator [`SRF`](https://geostat-framework.readthedocs.io/projects/gstools/en/stable/field.htmlgstools.field.SRF), which can generate random fields following a given covariance model. These can be used for hypothesis testing and simulations.  A Very Simple ExampleA spatial random field with an isotropic Gaussian covariance model and following parameters:- variance $\sigma^2=1$- correlation length $\ell=10$<jupyter_code>import gstools as gs
# 100x100 field
x = y = range(100)
model = gs.Gaussian(dim=2, var=1, len_scale=10)
srf = gs.SRF(model, seed=20170519)
field = srf.structured([x, y])
ax = srf.plot()<jupyter_output><empty_output><jupyter_text>Fancier FieldsUsing an anisotropic Matern covariance model.<jupyter_code>import numpy as np
x = y = np.arange(100)
model = gs.Matern(dim=2, var=1, len_scale=[12.0, 3.0], angles=np.deg2rad(22.5))
srf = gs.SRF(model, seed=20170519)
srf.structured([x, y])
ax = srf.plot()<jupyter_output><empty_output><jupyter_text>2. KrigingThe following kriging methods are provided within the submodule `gstools.krige`.Name | Description | Properties--- | --- | ---`Krige` | Swiss-Army-Knife for kriging. | Including all present methods.`Simple` | Simple kriging. | Given mean.`Ordinary` | Ordinary kriging. | Estimated mean.`Universal` | Universal kriging. | Functional trend.`ExtDrift` | External drift kriging (EDK) | Given trend.`Detrended` | Detrended simple kriging. | Used for regression kriging.  Interpolating spatial dataGSTools provides routines to automatically normalize data, fit a variogram model and interpolate the data with kriging.<jupyter_code>import numpy as np
import gstools as gs
# temperature station data from DWD
lat = np.array([52.9336, 48.6159, 52.4853, 50.7446, 52.9437, 53.8633, 47.8342,
                51.0881, 48.406 , 49.7273, 49.4691, 48.0197, 50.4237, 53.0316,
                53.8412, 54.6792, 49.9694, 51.3745, 47.8774, 50.5908])
lon = np.array([ 8.237 , 13.0506,  7.9126,  9.345 , 12.8518,  8.1275, 10.8667,
                12.9326, 11.3117,  8.1164, 11.8546, 12.2925,  7.4202, 13.9908,
                13.6846, 13.4343,  9.9114, 11.292 , 11.3643, 12.7139])
temp = np.array([15.7, 13.9, 15.1, 17. , 21.9, 11.9, 11.4, 17.2, 12.9, 17.2, 13.4,
                 13.9, 18.1, 21.3, 21.3, 17.4, 18.6, 20.2, 12.7, 15.8])
# output grid
grid_lat = np.linspace(47, 55)
grid_lon = np.linspace( 7, 15)
model = gs.Spherical(latlon=True, rescale=gs.EARTH_RADIUS)
krige = gs.krige.Ordinary(
    model, cond_pos=[lat, lon], cond_val=temp, fit_variogram=True)
field = krige.structured([grid_lat, grid_lon])
ax = krige.plot()
# plotting lat on y-axis and lon on x-axis
ax.scatter(lon, lat, 50, c=temp, edgecolors="k")
krige.model<jupyter_output><empty_output><jupyter_text>3. Conditioned FieldsKriged fields tend to approach the field mean with increasing distance to observations. GSTools provides a combination of the Randomization Method and the Kriging routines to provided conditioned random fields.  Conditioning with Ordinary KrigingHere we use ordinary kriging in 1D (for plotting reasons) with 5 given observations/conditions, to generate an ensemble of conditioned random fields.<jupyter_code>import matplotlib.pyplot as plt
# condtions
cond_pos = [0.3, 1.9, 1.1, 3.3, 4.7]
cond_val = [0.47, 0.56, 0.74, 1.47, 1.74]
gridx = np.linspace(0.0, 15.0, 151)
model = gs.Gaussian(dim=1, var=0.5, len_scale=2)
krige = gs.Krige(model, cond_pos, cond_val)
cond_srf = gs.CondSRF(krige)
fields = []
for i in range(100):
    fields.append(cond_srf(gridx, seed=i))
    plt.plot(gridx, fields[i], color="k", alpha=0.1)
plt.plot(gridx, cond_srf.krige.field, linestyle="dashed", label="kriged field")
plt.scatter(cond_pos, cond_val, color="k", zorder=10, label="Conditions")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>4. NormalizationWhen dealing with real-world data, one can’t assume it to be normal distributed. In fact, many properties are modeled by applying different transformations, for example conductivity is often assumed to be log-normal or precipitation is transformed using the famous box-cox power transformation.  Interpolating non-normal dataHere we generate synthetic data, sample from it and try to reconstruct the field.<jupyter_code>import numpy as np
import gstools as gs
import matplotlib.pyplot as plt
# structured field with edge length of 50
x = y = range(51)
pos = gs.generate_grid([x, y])
model = gs.Gaussian(dim=2, var=1, len_scale=10)
srf = gs.SRF(model, seed=20170519, normalizer=gs.normalizer.LogNormal)
# generate the original field
srf(pos)
# we sample 60 points and set the conditioning points and values.
ids = np.arange(srf.field.size)
samples = np.random.RandomState(20210201).choice(ids, size=60, replace=False)
# sample conditioning points from generated field
cond_pos = pos[:, samples]
cond_val = srf.field[samples]
krige = gs.krige.Ordinary(
    model=gs.Stable(dim=2),
    cond_pos=cond_pos, cond_val=cond_val,
    normalizer=gs.normalizer.BoxCox,
    fit_normalizer=True,
    fit_variogram=True,
)
print(krige.model)
print(krige.normalizer)
fig, axs = plt.subplots(1, 2, figsize=[6, 3])
hist1 = axs[0].hist(cond_val)
hist2 = axs[1].hist(krige.normalizer.normalize(cond_val))
field = krige(pos, return_var=False)
fig, ax = plt.subplots(1, 3, figsize=[8, 3])
ax[0].imshow(srf.field.reshape(len(x), len(y)).T, origin="lower")
ax[1].scatter(*cond_pos, c=cond_val)
ax[2].imshow(krige.field.reshape(len(x), len(y)).T, origin="lower")
# titles
ax[0].set_title("original field")
ax[1].set_title("sampled field")
ax[2].set_title("interpolated field")
# set aspect ratio to equal in all plots
_ = [ax[i].set_aspect("equal") for i in range(3)]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12638121546961326, 'fraction_numerical': 0.061878453038674036, 'mean_word_length': 5.215450643776824, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '185913', 'license': 'MIT', 'n_tokens_mistral': 2815, 'n_tokens_neox': 2479, 'n_words': 692, 'path': 'Transform2021.ipynb', 'repo_name': 'MuellerSeb/GSTools-EGU2020', 'size': '7277'}