<jupyter_start><jupyter_text>Tutorial .. contents::    :local:  define a dimarray A **``DimArray``** can be defined just like a numpy array, withadditional information about its axes, which can be givenvia `axes` and `dims` parameters.<jupyter_code>from dimarray import DimArray, Dataset
a = DimArray([[1.,2,3], [4,5,6]], axes=[['a', 'b'], [1950, 1960, 1970]], dims=['variable', 'time'])
a<jupyter_output><empty_output><jupyter_text>data structure Array data are stored in a `values` **attribute**:<jupyter_code>a.values<jupyter_output><empty_output><jupyter_text>while its axes are stored in `axes`<jupyter_code>a.axes<jupyter_output><empty_output><jupyter_text>As a convenience, axis labels can be accessed directly by name, as an alias for `a.axes['time'].values`:<jupyter_code>a.time<jupyter_output><empty_output><jupyter_text>For more information refer to section on :ref:`page_data_structure_dimarray` (as well as :py:class:`dimarray.Axis` and :py:class:`dimarray.Axes`)  numpy-like attributes Numpy-like attributes `dtype`, `shape`, `size` or `ndim` are defined, and are now augmented with `dims` and `labels`<jupyter_code>a.shape
a.dims      # grab axis names (the dimensions)
a.labels   # grab axis values<jupyter_output><empty_output><jupyter_text>indexing  **Indexing** works on labels just as expected, including `slice` and boolean array.<jupyter_code>a['b', 1970]<jupyter_output><empty_output><jupyter_text>but integer-index is always possible via `ix` toogle between `labels`- and `position`-based indexing:<jupyter_code>a.ix[0, -1]<jupyter_output><empty_output><jupyter_text>.. seealso:: :ref:`page_indexing`  transformation Standard numpy transformations are defined, and now accept axis name:<jupyter_code>a.mean(axis='time')<jupyter_output><empty_output><jupyter_text>and can ignore **missing values (nans)** if asked to:<jupyter_code>import numpy as np 
a['a',1950] = np.nan
a.mean(axis='time', skipna=True)<jupyter_output><empty_output><jupyter_text>.. seealso:: :ref:`page_transformations`  alignment in operations During an operation, arrays are **automatically re-indexed** to span the same axis domain, with nan filling if needed. This is quite useful when working with partly-overlapping time series or with incomplete sets of items.<jupyter_code>yearly_data = DimArray([0, 1, 2], axes=[[1950, 1960, 1970]], dims=['year'])  
incomplete_yearly_data = DimArray([10, 100], axes=[[1950, 1960]], dims=['year']) # last year 1970 is missing
yearly_data + incomplete_yearly_data<jupyter_output><empty_output><jupyter_text>.. seealso:: :meth:`reindex_axis `, :meth:`reindex_like ` and :func:`align_axes ` A check is also performed on the dimensions, to ensure consistency of the data.If dimensions do not match this is not interpreted as an error but rather as a combination of dimensions. For example, you may want to combine some fixed spatial pattern (such as an EOF) with a time-varying time series (the principalcomponent). Or you may want to combine results from a sensitivity analysiswhere several parameters have been varied (one dimension per parameter). Here a minimal example where the above-define annual variable is combined with seasonally-varying data (camping summer and winter prices). Arrays are said to be **broadcast**:<jupyter_code>seasonal_data = DimArray([10, 100], axes=[['winter','summer']], dims=['season'])
combined_data = yearly_data * seasonal_data
combined_data<jupyter_output><empty_output><jupyter_text>.. seealso:: :func:`broadcast_arrays ` and :meth:`reshape `  dataset .. versionchanged :: 0.1.9As a commodity, the **`Dataset`** class is an ordered dictionary of DimArray instances which all share a common set of axes.<jupyter_code>dataset = Dataset({'combined_data':combined_data, 'yearly_data':yearly_data,'seasonal_data':seasonal_data})
dataset<jupyter_output><empty_output><jupyter_text>At initialization, the arrays are aligned on-the-fly. Later on, it is up to the user to reindex the arrays to match the Dataset axes... note :: since Dataset elements share the same axes, any axis modification will also impact all contained DimArray instances. If this behaviour is not desired, a copy should be made.  netCDF reading and writing A natural I/O format for such an array is netCDF, common in geophysics, which rely onthe netCDF4 package. If netCDF4 is installed (much recommanded), a dataset can easily read and write to the netCDF format:<jupyter_code>dataset.write_nc('/tmp/test.nc', mode='w')
import dimarray as da
da.read_nc('/tmp/test.nc', 'combined_data')<jupyter_output><empty_output><jupyter_text>.. seealso:: :ref:`page_netcdf`  metadata It is possible to define and access metadata via the standard `.` syntax to access an object attribute:<jupyter_code>a = DimArray([1, 2])
a.name = 'myarray'
a.units = 'meters'<jupyter_output><empty_output><jupyter_text>Any non-private attribute is automatically added to `a.attrs` ordered dictionary:<jupyter_code>a.attrs<jupyter_output><empty_output><jupyter_text>Metadata can also be defined for :class:`dimarray.Dataset` and :class:`dimarray.Axis` instances, and will be written to / read from netCDF files. .. note:: Metadata that start with an underscore `_` or use any protected class attribute as name (e.g. `values`, `axes`, `dims` and so on) must be set directly in `attrs`... seealso:: :ref:`page_metadata` for more information.  join arrays DimArrays can be joined along an existing dimension, we say `concatenate` (:func:`dimarray.concatenate`):<jupyter_code>a = DimArray([11, 12, 13], axes=[[1950, 1951, 1952]], dims=['time'])
b = DimArray([14, 15, 16], axes=[[1953, 1954, 1955]], dims=['time'])
da.concatenate((a, b), axis='time')<jupyter_output><empty_output><jupyter_text>or they can be stacked along each other, thereby creating a new dimension (:func:`dimarray.stack`)<jupyter_code>a = DimArray([11, 12, 13], axes=[[1950, 1951, 1952]], dims=['time'])
b = DimArray([21, 22, 23], axes=[[1950, 1951, 1952]], dims=['time'])
da.stack((a, b), axis='items', keys=['a','b'])<jupyter_output><empty_output><jupyter_text>In the above note that new axis values were provided via the parameter `keys=`. If the common "time" dimension was not fully overlapping, array can be aligned prior to stacking via the `align=True` parameter.<jupyter_code>a = DimArray([11, 12, 13], axes=[[1950, 1951, 1952]], dims=['time'])
b = DimArray([21, 23], axes=[[1950, 1952]], dims=['time'])
c = da.stack((a, b), axis='items', keys=['a','b'], align=True)
c<jupyter_output><empty_output><jupyter_text>.. seealso:: :ref:`ref_api_join`  drop missing data Say you have data with NaNs:<jupyter_code>a = DimArray([[11, np.nan, np.nan],[21,np.nan,23]], axes=[['a','b'],[1950, 1951, 1952]], dims=['items','time'])
a<jupyter_output><empty_output><jupyter_text>You can drop every column that contains a NaN<jupyter_code>a.dropna(axis=1) # drop along columns<jupyter_output><empty_output><jupyter_text>or actually control decide to retain only these columns with a minimum number of valid data, here one:<jupyter_code>a.dropna(axis=1, minvalid=1) # drop every column with less than one valid data<jupyter_output><empty_output><jupyter_text>.. seealso:: :ref:`ref_api_missingvalues`  reshaping arrays Additional novelty includes methods to reshaping an array in easy ways, very useful for high-dimensional data analysis.<jupyter_code>large_array = DimArray(np.arange(2*2*5*2).reshape(2,2,5,2), dims=('A','B','C','D'))
small_array = large_array.reshape('A,D','B,C')
small_array<jupyter_output><empty_output><jupyter_text>.. seealso:: :ref:`ref_api_reshaping` and :ref:`page_reshape`  interfacing with pandas For things that pandas does better, such as pretty printing, I/O to many formats, and 2-D data analysis, just use the :py:meth:`dimarray.DimArray.to_pandas` method. In the ipython notebook it also has a nice html rendering.<jupyter_code>small_array.to_pandas()<jupyter_output><empty_output><jupyter_text>And :py:meth:`dimarray.DimArray.from_pandas` works to convert pandas objects to `DimArray` (also supports `MultiIndex`):<jupyter_code>import pandas as pd
s = pd.DataFrame([[1,2],[3,4]], index=['a','b'], columns=[1950, 1960])
da.from_pandas(s)<jupyter_output><empty_output><jupyter_text>plotting dimarray comes with basic plotting facility. For 1-D and 2-D data, it simplies interfaces pandas' plot command (therefore pandas needs to be installed to use it). From the example above:<jupyter_code>%pylab
%matplotlib inline
a = dataset['combined_data']
a.plot()<jupyter_output>Using matplotlib backend: TkAgg
Populating the interactive namespace from numpy and matplotlib<jupyter_text>In addition, it can also display 2-D data via its methods `contour`, `contourf` and `pcolor` mapped from matplotlib.<jupyter_code># create some data
lon = np.linspace(-180, 180, 10)
lat = np.linspace(-90, 90, 10)
LON, LAT = np.meshgrid(lon, lat)
DATA = np.cos(np.radians(LON)) + np.cos(np.radians(LAT))
# define dimarray
a = DimArray(DATA, axes=[lat, lon], dims=['lat','lon'])
# plot the data
c = a.contourf()
colorbar(c)  # explicit colorbar creation  # doctest: +SKIP
a.contour(colors='k') # doctest: +SKIP
# plot the data
a.pcolor(colorbar=True)  # colorbar as keyword argument # doctest: +SKIP<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1300141319708664, 'fraction_numerical': 0.025328840091314274, 'mean_word_length': 6.972270363951473, 'pattern_counts': {'":': 0, '<': 104, '<?xml version=': 0, '>': 104, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '26', 'dir': 'jupyter-structured-clean-dedup', 'id': '568876', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 3211, 'n_tokens_neox': 2971, 'n_words': 928, 'path': 'docs/notebooks/tutorial.ipynb', 'repo_name': 'perrette/dimarray', 'size': '9199'}