<jupyter_start><jupyter_text>Diagnostic Parsing Array Array | Value | Description------|-------|-------1 | S | Scalar Diagnostic  | U | U-vector component Diagnostic  | V | V-vector component Diagnostic2 | U | C-Grid U-Point  | V | C-Grid V-Point  | M | C-Grid Mass Point  | Z | C-Grid Vorticity (Corner) Point3 |	  |Used for Level Integrated output: cumulate levels  | r | same but cumulate product by model level thickness  | R | same but cumulate product by hFac & level thickness4 | P | Positive Definite Diagnostic5 | C | with Counter array  | P | post-processed (not filled up) from other diags  | D | Disabled Diagnostic for output6-8|  | retired, formerly: 3-digit mate number9 | U | model-level plus 1/2  | M | model-level middle  | L | model-level minus 1/210| 0 | levels = 0  | 1 | levels = 1  | R | levels = Nr  | L | levels = MAX(Nr,NrPhys)  | M | levels = MAX(Nr,NrPhys) - 1  | G | levels = Ground_level Number  | I | levels = sea-Ice_level Number  | X | free levels option (need to be set explicitly)<jupyter_code>state_vars = {
    'U': MITgcmDiagnosticDescription(
         'U', 'UUR     MR', 'm/s', 'Zonal Component of Velocity (m/s)'),
    'V': MITgcmDiagnosticDescription(
         'V', 'VVR     MR', 'm/s', 'Meridional Component of Velocity (m/s)'),
    'W': MITgcmDiagnosticDescription(
         'W', 'WM      LR', 'm/s', 'Vertical Component of Velocity (r_units/s)'),
    'T': MITgcmDiagnosticDescription(
         'T', 'SMR     MR', 'degC', 'Potential Temperature'),
    'S': MITgcmDiagnosticDescription(
         'S', 'SMR     MR', 'psu', 'Salinity'),
    'PH': MITgcmDiagnosticDescription(
          'PH', 'SMR     MR', 'm^2/s^2', 'Hydrostatic Pressure Pot.(p/rho) Anomaly'),
    'PHL': MITgcmDiagnosticDescription(
          'PHL', 'SM      M1', 'm^2/s^2', 'Bottom Pressure Pot.(p/rho) Anomaly'),
    'Eta': MITgcmDiagnosticDescription(
          'Eta', 'SM      M1', 'm', 'Surface Height Anomaly'),
}
iternum = 1728000
for k, v in state_vars.iteritems():
    data = read_mds(grid_path(k), iternum)
    ds[k] = (v.coords(), data)
    ds[k].attrs = {'description': v.desc, 'units': v.units}
ds<jupyter_output><empty_output><jupyter_text>Time-average variables     Convtave    Eta2tave    ETAtave    PhHytave    PHL2tave    PHLtave    sFluxtave    Stave    Tdiftave    tFluxtave    Ttave    TTtave    uFluxtave    UStave    UTtave    UUtave    uVeltave    UVtave    vFluxtave    VStave    VTtave    vVeltave    VVtave    WStave    WTtave    wVeltave<jupyter_code>ds['T'] * ds['PH']
plt.imshow(ds['T'].mean(dim='XC'))
x = xray.DataArray( np.ones(10, dtype='>f4'))
print float(x.sum()), x.data.sum()
np.ones(10)#, dtype='>4f')
dt = np.dtype('>f4')
dt
type(ds['T'].data)
dtnew
import bottleneck as bn
bn.bench()
import mdsxray
reload(mdsxray)
#iters = range(1036800, 2073600,345600)
iters = range(2073840, 2384880, 480)
store = mdsxray.MDSDataStore(ddir, iters, deltaT=900)
ds = xray.Dataset.load_store(store)
ds
#ds.set_coords(dsc.variables.keys(), inplace=True)
#dsc = ds.chunk({'X': 100, 'Y':100})
#dsc
u = ds['U']
type(u.data)
xc = read_mds(grid_path('XC'))
xg = read_mds(grid_path('XG'))
yc = read_mds(grid_path('YC'))
xcd = da.from_array(xc, xc.shape)
xgd = da.from_array(xg, xg.shape)
ycd = da.from_array(yc, yc.shape)

import dask.array as da
dstacked = da.stack([xcd, xgd, ycd])
dstacked
type(np.asarray(xc))
from xray.backends import ScipyDataStore
sds = ScipyDataStore('/data/scratch/rpa/aviso/ftp.aviso.altimetry.fr/global/delayed-time/grids/msla/all-sat-merged/uv/all-monthly/dt_global_allsat_msla_uv_2014-12.nc')
sds.get_variables()['lat_bnds']
varnames = ['a', 'b']
vardata = {}
[vardata[k] = [] for k in varnames]
dims = ('Y','X')
('time',) + dims
m = re.match('pickup', '/pickup.0001036800.meta')
if re.match('pickup', '/pickuP.0001036800.meta'):
    print 'ok'
else:
    print 'no match'
f = 'DiagLAYERS-diapycnal.0002073600.meta'
re.match('(^.+?)\..+', f).groups()[0]
vardata = ['a', 'b']
vardata.remove('a')
vardata
2074320 - 2073840<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1637952559300874, 'fraction_numerical': 0.03470661672908864, 'mean_word_length': 3.7074030552291424, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '249717', 'license': 'MIT', 'n_tokens_mistral': 1674, 'n_tokens_neox': 1556, 'n_words': 366, 'path': 'test_xray.ipynb', 'repo_name': 'rabernat/mitgcm-xray', 'size': '4005'}