<jupyter_start><jupyter_text>VERSION DE DESARROLLO: Niveles de agua del Lago Cocibolca basado en altimetria satelital- 4/24. Ya estan actualizados los datos de G-REALM- 3/22 (todavia no se actualizan los datos de G-REALM), 7, 2/13,7, 1/25,23,18/2021**TO DO:**- Include `lakepy` examples for both metadata and data- Print out start and end datetimes- Extract nivel min & max date per year. Use groupby to extract the nivel and datetime values- Include some WMS map examples of the Global Surface Water (GSW) product, and links and pointers to the GSW explorer application- Include HydroLAKES? https://www.hydrosheds.org/pages/hydrolakes (I've downloaded the global shape file)- DONE. Instalar `lakepy` en el ambiente, y usarlo en este cuaderno**Apuntes:**- G-REALM,     - [IPAD pagina web, Lago Cocibolca](https://ipad.fas.usda.gov/cropexplorer/global_reservoir/gr_regional_chart.aspx?regionid=ca&reservoir_name=Nicaragua)    - [IPAD archivo KML con puntos del "track" del sensor altimetrico](https://ipad.fas.usda.gov/rssiws/ggeoxml/351_Nicaragua.kml)    - Dos tipos de datos:      - Tipo 1: Status products with datum based on a multi-year mean. Pagina de G-REALM para el Lago Nicaragua esta [aqui](https://ipad.fas.usda.gov/cropexplorer/global_reservoir/gr_regional_chart_jason1.aspx?regionid=ca&reservoir_name=Nicaragua). Ejemplo de archivo de datos: `lake0351.10d.1.txt`      - Tipo 2: Near Real Time products with datum based on a single satellite overpass (1 day). Pagina de G-REALM para el Lago Nicaragua esta [aqui](https://ipad.fas.usda.gov/cropexplorer/global_reservoir/gr_regional_chart.aspx?regionid=ca&reservoir_name=Nicaragua). Ejemplo de archivo de datos: `lake0351.10d.2.txt`    - Por cado tipo, datos "smooth" tambien existen: `lake0351.10d.1.smooth.txt` y `lake0351.10d.2.smooth.txt`- [https://appliedsciences.nasa.gov/join-mission/training/english/mapping-and-monitoring-lakes-and-reservoirs-satellite-observations](https://appliedsciences.nasa.gov/join-mission/training/english/mapping-and-monitoring-lakes-and-reservoirs-satellite-observations) - [https://appliedsciences.nasa.gov/our-impact/news/africa-flores-breakthrough-scientist](https://appliedsciences.nasa.gov/our-impact/news/africa-flores-breakthrough-scientist)- https://www.google.com/search?q=altimetria+satelite+radar- https://www.aviso.altimetry.fr/es/casero.html- https://en.wikipedia.org/wiki/Sentinel-6_Michael_Freilich- *From <NAME>*  - 1/25: The USDA G-REALM site is back up and running  - 1/22: Yes, all product creation has stopped while the system transfers to a new server, we have also been short of staff to post a news bulletin. We hope the service will resume by Feb 1st. There is no issue with the Jason-3 data. In fact the follow-on mission to Jason-3 (called Sentinel-6) was launched successfully in November 2020. Jason-3 will stop monitoring at the end of 2021.<jupyter_code>import numpy as np
import pandas as pd
import geopandas as gpd
import matplotlib.pyplot as plt
import matplotlib
import seaborn as sns
import folium
%matplotlib inline
import lakepy as lk

import warnings
warnings.simplefilter('ignore') # filter some warning messages
matplotlib.style.use('ggplot')<jupyter_output><empty_output><jupyter_text>Explorar lakepyIncluyendo accesso a datos del Lago de Managua (Xolotlan)<jupyter_code>lago_nica_lk = lk.search(name='%nicaragua%')
lago_nica_lk_grealm = lk.search(id_No=1943)
lago_nica_lk_grealm.metadata
lago_nica_lk_grealm.data.tail()
lago_nica_lk_hw = lk.search(id_No=129)
lago_nica_lk_hw.data.tail()<jupyter_output><empty_output><jupyter_text>**Resen-as**- En general, le falta "polish" a este paquete- La columna `date` no ha sido convertida en una de tipo `date` o `datetime`. Es tipo `str`, lo que no es ideal del todo.- Los datos de G-REALM no incluyen el nivel *relativo*, no incluye los metadatos adicionales (por ej., lat & lon; satelite; etc), y no especifica cual producto esta siendo usado (tipo 1 vs 2, crudos vs lisos)- En G-REALM, los datos no estan actualizados (terminan en Oct 18)- En Hydroweb, `end_date` no corresponde a la fecha mas reciente en el dataframe de datos (`lago_nica_lk_hw.data`)- `lago_nica_lk_hw.plot_mapview()` estaba muy lento, y lo termine cancelando- Me parece "bloat" incluir `id_No` y `lake_name` en el `data` dataframe- En vez de siempre imprimir los metadatos, deberia no imprimirlos por default y talvez incluir un argumento `verbose=False`<jupyter_code>lago_nica_lk_grealm.data.plot(x='date', y='water_level', figsize=(10,5));
lago_nica_lk_hw.plot_timeseries(how='matplotlib')
lago_managua_lk = lk.search(name='%managua%')
lago_managua_lk_hw = lk.search(id_No=109)
lago_managua_lk_hw.plot_timeseries(how='matplotlib')<jupyter_output><empty_output><jupyter_text>Acceder e ingerir los datos de niveles de agua<jupyter_code>datos_tipo = '1'    # '1' o '2'
crudos_o_lisos = '' # '' o '.smooth'

lago_id = '0351'
fname = f"lake{lago_id}.10d.{datos_tipo}{crudos_o_lisos}.txt"
fname
url = f"https://ipad.fas.usda.gov/lakes/images/{fname}"<jupyter_output><empty_output><jupyter_text>- Try using `infer_datetime_format` or `date_parser` **Para usar los datos "crudos":**```Column 1: Satellite mission nameColumn 2: Satellite repeat cycleColumn 3: Calendar year/month/day of along track observations traversing targetColumn 4: Hour of day at mid point of along track pass traversing targetColumn 5: Minutes of hour at mid point of along track pass traversing targetColumn 6: Target height variation with respect to Jason-2 reference pass mean level (meters, default=999.99)Column 7: Estimated error of target height variation with respect to reference mean level (meters, default=99.999)Column 8: Mean along track Ku-band backscatter coefficient (decibels, default=999.99)Column 9: Wet tropospheric correction applied to range observation (TMR=T/P radiometer, JMR=Jason-1 radiometer,     AMR=Jason-2 and Jason-3 radiometer, ERA=ECMWF ERA Interim Reanalysis model, ECM=ECMWF Operational model,     MER=MERRA model, MIX=combination, U/A=unavailable, N/A=not applicable)Column 10: Ionosphere correction applied to range observation (GIM=GPS model, NIC=NIC09 model, DOR=DORIS,     IRI=IRI2007 model, BNT=Bent model, MIX=combination, U/A=unavailable, N/A=not applicable)Column 11: Dry tropospheric correction applied to range observation (ERA=ECMWF ERA Interim Reanalysis model,     ECM=ECMWF Operational model, U/A=unavailable, N/A=not applicable)Column 12: Instrument operating mode 1 (default=9)Column 13: Instrument operating mode 2 (default=9)Column 14: Flag for potential frozen surface (ice-on=1, ice-off or unknown=0)Column 15: Target height variation in EGM2008 datum (meters above mean sea level, default=9999.99)Column 16: Flag for data source (GDR=0, IGDR=1)```**Para usar los datos "lisos" (`smooth`):**```Column 1: Calendar year/month/dayColumn 2: Hour of dayColumn 3: Minute of hourColumn 4: Smoothed target height variation with respect to Jason-2 reference pass level (meters, default=999.99)Column 5: Smoothed target height variation in EGM2008 datum (meters above mean sea level, default=9999.99)```<jupyter_code>nombres_cols = {
    '': ['satmision', 'satrepetciclo', 
         'fecha', 'hora', 'minuto', 'nivel', 
         'est_error_nivel', 'kuband_scattcoef', 'wettropcorr', 'ionoscorr', 'drytropcorr',
         'instr_opmod1', 'instr_opmod2', 'superfcongelada_flag',
         'nivel_egm2008',
         'fuente_flag'],
    '.smooth': ['fecha', 'hora', 'minuto', 
                'nivel', 
                'nivel_egm2008']
}

skiprows = {
    '1': {'':27, '.smooth':12},
    '2': {'':33, '.smooth':12}
}
df0 = pd.read_csv(
    url, index_col=False, sep='\s+', 
    names=nombres_cols[crudos_o_lisos],
    skiprows=skiprows[datos_tipo][crudos_o_lisos]
)
len(df0)
df0.tail(10)
df0.dtypes<jupyter_output><empty_output><jupyter_text>Pre-procesar y suplementar los datos ingeridos<jupyter_code>df = df0[(df0['fecha'] != 99999999) & (df0['nivel'] < 999)].copy()
len(df)
# Time *must* be zero-padded in conversion from integer to string
df.insert(1, 'FechaTiempo', 
          pd.to_datetime(
              df['fecha'].map(str) 
              + df['hora'].map(lambda nbr: "{0:02d}".format(nbr))
              + df['minuto'].map(lambda nbr: "{0:02d}".format(nbr)),
              format='%Y%m%d%H%M')
         )
df.drop(['fecha', 'hora', 'minuto'], axis=1, inplace=True)
df.FechaTiempo.min(), df.FechaTiempo.max()
df.tail(15)<jupyter_output><empty_output><jupyter_text>**Hay que averiguar porque los datos terminan en Octubre.**<jupyter_code>df['año'] = df.FechaTiempo.dt.year
df['mes'] = df.FechaTiempo.dt.month
df['dia'] = df.FechaTiempo.dt.day
df['dda'] = df.FechaTiempo.dt.dayofyear<jupyter_output><empty_output><jupyter_text>Explorar patrones temporales<jupyter_code>df.plot(x='FechaTiempo', y='nivel', ylim=(-1.5, 1.5), figsize=(14, 7))
plt.ylabel('desviación de elevación (m)')
plt.xlabel('fecha');
sns.boxplot(data=df, x='mes', y='nivel', color='g');
plt.ylim(-1.5, 1.5)
plt.xlabel('mes')
plt.ylabel('desviación de elevación (m)'); 
# df.plot.scatter(x="dda", y="nivel", c="año");<jupyter_output><empty_output><jupyter_text>x-axis date formatting. Probably better to use matplotlib date formatting. See https://www.earthdatascience.org/courses/use-data-open-source-python/use-time-series-data-in-python/date-time-types-in-pandas-python/customize-dates-matplotlib-plots-python/<jupyter_code># markers=True, dashes=False: these have no effect
palette = sns.cubehelix_palette(light=.9, n_colors=len(df['año'].unique()))

sns.relplot(data=df, x="dda", y="nivel", hue="año", kind='line', palette=palette,
            height=5, aspect=1.5)
# Change x axis labelling to show months rather than just dda integers
# 0,365 for start of month; 15,380 for middle of month
plt.xticks(np.linspace(0,365,13)[:-1], 
           ('Ene', 'Feb', 'Mar', 'Abr', 'May', 'Jun', 'Jul', 'Ago', 'Sep', 'Oct', 'Nov', 'Dic')
          );<jupyter_output><empty_output><jupyter_text>Nota: Para estas figuras, hay que eliminar los años incompletos -- el primero y tipicamente el ultimo (a menos que la fecha mas reciente sea mediados de diciembre)<jupyter_code>dfg = df.groupby('año')<jupyter_output><empty_output><jupyter_text>Una sola figura con el promedio, minimo y maximo anual<jupyter_code>fig, ejes = plt.subplots(nrows=2, figsize=(14,10), sharex=True)

dfg.nivel.mean().iloc[1:-1].plot(ax=ejes[0], label='promedio', linewidth=3)
dfg.nivel.max().iloc[1:-1].plot(ax=ejes[0], label='maximo')
dfg.nivel.min().iloc[1:-1].plot(ax=ejes[0], label='minimo')
ejes[0].set_ylabel('nivel anual (m)')
ejes[0].legend()

dfheightrange = dfg.nivel.max().iloc[1:-1] - dfg.nivel.min().iloc[1:-1]
dfheightrange.plot(ax=ejes[1])
ejes[1].set_ylabel('variacion anual del nivel (m)');<jupyter_output><empty_output><jupyter_text>Leer el paso satelital y visualizar en un mapa<jupyter_code>kmlurl = "https://ipad.fas.usda.gov/rssiws/ggeoxml/351_Nicaragua.kml"
gpd.io.file.fiona.drvsupport.supported_drivers['KML'] = 'rw'<jupyter_output><empty_output><jupyter_text>See this good reference for reading KMLs and their complexities: https://gis.stackexchange.com/questions/328525/geopandas-read-file-only-reading-first-part-of-kml<jupyter_code>gdf = gpd.read_file(kmlurl, driver='KML')
len(gdf)
gdf.tail()
gdf.geometry.type.value_counts()
# usando folium.Figure para controlar el taman-o sin introducir "padding"
fig = folium.Figure(width='60%')
mapa = folium.Map(tiles='StamenTerrain').add_to(fig)

folium.GeoJson(
    gdf[gdf.Name == 'Topex/Jason Pass'].geometry,
    tooltip="Paso satelital Topex/Jason"
).add_to(mapa)

# Set the map extent (bounds) to the extent of the bounding box
mapa.fit_bounds(mapa.get_bounds())
mapa<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13201404951597703, 'fraction_numerical': 0.026128673006082414, 'mean_word_length': 5.439051296194154, 'pattern_counts': {'":': 1, '<': 55, '<?xml version=': 0, '>': 54, 'https://': 16, 'lorem ipsum': 0, 'www.': 4, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '442852', 'license': 'MIT', 'n_tokens_mistral': 4618, 'n_tokens_neox': 4293, 'n_words': 1042, 'path': 'libro/cuadernos/_desar_LagoNicaraguaNiveles_altimetro.ipynb', 'repo_name': 'emiliom/nica-natur-cienciadedatos', 'size': '11681'}