<jupyter_start><jupyter_text>Explore the dataWe have downloaded the ICESat-2 data and saved them as `./download/processed_ATL03_20190805232841_05940403_004_01.h5`. Before diving into a specific analysis routine, let us see if we can have a general overview about the data using the Jupyter tools. For this stage, we want tools that provide quick access to data, preferably in many ways. And we also want to have some funcionaility to manually navigate to different part of the data. Jupyter's language-agnostic nature (i.e., not bonded to any specific programming languages) and support to interactive plotting widgets are designed to address these needs.```{admonition} Goals:class: tipExplore the data file, including its data structure, size, geospatial information, and so on.``` Steps Check data structureSince the data are stored using the HDF5 format (as indicated by the file extension), we will need certain tools to read them. For example, we can use the `h5ls` command-line tool to have a quick look of the data structure. On Jupyter notebook, we can use the `!` character to use any command-line tool, and we can even pass the variable defined in the other cell (Python or Shell block) to the `h5ls` command.<jupyter_code>filename = 'download/processed_ATL03_20190805232841_05940403_004_01.h5'<jupyter_output><empty_output><jupyter_text>This string variable is now shared by the Python kernel and the shell environment. Note that we also use `grep` here because the full, nested data structure is very long, and we are only interested in the gt1l beam (which {cite}`Herzfeld2021` use in their study).<jupyter_code>!h5ls -r $filename | grep ^/gt1l/heights
!h5ls -r $filename | grep ^/gt1l/geolocation/segment<jupyter_output>/gt1l/heights            Group
/gt1l/heights/delta_time Dataset {312012/Inf}
/gt1l/heights/dist_ph_across Dataset {312012/Inf}
/gt1l/heights/dist_ph_along Dataset {312012/Inf}
/gt1l/heights/h_ph       Dataset {312012/Inf}
/gt1l/heights/lat_ph     Dataset {312012/Inf}
/gt1l/heights/lon_ph     Dataset {312012/Inf}
/gt1l/heights/pce_mframe_cnt Dataset {312012/Inf}
/gt1l/heights/ph_id_channel Dataset {312012/Inf}
/gt1l/heights/ph_id_count Dataset {312012/Inf}
/gt1l/heights/ph_id_pulse Dataset {312012/Inf}
/gt1l/heights/quality_ph Dataset {312012/Inf}
/gt1l/heights/signal_conf_ph Dataset {312012/Inf, 5}
/gt1l/geolocation/segment_dist_x Dataset {993/Inf}
/gt1l/geolocation/segment_id Dataset {993/Inf}
/gt1l/geolocation/segment_length Dataset {993/Inf}
/gt1l/geolocation/segment_ph_cnt Dataset {993/Inf}<jupyter_text>Load the dataNow we use h5py (the python library for working with HDF5) and numpy to open the file and load the data we want.<jupyter_code>import h5py
import numpy as np
with h5py.File(filename, 'r') as f:
    lon_ph = f['gt1l']['heights']['lon_ph'][:]    # photon longitude (x)
    lat_ph = f['gt1l']['heights']['lat_ph'][:]    # photon latitude  (y)
    h_ph = f['gt1l']['heights']['h_ph'][:]        # photon elevation (z), in m
    dist_ph = f['gt1l']['heights']['dist_ph_along'][:]            # photon horizontal distance from the beginning of the parent segment, in m
    seg_length = f['gt1l']['geolocation']['segment_length'][:]    # horizontal of each segment, in m
    seg_ph_count = f['gt1l']['geolocation']['segment_ph_cnt'][:]  # photon count in each segment, in m<jupyter_output><empty_output><jupyter_text>We can easily check the content and statistical information of each variable. For example:<jupyter_code>print(sum(seg_ph_count))    
print(h_ph.shape[0])        # this should equal to the summation of the photon count.<jupyter_output>312012
312012<jupyter_text>Prepare the dataWe often need to apply a few steps to the raw data before visualizing or analyzing them. Take these ICESat-2 data for example: if we want to plot the elevation along this track, we need the distance along the track as x values, which is not provided but can be calculated on our own. Using Jupyter Notebook, we can quickly design a Python function for this variable.<jupyter_code>def make_dist_alongtrack(ph_count, seg_length, dist_ph):
    '''
    For detailed explanation of each variable and reasoning of the code, see ICESat-2 ATL03 documentation.
    '''
    repeat_num = np.cumsum(seg_length) - seg_length[0]
    dist_alongtrack = np.repeat(repeat_num, ph_count)
    dist_alongtrack += dist_ph
    return dist_alongtrack
dist_alongtrack = make_dist_alongtrack(seg_ph_count, seg_length, dist_ph)   # distance along track for each photon, in m<jupyter_output><empty_output><jupyter_text>(sec:explore-plot)= Plot the dataPlotting the data is a great way to obtain a brief overview. Using interactive matplotlib figures powered by `ipympl`, we can speed up the exploration and quickly focus on the key elements of our data.Use the following Notebook command to activate interactive matplotlib environment:<jupyter_code>%matplotlib widget<jupyter_output><empty_output><jupyter_text>And then we import matplotlib.<jupyter_code>import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Now every single figure will come with a control panel, and we can use the buttons to pan, zoom, and save the figure. This is especially very helpful in our case because crevasses are small scale features and do not show everywhere along the track. We have to really zoom in to a certain area in order to see them.<jupyter_code>fig, ax = plt.subplots(1, 1, figsize=(7, 3))
ax.plot(dist_alongtrack, h_ph, '.', markersize=1)<jupyter_output><empty_output><jupyter_text>After a careful check up, we are able to locate the segment where [Figure 6a](sec:ideas-goals) of {cite}`Herzfeld2021` uses. (Note that the along-track distance readings are different because we are using a subset of the original data granule .)<jupyter_code>fig, ax2 = plt.subplots(1, 1, figsize=(7, 3))
ax2.plot(dist_alongtrack, h_ph, '.', markersize=1)
ax2.set_xlim(15090, 16090)
ax2.set_ylim(320, 363)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08767215317433658, 'fraction_numerical': 0.03997312730937185, 'mean_word_length': 5.255252100840337, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '582072', 'license': 'MIT', 'n_tokens_mistral': 2065, 'n_tokens_neox': 1895, 'n_words': 674, 'path': 'docs/icesat2-crevasses/explore.ipynb', 'repo_name': 'geostacks/toy-research-workflow', 'size': '5954'}