<jupyter_start><jupyter_text>Harmonize the x-values!!!In the regularization step we bin the date range by looking at the range of images. Therefore, for a given fieldwe might have a 10-day bin that goes from Jan 1 to Jan 10 and for another field it goes from Jan 8 to Jan 17. Therefore, let us just to assume the dates are the same as opposed to changing the code to fit the damn clustering step! Anyway, we are picking maximum value of a VI in a 10-day window. The date/x-values are not exact anyway!<jupyter_code>fields_IDs = df.ID.unique()
field_0 = df[df.ID == fields_IDs[0]].copy()
field_0 = field_0[field_0['human_system_start_time'].dt.year == 2016].copy()
print (field_0.shape)
field_0.head(2)
field_1 = df[df.ID == fields_IDs[20]].copy()
field_1 = field_1[field_1['human_system_start_time'].dt.year == 2016].copy()
print (field_1.shape)
field_1 = df[df.ID == fields_IDs[21]]
field_1 = field_1[field_1['human_system_start_time'].dt.year == 2016].copy()
print (field_1.shape)
print (df.shape)
df.drop(field_1.index[35:], inplace=True)
print (df.shape)
print ("it took {:.0f} seconds to run this code.".format(0.1234))
a = 1201026676960 / 1000
time.strftime('%Y-%m-%d', time.localtime(a))
A = pd.read_csv("/Users/hn/Documents/01_research_data/NASA/" + \
                "VI_TS/07_2crop_acr/doubleAcr_perCounty_perCrop_EVI_JFD.csv")
A.head(2)
A.head(2)
A.shape
A.CropTyp.unique
B = A[A.CropTyp != "alfalfa hay"]
B.CropTyp.unique<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1346548188653452, 'fraction_numerical': 0.05263157894736842, 'mean_word_length': 5.0, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 1, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '578416', 'license': 'MIT', 'n_tokens_mistral': 591, 'n_tokens_neox': 538, 'n_words': 148, 'path': 'NASA/Python_codes/k-means-initiation.ipynb', 'repo_name': 'HNoorazar/Kirti', 'size': '1463'}