<jupyter_start><jupyter_text>Next, we need to load the temperature data1. Parsing a fixed-field text file using np.genfromtxt2. Using ranges of NumPy datetime objects<jupyter_code># Let's look at the contents of the daily weather files for station Lihue

open('USW00022536.dly','r').readlines()[:10]<jupyter_output><empty_output><jupyter_text>This format is described in detail in the readme.txt file<jupyter_code>#Relevant lines of the description

open('readme.txt','r').readlines()[98:121]<jupyter_output><empty_output><jupyter_text>This table describes fixed text format, but the value fields always occupy the same columns. NumPy has a special function that can read such a file.<jupyter_code># first write a general call and then figure out the values for all its arguments 
def parsefile(filename):
    return np.genfromtxt(filename,  #numpy needs to be told the filename
                         delimiter = dly_delimiter,  #the size of all fields
                         usecols = dly_usecols,  # which columns we wish to keep
                         dtype = dly_dtype,  #the type of all the fields
                         names = dly_names)  #the names that we want to give to all the fields<jupyter_output><empty_output><jupyter_text>The result will be a NumPy record array<jupyter_code>#Looking at the format above...fill out the arguments

dly_delimiter = [11,4,2,4] + [5,1,1,1] * 31
dly_usecols = [1,2,3] + [4*i for i in range(1,32)]
dly_dtype = [np.int32,np.int32,(np.str_,4)] + [np.int32] * 31
dly_names = ['year','month','obs'] + [str(day) for day in range(1,31+1)]
lihue = parsefile('USW00022536.dly')
lihue
lihue[:10]<jupyter_output><empty_output><jupyter_text>We get a numpy record array with all the information contained in the original file, but needs to be made into a better formThe temperatures for all the days of the month sit on the same row, which is inconvenient because different months have different number of days. Instead, each day should have a separate row. And also each datapoint needs to be associated with a proper NumPy datetime object.<jupyter_code>def unroll(record):
    
    #begin by creating a range of dates that correspond to a row
    #specify the beginning of the month by including only the year and the month in a string fed to the datetime64 object.
    #>>> print '{0:03d}, {1:03d}'.format(1,2) >>> 001, 002 
    startdate = np.datetime64('{}-{:02}'.format(record['year'],record['month']))
    
    #Then create a range of dated using numpy arange starting at the 
    #start date, ending at the start date plus one month and with a step of one day.
    dates = np.arange(startdate,startdate + np.timedelta64(1,'M'),np.timedelta64(1,'D'))
    
    #Next, collect the data for the days from the record
    #Specify the field for which we're going to extract the data by enclosing dates in
    #enumerate by looping over an index and a date at the same time.
    #Divide each datapoint by 10 since temps are specified in tens of degrees.
    
    rows = [(date,record[str(i+1)]/10) for i,date in enumerate(dates)]
    #return rows ---try this
    
    return np.array(rows,dtype = [('date','M8[D]'),('value','d')])
unroll(lihue[0])<jupyter_output><empty_output><jupyter_text>Next, we want to take all the months contained in a file and concatenate them together into a single NumPy record array.We also want to select a single observable, such as  minimum temperature.<jupyter_code>#write a list comprehension containing the unrolled rows for each row of the parsed file.
#And select only those that contain our desired observable

def getobs(filename,obs):
    return np.concatenate([unroll(row) for row in parsefile(filename) if row[2] == obs])

getobs('USW00022536.dly','TMIN')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09315286624203821, 'fraction_numerical': 0.02653927813163482, 'mean_word_length': 4.2202216066482, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '304772', 'license': 'MIT', 'n_tokens_mistral': 1200, 'n_tokens_neox': 1141, 'n_words': 451, 'path': 'NumPy - Weather data Project/02 temperatures.ipynb', 'repo_name': 'AryaDevops/Python-for-DataScience', 'size': '3768'}