<jupyter_start><jupyter_text>Download and process the weather file for the NUS case study  Author : <NAME> Intent : To generate the Singapore weather file using NUS weather files Date : 28/04/2017 1. Download the Singapore weather data from this [link](https://github.com/architecture-building-systems/cea-reference-case).2. Unzip the file. The unzipped folders are organized based on year. 3. Select all the files corresponding to the year into a single folder on local computer. (`cea-nus-weather` in the temp folder, on windows, `%temp%\cea-nus-weather`)<jupyter_code>import pandas as pd
import datetime
import tempfile
import os
# inputs
YEAR = 2015

# This corresponds to the saved location on the computer in the temp folder
SAVED_LOCATION = os.path.join(tempfile.gettempdir(), 'cea-nus-weather')
today = datetime.datetime(YEAR, 1, 4) # This corresponds to the date of the first excel sheet saved. 
                                      # This represents the first date in a year when the weather readings were recorded
all_data = pd.DataFrame()
all_data = pd.DataFrame()

for i in range(52):    
    current_file = os.path.join(SAVED_LOCATION, 'W_%s.xlsx' % today.strftime('%Y%m%d'))
        
    print current_file
    df = pd.read_excel(current_file)
    all_data = all_data.append(df[2:], ignore_index=True)
    today = today + datetime.timedelta(days=7)

print all_data
all_data.to_csv(os.path.join(SAVED_LOCATION, '%(YEAR)s_weather.csv' % locals()), encoding="utf-8")<jupyter_output>c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150104.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150111.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150118.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150125.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150201.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150208.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150215.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150222.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150301.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150308.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150315.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150322.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150329.xlsx
c:\users\bhargava\appdata\local\temp\cea-nus-weather\W_20150405.xlsx
c:\users\bhargava\appdata\local\te[...]
{'fraction_non_alphanumeric': 0.1248, 'fraction_numerical': 0.0536, 'mean_word_length': 4.484649122807017, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '172741', 'license': 'MIT', 'n_tokens_mistral': 1053, 'n_tokens_neox': 938, 'n_words': 169, 'path': 'notebooks/NUS-weather-file-processing.ipynb', 'repo_name': 'AlexJew/CityEnergyAnalyst', 'size': '2520'}