<jupyter_start><jupyter_text>"for" loops and date-time formats: converting string values into usable time data  In this lecture- [Outcome](Outcome)- [Converting a date string to DateTime format](Converting-a-date-string-to-DateTime-format)- ["for" loops]("for"-loops)- [Converting column 1 DateTime type](Converting-column-1-DateTime-type)- [Creating data giving time in days since 22 March 2014](Creating-data-giving-time-in-days-since-22-March-2014)- [Exporting the converted data](Exporting-the-converted-data) [Back to the top](In-this-lecture) OutcomeAfter this lecture, you will be able to- Convert a string with date data to Julia's DateTime format- Use Dates.datetime2rata() to calculate the number of days between two DateTime dates- Use a for loop to convert the dates of the epidemic data to days since 22 March 2014- Use writedlm to save the converted data as .csv file[Back to the top](In-this-lecture) We start by reading in the data:<jupyter_code>wikiEVDraw = readdlm("wikipediaEVDraw.csv", ',')  #<jupyter_output><empty_output><jupyter_text>[Back to the top](In-this-lecture) Converting a date string to DateTime formatRemarkably enough, data on dates and times are among the fiddliest things a data scientist has to deal with. There is a huge number of different ways in which such data are reported, and moreover there are conflicting standards of how to deal irregularities (month lengths aren't all the same, some years are leap years, time zones shift ...).In consequence, every computer language that deals with date-time data has a rich array of  functions to deal with it. In Julia, they are in a package called Dates. Of this package, we will use the functions DateTime() and Dates.datetime2rata().Why does one of them use a full stop and the other not? The answer is that when you start up Julia, only a few of the functions in the package Dates are visible. These functions include Date() but not datetime2rata(). However, we are able to access the other functions by use of the colon. We will talk more about packages in the next lecture.The DateTime() function uses a format string convert string data such as we see in column one into Julia DateTime data.A format string is something one sees in many computation contexts. Here, it tells Julia in what form to expect the data. Looking at the string, it is a number for the day, then space, then an abbreviation for the month, then a space, then a number for the year. The appropriate format string is therefore "d u y". These formats have limitations: "d" accepts one- and two-digit days (which should always work) and "y" accepts two- and four-digit years (which should mostly work), but "u" accepts only three-letter abbreviations. Unfortunately, data where the month names otherwise abbreviated are fairly common.Here is an example of how the conversion works<jupyter_code>DateTime(wikiEVDraw[1,1], "d u y")<jupyter_output><empty_output><jupyter_text>[Back to the top](In-this-lecture) "for" loopsNow we need to do this conversion for every element in column 1 of the matrix. The way to do this is with a "for" loop."for" loops are extremely important in computing, and in Julia even more so. This is because many items that are vectorised in languages like Matlab and Python are explicitly computed in "for" loops in Julia. It may surprise many of you who know about speeding up computations using vectorisation, but it is frequently the case that a loop in Julia runs *faster* than the equivalent vectorised code."for" loops have a simple structure: the outside is the "for ... end" part and the inside is a code block executed repeatedly. Exactly how many times is determined by the the "for ... end" part.In the two examples below, we use println() to show the value of the variable over which the "for" loop runs. Notice that these values do not have to be a sequence of integers.<jupyter_code>for num = 3:7    # here, the colon is used to specify a range; we will see this again
    println("num is now $num")
end

values = [23, "my name is not a name", 'ℵ']      # an array with some rather odd elements
for x in values    # a for loop can iterate over an array
        println("The value of x is now $x")
    end<jupyter_output>num is now 3
num is now 4
num is now 5
num is now 6
num is now 7
The value of x is now 23
The value of x is now my name is not a name
The value of x is now ℵ<jupyter_text>It is important to get the first line of a "for" loop exactly right. It has the structure "variable = iterable"Here, "iterable" is anything that is arranged in a sequence in memory. Not all types are, but they certainly include ranges (created with the colon operator ":") and any single dimension of an array. The "=" is an assignment operator, and it assigns to "variable" the values in "iterable", one after the other. That is, during each pass through the loop, "variable" has the value of exactly one of the items in "iterable".  [Back to the top](In-this-lecture) Converting column 1 DateTime typeNow we use a "for" loop twice. Firstly we create a one-dimensional array containing just column one---it uses array slicing, for conversion to values with DateTime type.<jupyter_code>col1 = wikiEVDraw[:, 1]  # the colon means all the data in the column, the 1 means the first column<jupyter_output><empty_output><jupyter_text>We use a "for" loop to overwrite the data in the variable col1 with the data in converted using DateTime():<jupyter_code>for i = 1:54
    col1[i] = DateTime(col1[i], "d u y")  # note that this replaces the previous value in col1[i]
end
col1<jupyter_output><empty_output><jupyter_text>Creating data giving time in days since 22 March 2014Finally, we create the variable "epidays". This calls to mind the concept of *epidemic day*, which is simply a way to indicate how long an epidemic has been running. We will assume that the epidemic started on 22 March 2014, with a total of 49 cases, because that is the  first date for which we have data.Note that this is in keeping with the spirit of modelling: we are trying to do the best we can with the data we have. Even when we know that the epidemic has been traced back to a single case in early December 2013, that information is not in the table of data before us. We should not forget about it, but neither should we attempt to include it in the data.The function we use is Dates.datetime2rata(). The "Rata Die days" format is a specialised date format we will not discuss here (see https://en.wikipedia.org/wiki/Rata_Die for information). The important thing is that this function, applied to a given date, gives the number of days since 1 January of the year 0001. As follows:<jupyter_code>Dates.datetime2rata(col1[1])<jupyter_output><empty_output><jupyter_text>Exporting the converted dataWe create a function to express the number of days since epidemic day zero, which is the value of col[54] which is of course 22 March 2014.Then we iterate that function with a for loop over all the elements in col1 to create epidays. Note that the variable epidays is created before the start of the loop. This is, in general, good practice: if you know what array you want to fill, then initialise that array before you start filling it.<jupyter_code># New in 0.6
dayssincemar22(x) = Dates.datetime2rata(x) - Dates.datetime2rata(col1[54])
epidays = Array{Int64}(54)
for i = 1:54
    epidays[i] = dayssincemar22(col1[i])
end<jupyter_output><empty_output><jupyter_text>Finally, we overwite column 1 of our data array with epidays, and save it using writedlm(). It is a good idea to use a new filename, so that all the work that went into extracting the data from wikipedia is not lost. You never know when you might want the original dates again!<jupyter_code>wikiEVDraw[:, 1] = epidays
writedlm("wikipediaEVDdatesconverted.csv", wikiEVDraw, ',')  
#         note the delimiter ... the Julia default is a tab; to get .csv, we must specify the comma<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06310618465801739, 'fraction_numerical': 0.013351807532434815, 'mean_word_length': 4.560224089635854, 'pattern_counts': {'":': 1, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '92732', 'license': 'MIT', 'n_tokens_mistral': 2212, 'n_tokens_neox': 2083, 'n_words': 1218, 'path': 'Julia Scientific Programming - University of Cape Town/JuliaCourseNotebooks/Week2_3-ForLoops.ipynb', 'repo_name': 'mohammadzainabbas/Coursera_Work', 'size': '7939'}