<jupyter_start><jupyter_text>______Copyright by Pierian Data Inc.For more information, visit us at www.pieriandata.com  Time Methods  Python Datetime ReviewBasic Python outside of Pandas contains a datetime library:<jupyter_code>from datetime import datetime
# To illustrate the order of arguments
my_year = 2017
my_month = 1
my_day = 2
my_hour = 13
my_minute = 30
my_second = 15
# January 2nd, 2017
my_date = datetime(my_year,my_month,my_day)
# Defaults to 0:00
my_date 
# January 2nd, 2017 at 13:30:15
my_date_time = datetime(my_year,my_month,my_day,my_hour,my_minute,my_second)
my_date_time<jupyter_output><empty_output><jupyter_text>You can grab any part of the datetime object you want<jupyter_code>my_date.day
my_date_time.hour<jupyter_output><empty_output><jupyter_text>Pandas Converting to datetimeOften when data sets are stored, the time component may be a string. Pandas easily converts strings to datetime objects.<jupyter_code>import pandas as pd
myser = pd.Series(['Nov 3, 2000', '2000-01-01', None])
myser
myser[0]<jupyter_output><empty_output><jupyter_text>pd.to_datetime()https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.htmlconverting-to-timestamps<jupyter_code>myser
# Q 将myser变成如下
# A:
pd.to_datetime(myser)
pd.to_datetime(myser)[0]
obvi_euro_date = '31-12-2000'
pd.to_datetime(obvi_euro_date) 
# 10th of Dec OR 12th of October?
# We may need to tell pandas
euro_date = '10-12-2000'
#将10设为日，输出如下
# A:
pd.to_datetime(euro_date) 
pd.to_datetime(euro_date,dayfirst=True)<jupyter_output><empty_output><jupyter_text>Custom Time String FormattingSometimes dates can have a non standard format, luckily you can always specify to pandas the format. You should also note this could speed up the conversion, so it may be worth doing even if pandas can parse on its own. A full table of codes can be found here: https://docs.python.org/3/library/datetime.htmlstrftime-and-strptime-format-codes<jupyter_code>style_date = '12--Dec--2000'
#Q custome time string formatting (Hint: use %)
pd.to_datetime(style_date,format='%d--%b--%Y')
pd.to_datetime(style_date, format='%d--%b--%Y')
strange_date = '12th of Dec 2000'
pd.to_datetime(strange_date)<jupyter_output><empty_output><jupyter_text>DataRetail Sales: Beer, Wine, and Liquor StoresUnits:  Millions of Dollars, Not Seasonally AdjustedFrequency:  MonthlyU.S. Census Bureau, Retail Sales: Beer, Wine, and Liquor Stores [MRTSSM4453USN], retrieved from FRED, Federal Reserve Bank of St. Louis; https://fred.stlouisfed.org/series/MRTSSM4453USN, July 2, 2020.<jupyter_code>sales = pd.read_csv('RetailSales_BeerWineLiquor.csv')
sales
sales.iloc[0]['DATE']
type(sales.iloc[0]['DATE'])
# Q 将列表中的DATE 的datatype 改为timestamp
type(sales.iloc[0]['DATE'])
#type(a)要注意a不一样 输出也不一样。a如果是series会输出series,是特定的行才会输出是str还是timestamp;
type(sales['DATE'])
sales['DATE'] = pd.to_datetime(sales['DATE'])
sales
sales.iloc[0]['DATE']
type(sales.iloc[0]['DATE'])<jupyter_output><empty_output><jupyter_text>------  Attempt to Parse Dates Automatically**parse_dates** - bool or list of int or names or list of lists or dict, default FalseThe behavior is as follows:    boolean. If True -> try parsing the index.    list of int or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3 each as a separate date column.    list of lists. e.g. If [[1, 3]] -> combine columns 1 and 3 and parse as a single date column.    dict, e.g. {‘foo’ : [1, 3]} -> parse columns 1, 3 as date and call result ‘foo’    If a column or index cannot be represented as an array of datetimes, say because of an unparseable value or a mixture of timezones, the column or index will be returned unaltered as an object data type. For non-standard datetime parsing, use pd.to_datetime after pd.read_csv. To parse an index or column with a mixture of timezones, specify date_parser to be a partially-applied pandas.to_datetime() with utc=True. See Parsing a CSV with mixed timezones for more.<jupyter_code># Parse Column at Index 0 as Datetime
sales = pd.read_csv('RetailSales_BeerWineLiquor.csv',parse_dates=[0])
sales
type(sales.iloc[0]['DATE'])<jupyter_output><empty_output><jupyter_text>Resample A common operation with time series data is resampling based on the time series index. Let's see how to use the resample() method. [[reference](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.resample.html)]<jupyter_code># Our index
sales.index
# Reset DATE to index
sales = sales.set_index("DATE")
sales<jupyter_output><empty_output><jupyter_text>When calling `.resample()` you first need to pass in a **rule** parameter, then you need to call some sort of aggregation function.The **rule** parameter describes the frequency with which to apply the aggregation function (daily, monthly, yearly, etc.)It is passed in using an "offset alias" - refer to the table below. [[reference](http://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.htmloffset-aliases)]The aggregation function is needed because, due to resampling, we need some sort of mathematical rule to join the rows (mean, sum, count, etc.)     TIME SERIES OFFSET ALIASESALIASDESCRIPTIONBbusiness day frequencyCcustom business day frequency (experimental)Dcalendar day frequencyWweekly frequencyMmonth end frequencySMsemi-month end frequency (15th and end of month)BMbusiness month end frequencyCBMcustom business month end frequencyMSmonth start frequencySMSsemi-month start frequency (1st and 15th)BMSbusiness month start frequencyCBMScustom business month start frequencyQquarter end frequencyintentionally left blankALIASDESCRIPTIONBQbusiness quarter endfrequencyQSquarter start frequencyBQSbusiness quarter start frequencyAyear end frequencyBAbusiness year end frequencyASyear start frequencyBASbusiness year start frequencyBHbusiness hour frequencyHhourly frequencyT, minminutely frequencySsecondly frequencyL, msmillisecondsU, usmicrosecondsNnanoseconds<jupyter_code>sales.head()
# Yearly Means
sales.resample(rule='A').mean()<jupyter_output><empty_output><jupyter_text>Resampling rule 'A' takes all of the data points in a given year, applies the aggregation function (in this case we calculate the mean), and reports the result as the last day of that year. Note 2020 in this data set was not complete.  .dt Method CallsOnce a column or index is ina  datetime format, you can call a variety of methods off of the .dt library inside pandas:https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.dt.html<jupyter_code>sales = sales.reset_index()
sales
help(sales['DATE'].dt)
sales['DATE'].dt.month
sales['DATE'].dt.is_leap_year<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09208131067961164, 'fraction_numerical': 0.018962378640776698, 'mean_word_length': 6.127567567567567, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 45, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '204983', 'license': 'Apache-2.0', 'n_tokens_mistral': 2271, 'n_tokens_neox': 2098, 'n_words': 684, 'path': '03-Pandas/08-Time-Methods.ipynb', 'repo_name': 'lili2hou/DSCIFinal', 'size': '6592'}