<jupyter_start><jupyter_text>NOTE: This assignment is adapted from: https://github.com/ajcr/100-pandas-puzzles/blob/master/100-pandas-puzzles.ipynb 1. Given a list of states = ['VA', 'MD', 'NY', 'NJ', 'TX'] and arbitrarary census data = list(range(10)), construct a MultiIndex object from the product of the two lists. Use it to index a ``Series`` of random numbers. Call this Series s.**Note you will have multiply indexed series with state name as the 1st level of index and census data (0 to 9) as the 2nd level index**<jupyter_code>print([(range(10))])
idx = pd.MultiIndex.from_product([['VA','MD','NY','NJ','TX'],[i for i in range(10)]]) 
idx
type(idx)
s = pd.Series(np.random.rand(50), index=idx)
s<jupyter_output><empty_output><jupyter_text>2. Check whether the index of s is lexicographically sorted. If not, apply sorting to ensure that you can use index slicing.<jupyter_code>try:
    s['VA':'TX']
except KeyError as e:
    # This is the result of the MultiIndex not being sorted.
    print(type(e))
    print(e)
s = s.sort_index()
s<jupyter_output><empty_output><jupyter_text>3. Select the labels 1, 3 and 6 from the second level of the MultiIndexed ``Series``.<jupyter_code>s.loc[:,[1,3,6]]
idx.set_codes([[0,0,0,1,1,1,2,2,2,3,3,3,4,4,4], [1,3,6,1,3,6,1,3,6,1,3,6,1,3,6]])<jupyter_output><empty_output><jupyter_text>4. Slice the ``DataFrame`` df; slice up to 4th state of the first level and from label 8 onwards for the second level.  **If I am slicing a series, my answers will be two ways:**- loc only - indexSlice with loc<jupyter_code># loc only
s.loc[:"TX", 8:9]
# Another way is using Index Slice
idx = pd.IndexSlice
s.loc[idx[:"TX"], idx[8:9]]<jupyter_output><empty_output><jupyter_text>**If I am slicing a dataframe, I am going to convert the series to a dataframe**- unstack- slice using one of two ways:    - loc    - IndexSlice and loc- depending on the requirement, most likely unstack again<jupyter_code>s.index.names = ["State", 'State index']
df = s.to_frame()
type(df)
df
df = df[0]
df.index
df
idx = pd.IndexSlice
l = df.loc[idx[:"TX"], idx[8:9]]
l
# Unstacking it would also good
df = df.unstack()
df
f = df.loc[:"TX", 8:9]
f
f.stack()<jupyter_output><empty_output><jupyter_text>5. Sum the values in s for each label in the first level (you should have Series giving you a total for labels 'VA', 'MD', 'NY', 'NJ' and 'TX').<jupyter_code>s.index.names = ["State", 'State index']
s.sum()
s.groupby('State').sum()
#Another way
s.sum(level=0)
s.sum(level=1)<jupyter_output><empty_output><jupyter_text>6. Using unstack, create a ``DataFrame`` from s. States index will become new columns.<jupyter_code>s.index.names = ["State", 'State index']
s.unstack()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15092250922509226, 'fraction_numerical': 0.026937269372693726, 'mean_word_length': 5.363849765258216, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '600889', 'license': 'MIT', 'n_tokens_mistral': 1024, 'n_tokens_neox': 993, 'n_words': 314, 'path': 'Data 690-3 Assignment12.ipynb', 'repo_name': 'tmarissa/DATA-690-3-Kim', 'size': '2710'}