<jupyter_start><jupyter_text>note: .fillna() does not change the groupby total rows<jupyter_code>pcts_cases_split.drop(columns=pcts_cases_split.columns[4:]).groupby([0,1]).first()
pcts_cases_split.groupby([0,1]).first()<jupyter_output><empty_output><jupyter_text>Type the case string parts<jupyter_code>def string_type(string):
    if string: # ignore None type
        if string.isdigit():
            return "numeric" + str(len(string))
        elif string.isalpha():
            return "alpha"
        elif string.isalnum():
            return "alphanumeric"
pcts_cases_typed = pcts_cases_split.copy()
for c in pcts_cases_split.columns[1:]:
    pcts_cases_typed[c] = pcts_cases_typed[c].map(string_type)
    
pcts_cases_typed.rename(columns={0:'prefix',1:'2',2:'3',3:'4'}, inplace=True)<jupyter_output><empty_output><jupyter_text>Groupby to see different combinations; unfortunately this only shows suffix types that appear in first suffix position<jupyter_code>pcts_cases_typed.groupby(['prefix','2','3','4'],dropna=False).first()<jupyter_output><empty_output><jupyter_text>Alternative to see suffix types in various suffix positions.<jupyter_code>group_suffixes = pcts_cases_typed.set_index(
    ['prefix','2','3']
).stack(
    dropna=False
).reset_index(
).groupby(
    ['prefix','2','3',0], dropna=False
).first(
).reset_index()#.to_csv('pcts_cases_typed_stacked.csv')
group_suffixes.rename(columns={0:'4','level_3':'suffix_position'}, inplace=True)
for c in group_suffixes.columns[1:4]:
    print(f'Position {c} contains alpha characters for the following: \n\
    {set(group_suffixes[group_suffixes[c].str.contains("alpha").fillna(False)]["prefix"].to_list())}')<jupyter_output>Position 2 contains alpha characters for the following: 
    {'ZA'}
Position 3 contains alpha characters for the following: 
    {'VTT', 'TT', 'PS'}
Position 4 contains alpha characters for the following: 
    {'CHC', 'EIR', 'APCW', 'APCE', 'DIR', 'APCSV', 'APCS', 'TT', 'CUB', 'APCH', 'VTT', 'PAR', 'CPC', 'CUZ', 'APCC', 'ADM', 'ENV', 'PS', 'APCNV', 'ZAI', 'ZA', 'AA', 'PM'}<jupyter_text>Check ZA that showed up with alphanumeric year in second position.  Appears to be a year still, but using a convention that must be inquired about with systems or the planners.<jupyter_code>pcts.iloc[pcts_cases_split[(
    pcts_cases_split[0] == 'ZA') & (
    ~pcts_cases_split[1].str.isdigit()
)].index]<jupyter_output><empty_output><jupyter_text>Note that there are some auto-generated cases with suffixes that have a (n) or (N) attached.  The regular expressions won't catch those.  Based on the 2010-2019 backup it appears it is only 3 case records associated with DRB suffix.  More info [here](https://planning.lacity.org/dcpapi/general/prefixsuffix/active/pdf/).<jupyter_code>pcts[pcts.CASE_NBR.str.contains('\(')]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15038924274593066, 'fraction_numerical': 0.014154281670205236, 'mean_word_length': 5.980246913580247, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '63684', 'license': 'Apache-2.0', 'n_tokens_mistral': 1024, 'n_tokens_neox': 996, 'n_words': 218, 'path': 'notebooks/explore-case-strings.ipynb', 'repo_name': 'CityOfLosAngeles/planning-entitlements', 'size': '2826'}