<jupyter_start><jupyter_text>Digital Cities Project The Digital Cities Project was planned to modernize management and expand access to public services, as well as promoting the development of Brazilian cities through technology.All data were obtained through the website http://www.dados.gov.br/dataset/cidades-digitais<jupyter_code>import numpy as np
import pandas as pd
data = pd.read_csv('aux/datasets/cities.csv', sep = ';', encoding = 'latin-1')
data.head()
data.count()
del data['IBGE']
data.head()<jupyter_output><empty_output><jupyter_text>Group by UF<jupyter_code>dataByUf = data.groupby('UF')
len(dataByUf.groups)
dataByUf.size()
dataByUf.size().sort_values(ascending = False)<jupyter_output><empty_output><jupyter_text>Group by UF and Status<jupyter_code>dataByUfAndStatus = data.groupby(['UF', 'STATUS'])
len(dataByUfAndStatus.groups)
dataByUfAndStatus.size()
dataByUfAndStatus.size().sort_values(ascending = False)<jupyter_output><empty_output><jupyter_text>Set Index - Group By Status<jupyter_code>dataByIndexStatus = data.set_index('STATUS')
dataByIndexStatus = dataByIndexStatus.groupby(level = 'STATUS')
dataByIndexStatus.sum()<jupyter_output><empty_output><jupyter_text>Set Index - Group By UF and Status<jupyter_code>dataByIndexUfAndStatus = data.set_index(['UF', 'STATUS'])
dataByIndexUfAndStatus = dataByIndexUfAndStatus.groupby(level = ['UF', 'STATUS'])
dataByIndexUfAndStatus.sum()<jupyter_output><empty_output><jupyter_text>Aggregate<jupyter_code>dataByIndexUfAndStatus.aggregate([np.sum, np.mean, np.size])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12163892445582586, 'fraction_numerical': 0.0006402048655569782, 'mean_word_length': 9.48993288590604, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '549438', 'license': 'MIT', 'n_tokens_mistral': 555, 'n_tokens_neox': 546, 'n_words': 94, 'path': 'projects/digital-cities/main.ipynb', 'repo_name': 'cfascina/rtaps', 'size': '1562'}