<jupyter_start><jupyter_text><jupyter_code>import pandas as pd
df = pd.DataFrame({'a': ['w'] * 4 + ['x'] * 3 + ['y'] * 2 + ['z']+['v'], 
                   'b': [1, 1, 1, 1, 2, 2, 2, 3, 3, 4,5]})
df<jupyter_output><empty_output><jupyter_text>Encontrar los duplicados<jupyter_code>df.duplicated()<jupyter_output><empty_output><jupyter_text>Podemos usar keep='first' para marcar solo la primera ocurrencia o keep='last' para marcar la última:<jupyter_code>df.duplicated(keep='first')
df.duplicated(keep='last')
df[~ df.duplicated()]
df.duplicated(keep=False)
df[df.duplicated(keep=False)]<jupyter_output><empty_output><jupyter_text>Eliminando duplicados<jupyter_code>df.drop_duplicates()
df.drop_duplicates(['a'],keep='last')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.18326693227091634, 'fraction_numerical': 0.01859229747675963, 'mean_word_length': 7.976190476190476, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '333829', 'license': 'MIT', 'n_tokens_mistral': 324, 'n_tokens_neox': 308, 'n_words': 48, 'path': 'manipulacion_datos_pandas/lectura_manejo_duplicados.ipynb', 'repo_name': 'cristian-rincon/escuela-datascience', 'size': '753'}