<jupyter_start><jupyter_text>Expose explicitly missing values with `complete`<jupyter_code>import pandas as pd
import numpy as np 
import janitor
# from http://imachordata.com/2016/02/05/you-complete-me/
df = pd.DataFrame(
        {
            "Year": [1999, 2000, 2004, 1999, 2004],
            "Taxon": [
                "Saccharina",
                "Saccharina",
                "Saccharina",
                "Agarum",
                "Agarum",
            ],
            "Abundance": [4, 5, 2, 1, 8],
        }
    )

df<jupyter_output><empty_output><jupyter_text>Note that Year 2000 and Agarum pairing is missing in the DataFrame above. Let’s make it explicit:<jupyter_code>df.complete('Year', 'Taxon')
# A better viewing based on order
df.complete('Year', 'Taxon', sort = True)<jupyter_output><empty_output><jupyter_text>What if we wanted the explicit missing values for all the years from 1999 to 2004? Easy - simply pass a dictionary pairing the column name with the new values:<jupyter_code>new_year_values = {'Year': range(df.Year.min(), df.Year.max() + 1)}

df.complete(new_year_values, "Taxon")<jupyter_output><empty_output><jupyter_text>You can pass a callable as values in the dictionary:<jupyter_code>new_year_values = lambda year: range(year.min(), year.max() + 1)

df.complete({"Year": new_year_values}, "Taxon", sort = True)<jupyter_output><empty_output><jupyter_text>You can get explcit rows, based only on existing data:<jupyter_code># https://stackoverflow.com/q/62266057/7175713
df = {"Name" : ("Bob", "Bob", "Emma"), 
       "Age" : (23,23,78), 
       "Gender" :("Male", "Male", "Female"), 
       "Item" : ("house", "car", "house"), 
       "Value" : (5,1,3)
        }
df = pd.DataFrame(df)
df<jupyter_output><empty_output><jupyter_text>In the DataFrame above, there is no `car` Item value for the `Name`, `Age`, `Gender`  combination -> `(Emma, 78, Female)`. Pass `(Name, Age, Gender)` and `Item` to explicitly expose the missing row:<jupyter_code>df.complete(('Name', 'Age', 'Gender'), 'Item')<jupyter_output><empty_output><jupyter_text>The example above showed how to expose missing rows on a group basis. There is also the option of exposing missing rows with the `by` parameter:<jupyter_code>df = pd.DataFrame(
        {
            "state": ["CA", "CA", "HI", "HI", "HI", "NY", "NY"],
            "year": [2010, 2013, 2010, 2012, 2016, 2009, 2013],
            "value": [1, 3, 1, 2, 3, 2, 5],
        }
    )

df<jupyter_output><empty_output><jupyter_text>Let's expose all the missing years, based on the minimum and maximum year, for each state:<jupyter_code>result = df.complete(
            {'year': new_year_values},
            by='state',
            sort = True
        )

result<jupyter_output><empty_output><jupyter_text>You can fill the nulls with Pandas' `fillna`:<jupyter_code>result.fillna(0, downcast = 'infer')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1450831024930748, 'fraction_numerical': 0.037742382271468145, 'mean_word_length': 3.6150159744408947, 'pattern_counts': {'":': 7, '<': 37, '<?xml version=': 0, '>': 38, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '430369', 'license': 'MIT', 'n_tokens_mistral': 1097, 'n_tokens_neox': 973, 'n_words': 263, 'path': 'examples/notebooks/complete.ipynb', 'repo_name': 'farhanreynaldo/pyjanitor', 'size': '2888'}