<jupyter_start><jupyter_text>Mobile-app-analysisApple estore and google play app analysis to identify common features among the most downloaded apps<jupyter_code>from csv import reader
apple_file = list(reader(open('app-store-apple-data-set-10k-apps/AppleStore.csv', 'r')))
android_file = list(reader(open('google-play-store-apps/googleplaystore.csv', 'r')))

apple_header = apple_file[0]
apple_file = apple_file[1:]

android_header = android_file[0]
android_file = android_file[1:]
def explore_data(dataset, start, end, rows_and_columns=False):
    dataset_slice = dataset[start:end]    
    for row in dataset_slice:
        print(row)
        print('\n') # adds a new (empty) line after each row

    if rows_and_columns:
        print('Number of rows:', len(dataset))
        print('Number of columns:', len(dataset[0]))<jupyter_output><empty_output><jupyter_text>Exploring the data sets  Apple store header and first row<jupyter_code>apple_header
explore_data(apple_file, 0, 1, True)<jupyter_output>['1', '281656475', 'PAC-MAN Premium', '100788224', 'USD', '3.99', '21292', '26', '4', '4.5', '6.3.5', '4+', 'Games', '38', '5', '10', '1']


Number of rows: 7197
Number of columns: 17<jupyter_text>Google play header and first row<jupyter_code>android_header
explore_data(android_file, 0, 1, True)<jupyter_output>['Photo Editor & Candy Camera & Grid & ScrapBook', 'ART_AND_DESIGN', '4.1', '159', '19M', '10,000+', 'Free', '0', 'Everyone', 'Art & Design', 'January 7, 2018', '1.0.0', '4.0.3 and up']


Number of rows: 10841
Number of columns: 13<jupyter_text>Cleaning the dataAs discussed [here](https://www.kaggle.com/lava18/google-play-store-apps/discussion/66015latest-600082) the column "Category" is missing at row index 10472. So I removed this line from the dataset.<jupyter_code>explore_data(android_file, 10472, 10473, False)
#del android_file[10472]<jupyter_output><empty_output><jupyter_text>Duplicate EntriesThe google play dataset has some duplicate entries, probaly due to collecting the data at different times.First, I'll remove the duplicate lines, keeping olny one entry for any duplicate. As the lines are identical no information will be lost. After that, I will look at apps with duplicate names but with differences in any other columns.<jupyter_code>def find_duplicate(dataset, multi_index=True):
    unique_apps=[]
    duplicate_apps=[]
    
    if multi_index:
        for row in dataset:
            if row[0] in unique_apps:
                if row[0] not in duplicate_apps:
                    duplicate_apps.append(row[0])
            else:
                unique_apps.append(row[0])
    else:
        for row in dataset:
            if row in unique_apps:
                if row not in duplicate_apps:
                    duplicate_apps.append(row)
            else:
                unique_apps.append(row)
            
    return duplicate_apps
duplicate_apps = find_duplicate(android_file)
len(duplicate_apps)
duplicate = find_duplicate(duplicate_apps,False)
duplicate<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11349389640382712, 'fraction_numerical': 0.041900362916529196, 'mean_word_length': 4.104377104377105, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 19, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '642085', 'license': 'MIT', 'n_tokens_mistral': 1056, 'n_tokens_neox': 967, 'n_words': 270, 'path': 'app-analysis/Mobile-app-analysis.ipynb', 'repo_name': 'hfernandes1507/dataquest-projects', 'size': '3031'}