<jupyter_start><jupyter_text>First day!Congratulations! It's your first day as a data scientist in the company! Your first project is to build a model for predicting if a movie will get a positive or negative review.You need to start exploring your dataset. In order to create a function that will scan each movie review, you want to know how many characters every string has and print the result out together with a statement that indicate what the number refers to. To test if your function works correctly, you are going to start by analyzing only one example.The text of one movie review has been already saved in the variable `movie`.Instructions1. Find out how many characters the variable `movie` has.2. Convert the numeric variable `length_string` to a string representation.3. Concatenate the predefined variable `statement` and the variable `to_string` adding a space between them. Print out the result.<jupyter_code>statement = "Number of characters in this review:"
movie = 'fox and kelley soon become bitter rivals because the new fox books store is opening up right across the block from the small business .'
# Find characters in movie variable
length_string = len(movie)
# Convert to string
to_string = str(length_string)
# Concatenate strings and print result
print(statement + ' ' + to_string)<jupyter_output>Number of characters in this review: 135<jupyter_text>Artificial reviewsWhile checking out the movie reviews in your dataset, you realize that some of them show an atypical pattern. Since you should only include true reviews in your analysis, you decide to extract the suspicious ones that follow this pattern. You want to see if it is possible to artificially create reviews by using the first and last part of one example review and changing a keyword in the middle.The text of two movie reviews has been already saved in the variables `movie1` and `movie2`._Remember:_ The 1st character of a string has index 0.Instructions1. Select the first 32 characters of the variable `movie1` and assign it to the variable `first_part`.2. Select the substring going from the 43rd character to the end of `movie1`. Assign it to the variable `last_part`.3. Select the substring going from the 33rd to the 42nd character of `movie2`. Assign it to the variable `middle_part`.4. Print the concatenation of the variables `first_part`, `middle_part` and `last_part` in that order. Print the variable `movie2` and compare them.<jupyter_code>movie1 = 'the most significant tension of _election_ is the potential relationship between a teacher and his student .'
movie2 = 'the most significant tension of _rushmore_ is the potential relationship between a teacher and his student .'
# Select the first 32 characters of movie1
first_part = movie1[:32]
# Select from 43rd character to the end of movie1
last_part = movie1[42:]
# Select from 33rd to the 42nd character of movie2
middle_part = movie2[32:42]
# Print concatenation and movie2 variable
print(first_part+middle_part+last_part) 
print(movie2)<jupyter_output>the most significant tension of _rushmore_ is the potential relationship between a teacher and his student .
the most significant tension of _rushmore_ is the potential relationship between a teacher and his student .<jupyter_text>PalindromesNext, you are committed to find any peculiarity in the words included in the movie review dataset. A palindrome is a sequence of characters which can be read the same backward as forward, for example: `Madam` or `No lemon, no melon`. You realize that there are some funny movie names that can have this characteristic. You want to make a list of all movie titles that are funny palindromes but you will start by analyzing one example.In python, you can also specify steps by using a third index. If you don't specify the first or second index and the third one is negative, it will return the characters jumping and backwards.The text of a movie review for one example has been already saved in the variable `movie`.Instructions1. Extract the substring from the 12th to the 30th character from the variable `movie` which corresponds to the movie title. Store it in the variable `movie_title`.2. Get the palindrome by reversing the string contained in `movie_title`.3. Complete the code to print out the `movie_title` if it is a palindrome.<jupyter_code>movie = 'oh my God! desserts I stressed was an ugly movie'
# Get the word
movie_title = movie[11:30]

# Obtain the palindrome
palindrome = movie_title[::-1]

# Print the word if it's a palindrome
if movie_title == palindrome:
    print(movie_title)<jupyter_output>desserts I stressed<jupyter_text>Normalizing reviewsIt's time to extract some important words present in your movie review dataset. First, you need to normalize them and then, count their frequency. Part of the normalization implies converting all the words to lowercase, removing special characters and extracting the root of a word so you count the variants as one.So imagine you have the following reviews: `The movie surprises me very much` and `Marvel movies always surprise their audience`. If you count the word frequency, you will count `surprises` one time and `surprise` one time. However, the verb `surprise` appears in both and its frequency should be two.The text of a movie review for only one example has been already saved in the variable `movie`.Instructions1. Convert the string in the variable `movie` to lowercase. Print the result.2. Remove the `$` that occur at the start and at the end of the string contained in `movie_lower`. Print the results.3. Split the string contained in `movie_no_sign` into as many substrings as possible. Print the results.4. To get the root of the second word contained in `movie_split`, select all the characters except the last one.<jupyter_code>movie = '$I supposed that coming from MTV Films I should expect no less$'
# Convert to lowercase and print the result
movie_lower = movie.lower()
movie_lower
# Remove specified character and print the result
movie_no_sign = movie_lower.strip('$')
movie_no_sign
# Split the string into substrings and print the result
movie_split = movie_no_sign.split()
movie_split
# Select root word and print the result
word_root = movie_split[1][:-1]
word_root<jupyter_output><empty_output><jupyter_text>Time to join!While normalizing your text, you noticed that one review had a particular structure. This review ends with the HTML tag `` and it has a lot of commas in different places of the sentence. You decide to remove the tag from the end and use the strategy of splitting the string and joining it back again without the commas.The text of a movie review has been already saved in the variable `movie`.Instructions1. Remove tag `` from the end of the string. Print the results.2. Split the string contained in `movie_tag` using the commas as a separating element. Print the results.3. Join back together the list of substring contained in `movie_no_comma` using a space as a join element. Print the results.<jupyter_code>movie = 'the film,however,is all good<\\i>'
# Remove tags happening at the end and print results
movie_tag = movie.strip('<\i>')
movie_tag
# Split the string using commas and print results
movie_no_comma = movie_tag.split(",")
movie_no_comma
# Join back together and print results
movie_join = ' '.join(movie_no_comma)
movie_join<jupyter_output><empty_output><jupyter_text>Split lines or split the line?You are about to leave work when a colleague asks you to use your string manipulation skills to help him. You need to read strings from a file in a way that if the file contains strings on different lines, they are stored as separate elements. He also wants you to break the strings into pieces if you see that they contain commas.The text of the file has been already saved in the variable `file`.Instructions1. Split the string `file` into many substrings at line boundaries.2. Print out the resulting variable `file_split`.3. Complete the for-loop to split the strings into many substrings using commas as a separator element.<jupyter_code>file = 'mtv films election, a high school comedy, is a current example\nfrom there, director <NAME> wastes no time, taking us into the water on a midnight swim'
# Split string at line boundaries
file_split = file.splitlines()

# Print file_split
file_split

# Complete for-loop to split by commas
for substring in file_split:
    substring_split = substring.split(sep=',')
    print(substring_split)<jupyter_output>['mtv films election', ' a high school comedy', ' is a current example']
['from there', ' <NAME> no time', ' taking us into the water on a midnight swim']<jupyter_text>Finding a substringIt's a new day at work and you need to continue cleaning your dataset for the movie prediction project. While exploring the dataset, you notice a strange pattern: there are some repeated, consecutive words occurring between the character at position 37 and the character at position 41. You decide to write a function to find out which movie reviews show this peculiarity, remembering that the ending position you specify is not inclusive. If you detect the word, you also want to change the string by replacing it with only one instance of the word.Complete the if-else statement following the instructions.The text of three movie reviews has been already saved in the variable `movies`.Instructions1. Find if the substring `actor` occurs between the characters with index `37` and `41` inclusive. If it is not detected, print the statement `Word not found`.2. Replace `actor actor` with the substring `actor` if `actor` occurs only two repeated times.3. Replace `actor actor actor` with the substring `actor` if `actor` appears three repeated times.<jupyter_code>import pandas as pd

movies = pd.Series(data={'200': "it's clear that he's passionate about his beliefs , and that he's not just a punk looking for an excuse to beat people up .",
                         '201': 'I believe you I always said that the actor actor actor is amazing in every movie he has played',
                         '202': "it's astonishing how frightening the actor actor norton looks with a shaved head and a swastika on his chest."})
for movie in movies:
    # If actor is not found between character 37 and 41 inclusive
    # Print word not found
    if movie.find('actor', 37, 42) == -1:
        print('Word not found')
    # Count occurrences and replace two with one
    elif movie.count('actor') == 2:  
        print(movie.replace('actor actor', 'actor'))
    else:
        # Replace three occurrences with one
        print(movie.replace('actor actor actor', 'actor'))<jupyter_output>Word not found
I believe you I always said that the actor is amazing in every movie he has played
it's astonishing how frightening the actor norton looks with a shaved head and a swastika on his chest.<jupyter_text>Where's the word?Before finishing cleaning your dataset, you want to check if a specific word occurs in the reviews. You noticed earlier a specific pattern in the strings. Now, you want to create a function to check if a word is present between characters with index 12, and 50, remembering that ending position is exclusive, and print out the lowest index where this word occurs. There are two methods to handle this situation. You want to see which one works best.The text of two movie reviews has been already saved in the variable `movies`.Instructions1. Find the index where `money` occurs between characters with index 12 and 50. If not found, the method should return -1.2. Find the index where `money` occurs between characters with index 12 and 50. If not found, it should raise an error.<jupyter_code>movies = pd.Series(data={'137': "heck , jackie doesn't even have enough money for a haircut , looks like , much less a personal hairstylist .",
                         '138': "in condor , chan plays the same character he's always played , himself , a mixture of bruce lee and tim allen , a master of both kung-fu and slapstick-fu ."})
for movie in movies:
    # Find the first occurrence of word
    print(movie.find('money', 12, 51))
for movie in movies:
    try:
        # Find the first occurrence of word
        print(movie.index('money', 12, 51))
    except ValueError:
        print('substring not found')<jupyter_output>39
substring not found<jupyter_text>Replacing negationsIn order to keep working with your prediction project, your next task is to figure out how to handle negations that occur in your dataset. Some algorithms for prediction do not work well with negations, so a good way to handle this is to remove either `not` or `n't`, and to replace the next word by its antonym.Let's imagine that you have the string: `The movie isn't good`. You will need to remove `n't` and replace `good` for `bad`. This way, your string ends up being `The movie is bad`. You notice that in the first column of the dataset, you have a string that uses the word `isn't` followed by `important`.The text of this column has been already saved in the variable `movies` so you start working with it.Instructions1. Replace the substring `isn't` with the word `is`.2. Replace the substring `important` with the word `insignificant`.3. Print out the result contained in the variable `movies_antonym`.<jupyter_code>movies = "the rest of the story isn't important because all it does is serve as a mere backdrop for the two stars to share the screen ."
# Replace negations
movies_no_negation = movies.replace("isn't", 'is')

# Replace important
movies_antonym = movies_no_negation.replace('important', 'insignificant')

# Print out
movies_antonym<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05513950073421439, 'fraction_numerical': 0.010279001468428781, 'mean_word_length': 4.605349794238683, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '638335', 'license': 'MIT', 'n_tokens_mistral': 3593, 'n_tokens_neox': 3377, 'n_words': 1985, 'path': 'regular_expressions_in_python/1_basic_concepts_of_string_manipulation.ipynb', 'repo_name': 'vhsenna/datacamp-courses', 'size': '13656'}