<jupyter_start><jupyter_text>MIR calculatorThe other day I was talking with a friend that studies Medicine in Spain. After they finish their degree, all students must take the MIR [exam](https://www.wikiwand.com/es/Examen_MIR) that will rank them. The higher you are in that ranking, the better chances you have at selecting the speciality you like from a limited pool of job postings acrross diferent hospitals in the country. The "MIR" is a test exam with 225 questions, each with 4 options. If you get the answer right, you gain 3 points. If you select the wrong answer, you get deducted 1 point.We were discussing if it is optimal to answer a question even if you don't know the answer. Or is it better to not answer it, just in case you get deducted points?She told me she noticed how her mark improved when she started to answer randomly the questions for which she really had no clue what the answer was. I was curious if this was just luck, or if it actually makes sense to answer the unknown questions.This notebook explores to what extent this strategy is optimal, and quantitatively assesses what is the gain in mark you should expect by following the _answer-everything_ strategy.<jupyter_code>import seaborn as sns; sns.set_style("whitegrid")
import random
from matplotlib import pyplot as plt
%matplotlib inline
import numpy as np
class MIR(object):
    '''Class for the MIR exam.
    Parameters
    ----------
    study_level: int
        The level of preparation you have. Must be an integer between
        0 (you answer randomly everything) and 10 (you know all the
        correct answers).
    n_questions: int
        The number of questions that the exam hast. The MIR has 225
        questions, but this could change in principle.
    answer_unknowns: bool
        Whether we will randomly answer the questions we don't know
        or we will decide to not answer them.
    '''
    def __init__(self, study_level, n_questions, answer_unknowns):
        self.study_level = study_level
        self.n_questions = n_questions
        self.answer_unknowns = answer_unknowns

    def solveExam(self):
        '''Answer every question in the exam. Return the mark out of 10'''
        score = 0
        for question in range(0, self.n_questions):
            score = self._answerQuestion(score)
        return((score / (3 * self.n_questions)) * 10)

    def _answerQuestion(self, score):
        '''Answer a question. A higher study_level gives a higher chance
        of getting it right. If we dont know the answer, randomly guess
        if answer_unknowns is True. Returns the updated score'''
        chance = random.randint(1, 10)
        if chance <= self.study_level:
            score += 3
        elif self.answer_unknowns:
            result = random.choice(['a', 'b', 'c', 'd'])
            answer = random.choice(['a', 'b', 'c', 'd'])
            if result == answer:
                score += 3
            else:
                score -= 1
        return(score)

    def _getScoreProgress(self):
        '''Answer all questions in a single exam and returns the score
        progress as a list.'''
        score_Progress = []
        score = 0
        for question in range(0, self.n_questions):
            score = self._answerQuestion(score)
            score_Progress.append((score / (3 * self.n_questions)) * 10)
        return(score_Progress)<jupyter_output><empty_output><jupyter_text>What happens if we take the exam 10000 times randomly, without having studied at all?<jupyter_code>itera = 0
mir = MIR(study_level = 0, n_questions=225, answer_unknowns=True)
max_iter = 10000
while itera < max_iter:
    itera += 1
    plt.plot(mir._getScoreProgress())
plt.ylabel("Score / 10")
plt.xlabel("Question number")
plt.title("{} random MIR exams".format(max_iter))
plt.xlim(0, 225)
plt.show()<jupyter_output><empty_output><jupyter_text>There's chances that you can luck out and get close to a 1.5/10 even if you haven't studied at all! Notice, however, that if you are _unlucky_, you might score a -1.5/10. Ouch!But it looks like a normal distribution centered around 0 to me. Is it actually gaining you anything _on average_? Let's take a look at the distribution of marks for the two types of strategy, at the same study level.<jupyter_code>def solve_ManyExams(study_level, max_iter=1000):
    '''
    Returns two lists:
        distrib1: The marks answering randomly unknown questions
        distrib2: The marks answering only the questions you know.'''
    iteration = 0
    distrib1 = []
    distrib2 = []
    while iteration < max_iter:
        mir1 = MIR(
            study_level=study_level, n_questions=225, answer_unknowns=True)
        mir2 = MIR(
            study_level=study_level, n_questions=225, answer_unknowns=False)
        distrib1.append(mir1.solveExam())
        distrib2.append(mir2.solveExam())
        iteration += 1
    return(distrib1, distrib2)
distrib1, distrib2 = solve_ManyExams(study_level=5)
sns.distplot(distrib1, hist=False, label='Answering unknowns')
sns.distplot(distrib2, hist=False, label='Not answering unknowns')
plt.xlabel("Score / 10")
plt.ylabel("Density")
plt.title("10000 MIRs, study level of 5/10")
plt.legend()
plt.show()
print("Answering unknowns:\t%.2f+-%.2f\n" % (np.mean(distrib1), np.std(distrib1)))
print("Not Answering unknowns:\t%.2f+-%.2f" % (np.mean(distrib2), np.std(distrib2)))<jupyter_output>Answering unknowns:	5.02+-0.43

Not Answering unknowns:	5.00+-0.34<jupyter_text>Answering the questions you don't know doesn't affect your mean score. It just increases the variability, making it possible to get slightly higher (and lower) marks.Let's explore if this is affected by the study level.<jupyter_code>mean_list = []
std_list = []
for study_level in range(0,11):
    distrib1, distrib2 = solve_ManyExams(study_level, max_iter=50000) # Lots of iterations to get clean distributions
    mean_diff = np.mean(distrib1) - np.mean(distrib2)
    std_diff = np.std(distrib1) - np.std(distrib2)
    mean_list.append(mean_diff)
    std_list.append(std_diff)
plt.plot(list(range(0,11)),mean_list, label="Mean score gain")
plt.plot(list(range(0,11)),std_list, label="Variability gain")
plt.xlabel("Study level")
plt.ylabel("Score difference / 10")
plt.xticks([x for x in range(0, 11)])
plt.legend()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09201581027667984, 'fraction_numerical': 0.02292490118577075, 'mean_word_length': 3.4992887624466573, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 16, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '474989', 'license': 'MIT', 'n_tokens_mistral': 1997, 'n_tokens_neox': 1854, 'n_words': 695, 'path': 'MIR calculator.ipynb', 'repo_name': 'jeiros/Jupyter_notebooks', 'size': '6325'}