<jupyter_start><jupyter_text>![data-x](https://raw.githubusercontent.com/afo/data-x-plaksha/master/imgsource/dx_logo.png)**References and Additional Resources**> [ScikitLearn Regression Metrics](https://scikit-learn.org/stable/modules/model_evaluation.htmlregression-metrics)> [ScikitLearn Classification Metrics](https://scikit-learn.org/stable/modules/model_evaluation.htmlclassification-metrics) ___ NAME: STUDENT ID:___  Homework - Model EvaluationIt is now your turn to hone your model evaluation skills, and develop a logical and replicable machine learning model evaluation procedure. The skills you develop in this homework assignment will come in handy as you transition to more complicated models and datasets over the coming weeks. Using one of the following toy data sets easily loaded from the ScikitLearn library, perform an end-to-end model evaluation.> [Boston House Prices Dataset (Regression)](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_boston.htmlsklearn.datasets.load_boston)> [Diabetes Dataset (Regression)](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.htmlsklearn.datasets.load_diabetes)> [Breast Cancer Wisconsin Dataset (Classification)](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.htmlsklearn.datasets.load_breast_cancer)> [Iris Dataset (Classification)](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_wine.htmlsklearn.datasets.load_wine)**General Recommendations:** > 1. Your code should be legible and well documented.> 2. Your code should seek to 'tell a story' of how you logically progressed through the evaluation process. Not sure how to do such a thing. Here are [some examples](https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks)> 3. Site your sources in a comment line on the code you borrowed, or by referencing it in a markdown cell immediately below.> 4. Work on asking good questions, interpreting your models, visualizing your experiments, and using visualization and clear language to explain your decisions -- effort to master these areas will pay great dividends in the long run.<jupyter_code># load libraries here or as you go
# feel free to use the visualization library of your choice<jupyter_output><empty_output><jupyter_text>Load and Preprocess DataUsing what you learned from the m180_evaluating_and_improving_machine_learning_models notebook, prepare your data for modeling.  0. Set a random seed globally for reproducibility.1. Load and inspect your data. 2. Check for any issues that may influence your model (e.g. heteroscedasticity, data inbalance, missing values, etc.) and correct the issue(s).3. Clearly explain how you identified 2 or create two plots that show how you identified 2 above: One should show how you identified the issue, the other what the result of your correction was.___<jupyter_code># your code here (use as many cells as you need)
# your code here (use as many cells as you need)
# your code here (use as many cells as you need)<jupyter_output><empty_output><jupyter_text>Establish a Baseline Model> 1. Pick a baseline model and fit the model. > 2. Elect 2 the metrics you will use to evaluate this baseline model against other models (e.g. RMSE, Accuracy, etc.) and print them out.> 3. Write a brief description of why you chose the baseline model you did and comment on the metrics you will be using to compare your baseline against alternate models. > 4. Visualize your metrics of interest (Optional for Undergrads; Required for Grad Students).<jupyter_code># your code here (use as many cells as you need)
# your code here (use as many cells as you need)
# your code here (use as many cells as you need)<jupyter_output><empty_output><jupyter_text>Model Selection> 1. Pick two alternative models and train them (do not pre-tune the alternative models). > 2. Compare your alternatives to the baseline model using the metrics you chose previously.> 3. Write a brief description of why you chose the two alternative models, and elaborate on why a particular model did better on the particular data set you used, as well as how you discerned this.> 4. Visualize your metrics of interest accross all models (Optional for Undergrads; Required for Grad Students).<jupyter_code># your code here (use as many cells as you need)
# your code here (use as many cells as you need)
# your code here (use as many cells as you need)<jupyter_output><empty_output><jupyter_text>Model Evaluation and Tunning> 1. Using the best model from your previous work, use metrics and visualizations to diagnose problems with your model.> 2. Propose solutions to any issues to your model and implement them (e.g. High bias --> feature engineering; High variance --> regularization). Breifly explain your reasoning.> 3. Iteratively repeat process until you deem appropriate to stop. Explain how you knew when to stop the iterative process.> 4. Write a bried summary of the entire process, key observations, and conclusions. For good examples of how to create interesting and engaging jupyter notebooks, see [here](https://github.com/jupyter/jupyter/wiki/A-gallery-of-interesting-Jupyter-Notebooks)**Note:** at this time, take care to only implement one change at a time and re-evaluate your model prior to making any additional changes. Try to adhere to an [Occam Learning](https://en.wikipedia.org/wiki/Occam_learning) approach.<jupyter_code># your code here (use as many cells as you need)
# your code here (use as many cells as you need)
# your code here (use as many cells as you need)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07099751508697195, 'fraction_numerical': 0.004614838480653177, 'mean_word_length': 5.261111111111111, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 45, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '55837', 'license': 'Apache-2.0', 'n_tokens_mistral': 1542, 'n_tokens_neox': 1455, 'n_words': 699, 'path': '01-data-x-fundamentals/m180-cross-validation-regularization/m180-B/hw-m180-evaluating_and_tunning_machine_learning_models.ipynb', 'repo_name': 'UCBerkeley-SCET/DataX-Berkeley', 'size': '5634'}