<jupyter_start><jupyter_text>**General instructions:** - This task is intended to be a general DS assessment. If you have applied for an Operational Research role, please contact your recruiter. - Please, explain any step or though that you think may be important to evaluate your task. - The expected programming language is **python**- For the sake of the review, we **strongly prefer** to receive back a jupyter notebook containing all the code, comments and thoughts. This notebook should work from end to end, so we can `restart and run all` or  go through it, cell by cell, if we needed to do so.  TESCO STORES Dataset At Tesco, the location of a retail store plays a huge role in its commercial success. Our Stores Team use various data sources to better understand the potential of candidate locations for new stores in the UK. They need data science help in designing a model that can predict the future sales **[normalised_sales]** of a store based on location characteristics. Your task is to examine the provided dataset and answer the questions below.Dataset files* `tesco-dataset/train.csv`* `tesco-dataset/test.csv`Columns* `location_id`: id of Tesco property location* `normalised_sales`: normalised sales value of Tesco store* `crime_rate`: crime rate in the area (higher means more crime)* `household_size`: mean household size in the area* `household_affluency`: mean household affluency in the area (higher means more affluent)* `public_transport_dist`: index of public transport availability in the area* `proportion_newbuilds`: proportion of newly built property in the area* `property_value`: average property value in the area* `commercial_property`: percentage of commercial properties in the area* `school_proximity`: average school proximity in the area* `transport_availability`: availability of different transport* `new_store`: new Tesco store opened recently* `proportion_nonretail`: proportion of non-retail commercial properties in the area* `competitor_density`: density of competitor retailers* `proportion_flats`: proportion of blocks of flats in the area* `county`: county code of the area  Q1Before diving into the modelling, you are given the dataset and the Stores Team expect you to come back with an analysis of the data and any concerns you may have about it. They would also like to know which other information you think would be useful to collect for future developments.  Q2Build a model that can predict store sales based on the provided area features. Please show how you developed the model and report how well your model is performing. ***Constraint:*** Please use Random Forest as the model family to solve this problem.   Q3The dataset contains a test set of potential store locations. Use your developed model to predict the sales value in these areas and explain what recommendations you would give to the Stores Team to use it. Use any tools that may help you to share your findings with product owners and other non-technical decision makers in the team. Complete this task by explaining how you would improve the current results. -------------  Masked DatasetYou are given the following small dataset, which has been completely masked for privacy reasons. Please train the best model you can come up with to predict the target variable `y` based on the features `x1` and `x2`. Explain every step you take. Assuming that this model will be used for making decisions involving important sums of money, provide any comments that you think you should be giving to the business as a technical expert.Dataset files* `masked_dataset/train.csv`  Things to consider*   Outlier detection/anomaly detection*   encoding*   Cross validation*   Feature engineering* Hyper parameter tuning Time spent 26 Sep 10:30-11:30: 1h<jupyter_code><jupyter_output><empty_output><jupyter_text>Q1TODO* Load train/test data and append it* EDA:** Pandas profiling** Missing values** Outliers** Correlations** Data quality** Graphs** Distributions** Useful fuuture data<jupyter_code><jupyter_output><empty_output><jupyter_text>Q2* Train CV split* Feature engineering* * Polynomials?* Dropping correlated features* Random forest feature selection* RFE feature selection* Random forest* Metrics: R2, loss<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.055932600046805524, 'fraction_numerical': 0.00444652468991341, 'mean_word_length': 5.350668647845468, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '563661', 'license': 'MIT', 'n_tokens_mistral': 1050, 'n_tokens_neox': 986, 'n_words': 600, 'path': 'tesco_DS_assignment.ipynb', 'repo_name': 'konradbachusz/AI_For_Music_Composition', 'size': '4273'}