<jupyter_start><jupyter_text>Forest Fires Feature Selection Mini Project Introduction Data InformationThe Forest Fires data is available at UCI, to reach it please click [here](http://archive.ics.uci.edu/ml/datasets/Forest+Fires).The citation to this data set: [Cortez and Morais, 2007] <NAME> and <NAME>. A Data Mining Approach to Predict Forest Fires using Meteorological Data. In <NAME>, <NAME> and <NAME>., New Trends in Artificial Intelligence, Proceedings of the 13th EPIA 2007 - Portuguese Conference on Artificial Intelligence, December, Guimarães, Portugal, pp. 512-523, 2007. APPIA, ISBN-13 978-989-95618-0-9. Available at: [http://www.dsi.uminho.pt/~pcortez/fires.pdf](http://www3.dsi.uminho.pt/pcortez/fires.pdf)     Attributes:1. `X` - x-axis spatial coordinate within the Montesinho park map: 1 to 9 2. `Y` - y-axis spatial coordinate within the Montesinho park map: 2 to 9 3. `month` - month of the year: 'jan' to 'dec' 4. `day` - day of the week: 'mon' to 'sun' 5. `FFMC` - FFMC index from the FWI system: 18.7 to 96.20 6. `DMC` - DMC index from the FWI system: 1.1 to 291.3 7. `DC` - DC index from the FWI system: 7.9 to 860.6 8. `ISI` - ISI index from the FWI system: 0.0 to 56.10 9. `temp` - temperature in Celsius degrees: 2.2 to 33.30 10. `RH` - relative humidity in %: 15.0 to 100 11. `wind` - wind speed in km/h: 0.40 to 9.40 12. `rain` - outside rain in mm/m2 : 0.0 to 6.4 13. `area` - the burned area of the forest (in ha): 0.00 to 1090.84  Model and Feature Selection Process:I will also try predict the `area` variable via regression models. - First, I fit the data with all features to Random Forest Regression with pruned `depth` hyperparameters. - Then I will use to Lasso(L1 regularization) Regression and ElasticNet(L1+L2 regularization) Regression to select features. I will not use Ridge(L2 regularization) since it does not any exact zero weigthed features. - As last step, I will fit the data to Random Forest Regression with pruned `depth` hyperparameters onto both features selected by Lasso and ElasticNet. Response Variable and Predictors:**Response Variable:** `area` which is the burned area in forest. - We see the original paper used this variable after log transformation since *variable is very skewed towards 0.0*. After fitting the models, the outputs were post-processed with the inverse of the ln(x+1) transform**Predictiors:** We need to assign dummy variables for categorical variables `month` and `day`. The area variable before `log(area+1)` transformation:![AreaBeforeTransformation](./AreaBeforeTransformation.png)The area variable after `log(area+1)` transformation:![AreaAfterTransformation](./AreaAfterTransformation.png) As we can see from the histograms, log transformation helps the area variable to spread out.<jupyter_code>#the standard full feature predictors:
X.head()<jupyter_output><empty_output><jupyter_text>Lasso Regression (L1 Regularization )I apply lasso regression for feature selection. I get 18 features which are important according this criterion.(non-zero weigthed features). We have some features but this features may not always be the same since we do not use cross validation and L1 does not have only unique optimum result.Lasso gives `alpha=0.01` for optimum. I select 18 according to that value.![](./LassoError.png)<jupyter_code>#the after lasso regularization feature predictors:
Xlasso.head()<jupyter_output><empty_output><jupyter_text>ElasticNet Regression (L1+L2 Regularization )I apply ElasticNet regression for feature selection. I get 18 features which are important according this criterion.(non-zero weigthed features). We have some features but this features may not always be the same since we do not use cross validation. However, the optimum result of ElasticNet is unique result on the contrary of lasso.ElasticNet gives `alpha=0.01` for optimum. I select 18 according to that value.![](./ElasticNetError.png)<jupyter_code>#the after elasticNet regularization feature predictors:
XElasticNet.head()<jupyter_output><empty_output><jupyter_text>Random Forest Scores:We see the Lasso features give the best validation score but we cannot say it is the best method to use. Since score values are still low.<jupyter_code># Test scores of RandomForest with depth=5:
RandomForestScores<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08377813877929914, 'fraction_numerical': 0.0329542817359016, 'mean_word_length': 4.95303867403315, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 22, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '223084', 'license': 'MIT', 'n_tokens_mistral': 1352, 'n_tokens_neox': 1200, 'n_words': 566, 'path': 'results/Report.ipynb', 'repo_name': 'cemsinano/ForestFireData-ML', 'size': '4331'}