<jupyter_start><jupyter_text>First Kaggle Competition> Explaining how I navigated a Kaggle- toc: true - badges: false- comments: true- badges: false- categories: [kaggle] First I wanted to find the most basic image recognition competition to get a feel of how Kaggle works - I landed on [Aerial Cactus Identification](https://www.kaggle.com/c/aerial-cactus-identification). I downloaded all the data which consisted of a test/train folders which had all the images and a train.csv which held all the labels to the train folder images. They also included a sample_submission.csv which is the format they would like to submission to be in. Easy. As I am using the pre-release of fastai2 so my first step is download it and import the vision library<jupyter_code>!pip install git+https://github.com/fastai/fastai2
from fastai2.vision.all import *<jupyter_output><empty_output><jupyter_text>I create a folder and unload all the files. Then I create a path to the image files I want to train on.<jupyter_code>!mkdir kaggle-cactus
path = Path('../kaggle-cactus/')
fnames = get_image_files(path/"train")<jupyter_output><empty_output><jupyter_text>I tell fastai what type of DataBlock we will be using<jupyter_code>dblock = DataBlock(get_items = get_image_files)<jupyter_output><empty_output><jupyter_text>I then direct the datablock to look at my train folderI do a sanity check to see if its pulling correctly<jupyter_code>dsets = dblock.datasets(fnames)
dsets.train[0]<jupyter_output><empty_output><jupyter_text>pass the dataset to a dataloader. A dataloader sets all the preferences up.<jupyter_code>data = DataBlock(blocks = (ImageBlock, CategoryBlock),
                 #how it should split the train
                 splitter=RandomSplitter(),
                 #file names in first column
                 get_x=ColReader(0, pref=path/"train"),
                 #labels in second column
                 get_y=ColReader(1),
                 #add some image augmentation
                 batch_tfms=(Warp(), Zoom(), Rotate()))<jupyter_output><empty_output><jupyter_text>load the csv with all the labels<jupyter_code>import pandas as pd
df = pd.read_csv('../kaggle-cactus/labels.csv')
dls = data.dataloaders(df)<jupyter_output><empty_output><jupyter_text>sanity check to see if the images are being labeled correctly<jupyter_code>dls.show_batch(nrows=1, ncols=5)<jupyter_output><empty_output><jupyter_text>put it all together into a learner<jupyter_code>learn = cnn_learner(dls, resnet34, metrics=error_rate)
learn.fine_tune(5)
learn.show_results()<jupyter_output><empty_output><jupyter_text>lets apply the model to the test<jupyter_code>test_dl = dls.test_dl(get_image_files(path/"test"))<jupyter_output><empty_output><jupyter_text>pass it to the learner and get results (preds)<jupyter_code>preds,_ = learn.get_preds(dl=test_dl)
#only grab the first column and turn into a numpy array
preds.numpy()[:, 0]<jupyter_output><empty_output><jupyter_text>lets read the sample submission to fill in our data<jupyter_code>test_df = pd.read_csv('sample_submission.csv')<jupyter_output><empty_output><jupyter_text>fill in the has_cactus column<jupyter_code>test_df.has_cactus = preds.numpy()[:, 0]<jupyter_output><empty_output><jupyter_text>export<jupyter_code>test_df.to_csv('submission.csv', index=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09265944645006016, 'fraction_numerical': 0.003910950661853189, 'mean_word_length': 5.4437984496124034, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 54, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '392076', 'license': 'Apache-2.0', 'n_tokens_mistral': 1109, 'n_tokens_neox': 1092, 'n_words': 300, 'path': '_notebooks/Kaggle cactus.ipynb', 'repo_name': 'SleepyGinger/mlnotes2', 'size': '3324'}