<jupyter_start><jupyter_text>An End-to-end Malignancy Detector for Lung Cancer Project Outline1. **Load our raw CT scan data into a form that we can use with `PyTorch`.** Putting raw data into a for usable by PyTorch would be the first step in any project you face.2. **Identify the voxels of potentially malignant tumors** in the lungs using PyTorch to implement a technuque known as **segmentation**. This is roughly akin to producing a heatmap of areas that should be fed into our classifier in step 3. This will alow us to focus on potential malignant tumors inside the lungs, and ignore huge swathes of uninteresting anatomy.3. **Cluster interesting voxels into lumps called nodules,** here we will be finding the rough center of each hotspot on our heatmap.     - Each nodule can be located with by the `index`, `row`, and `column` of their center point. We do this to present a simple, constrained problem to the final classifier.    - This will not involve PyTorch directly, which is why we've pulled this out into a separate step.    - Often when working with multi-step solutions, there will be non-deep-learning glue steps between the larger, deep-learning-powered protions of the project.4. **Classify indentified nodules** as benign or malignant using 3D convolution.     - The features that determine malignancy of a tumor are local to the tumor in question, so this approach should provide a good balance between limiting input data size without excluding relevant information.     - Making scope-limiting decisions like this can help keep each individual task constrained, which can help by limiting the amount of things to examine when troubleshooting.5. **Diagnose** the patient using the combined per-nodule classifications.<jupyter_code>import torch
import numpy as np

print(torch.__version__)
print(np.__version__)<jupyter_output>1.3.0
1.19.2
{'fraction_non_alphanumeric': 0.05042918454935622, 'fraction_numerical': 0.0075107296137339056, 'mean_word_length': 4.567164179104478, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 4, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '474225', 'license': 'MIT', 'n_tokens_mistral': 481, 'n_tokens_neox': 442, 'n_words': 272, 'path': 'malignancy-detector.ipynb', 'repo_name': 'chengliu1107/malignancy-detector-LUNA16', 'size': '1864'}