<commit_before>
Time to try experiments in Neural Network, Deep Learning and any other AI methods :)
Neural Network is a universal approximator, which means you can use it to implment other machine learning algorithms


*****************************************************************

RESOURCES

* Database
  * ImageNet: http://www.image-net.org
    * Here, you can download images in different format
* Standford CNN for visual Recognition lectures: http://cs231n.stanford.edu/syllabus.html
  * I think, once you started to learn deep learning, you will really feel it's so deep to learn....
  * My notes: https://github.com/hanhanwu/Hanhan_Data_Science_Resources2/blob/master/Standford_CNN_Notes1.pdf

* Here is a bunch of libraries, tutorials you can try: https://www.analyticsvidhya.com/blog/2016/08/deep-learning-path/

* 10 Deep Learning Architectures: https://www.analyticsvidhya.com/blog/2017/08/10-advanced-deep-learning-architectures-data-scientists/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29

* Fundamentals of neural network: https://www.analyticsvidhya.com/blog/2016/03/introduction-deep-learning-fundamentals-neural-networks/
  * It's such a great feeling to review and to understand deeper
  * After learning NN a while, are you going to forget about the relationship between weights, activation function and EACH neuron like I did? This article is a great source to review. So! Each neuron has an activation function, the input for this single neuron can come from multiple sources, each input and bias could has weight.
  * I think, the activation function can be whatever you defined. The purpose for defining an activation is to guarantee it will give the output you want. For example, the input is x1, x2 and the output should be `x1 AND x2`, so your activation function should use mathematical method to guarantee the output will be `x1 AND x2` when the input is x1, x2.
  * The power of multiple layers of NN - you can break a complex problem into small problems, and each NN layer deals with one of the small problem, finally, you get what you finally want
  * The number of neurons in the output layer will depend on the type of problem. It can be 1 for regression or binary classification problem or multiple for multi-class classification problems.
  * <b>The ultimate objective is to update the weights of the model in order to minimize the loss function</b>. The weights are updated using a back-propogation algorithm.
  * `A XNOR B = (A' AND B') OR (A AND B) = NOT ((A OR B) AND (A' OR B'))`, `A' = NOT A`
  * Each optimization includes:
    * Select a network architecture, i.e. number of hidden layers,  number of neurons in each layer and activation function
    * Initialize weights randomly
    * Use forward propagation to determine the output node
    * Find the error of the model using the known labels
    * <b>Back-propogate the error</b> into the network and determine the error for each node
    * Update the weights to minimize the loss function
  * During back-propagation:
    * <b>The error at a node is based on weighted sum of errors on all the nodes of the next layer which take output of this node as input.</b>
  * Important params for optimizing neural network
    * Type of architecture
    * Number of Layers
    * Number of Neurons in a layer
    * Regularization parameters
    * Learning Rate
    * Type of optimization / backpropagation technique to use
    * Dropout rate
    * Weight sharing
  * Think about the activation function, the architecture, do you think Deep Learning can really generate infinite number of solutions?! So powerful isn't it? The only problem is, you still have to have ground truth first, and for Deep Learning, the ground truth has to be more accurate and informative
    
* Core concepts of neural network: https://www.analyticsvidhya.com/blog/2016/08/evolution-core-concepts-deep-learning-neural-networks/
  * The meaning of "sample", "epoch", "batch": https://keras.io/getting-started/faq/#what-does-sample-batch-epoch-mean
    * <b>Sample</b>: One element of a dataset. Such as, 1 image, 1 audio file
    * <b>Batch</b>: A set of N samples. The samples in a batch are processed independently, in parallel. If training, a batch results in only one update to the model. A batch generally approximates the distribution of the input data better than a single input. <b>The larger the batch, the better the approximation</b>, but also takes longer time.
    * <b>Epoch</b>: Epoch: an arbitrary cutoff, generally defined as "one pass over the entire dataset", used to separate training into <b>distinct phases</b>, which is useful for <b>logging and periodic evaluation</b>. When using Keras `evaluation_data` or `evaluation_split` with the `fit` method of Keras models, <b>evaluation will be run at the end of every epoch</b>.
    
* Activation functions and when to ues them: https://www.analyticsvidhya.com/blog/2017/10/fundamentals-deep-learning-activation-functions-when-to-use-them/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29
  * They help neural network choose the useful points and suppress irrelevant info. They can decide whether a neuron should be activated or not, whether the information the neuron is receiving is relevant or should be ignored
  * Without activation function, the weights, bias will simply do a linear transformation, and the neural network will work as a linear regression problem
  * Gradient helps back-propagation. <b>To check gradient, you just check derivate at each data point</b>
  * <b>Binary Step Function</b>
    * Threshold based classifier, decide whether or not to activate the neuron
    * Only serves for binary classification, cannot work for multi-class classification
    * The gradient is 0, and cannot help back-propagation. Because back-propagation expects to use gradients to update errors and improve the model, but this function will set gradients to 0 and cannot really make any improvement
  * <b>Linear Function</b>
    * When you have multiple classes, choose the one with max value
    * But for linear function, the derivate is constant, which means gradient will be the same each time, still cannot help back-propagation
  * <b>Sigmoid Function</b>
    * `f(x)=1/(1+e^-x)`
    * non-linear
    * y ranges between [0,1], x ranges between [-infinite, infinite]. But because of the y range, sigmoid function is not symmetric around the origin and the values received are all positive
    * When gradient is approaching 0 (sigmoid curve is flat), the neuron is not really learning
  * <b>Tanh</b>
    * Scaled version of sigmoid function
    * `tanh(x)=2sigmoid(2x)-1`
    * It works similar to the sigmoid function but is symmetric over the origin. it ranges from -1 to 1
    * Your choice of using sigmoid or tanh would basically depend on the requirement of gradient in the problem statement
    * But similar to the sigmoid function we still have the vanishing gradient problem. When the gradient approaches to 0, the neuron is not really learn
  * <b>ReLU</b>
    * `f(x)=max(0,x)`, f(x)= x if x>= 0, f(x)=0 if x<0
    * non-linear
    * The main advantage of using the ReLU function over other activation functions is that <b>it does not activate all the neurons at the same time</b>.
    * But the gradient is 0 for x<0, which made the neurons die for activations in that region. Because it's not learning but x is always negative, so it's never learn
    * ReLU function should <b>only be used in the hidden layers</b>
  * <b>Leaky ReLU</b>
    * f(x)=ax if x<0, f(x)=x if x>=0
    * Improved ReLU
    * By doing this, it removes 0 gradient and therefore no dead neurons
    * In case of a <b>parameterised ReLU function</b>, ‘a‘ is also a trainable parameter. The network also learns the value of ‘a‘ for faster and more optimum convergence. The parametrised ReLU function is used when the leaky ReLU function still fails to solve the problem of dead neurons and the relevant information is not successfully passed to the next layer.
  * <b>Softmax</b>
    * Idealy used we you want the output is showing probability, because all the output are in range between [0,1]
  * <b>Summarized Suggestions from the author</b>
    * Sigmoid functions and their combinations generally work better in the case of classifiers
    * Sigmoids and tanh functions are sometimes avoided due to the vanishing gradient problem
    * ReLU function is a general activation function and is used in most cases these days
    * If we encounter a case of dead neurons in our networks the leaky ReLU function is the best choice
    * Always keep in mind that ReLU function should only be used in the hidden layers
    * As <b>a rule of thumb</b>, you can begin with using ReLU function and then move over to other activation functions in case ReLU doesn’t provide with optimum results
    
* Deep Leaning for Computer Vision: https://www.analyticsvidhya.com/blog/2016/04/deep-learning-computer-vision-introduction-convolution-neural-networks/?lipi=urn%3Ali%3Apage%3Ad_flagship3_feed%3BxPn6EhynRquw3Evzrg79RA%3D%3D
  * Detailed analysis of activation functions
    * Sigmoid (not recommended for CNN)
      * Kill the gradient
      * Outputs are not zero-centered since it's between [0,1]
      * Taking the exp() is computationally expensive
    * Tanh (not recommended for CNN)
      * Still kill the gradient
      * But output can be zero-centered
    * ReLU (Rectified Linear Unit)
      * <b>most commonly used for CNN</b>
      * Gradient won’t saturate in the positive region
      * Computationally very efficient as simple thresholding is required
      * Empirically found to converge faster than sigmoid or tanh.
      * Drawbacks - Output is not zero-centered and always positive
      * Drawbacks - Gradient is killed for x<0. Few techniques like <b>leaky ReLU</b> and <b>parametric ReLU</b> are used to overcome this
      * Drawbacks - Gradient is not defined at x=0. But this can be easily catered using <b>sub-gradients</b> and posts less practical challenges as x=0 is generally a rare case
    * Data Preprocessing
      * Same image size
      * Mean centering: subtract mean value from each pixel
    * Weights Initialization
      * All zeros - a bad idea
      * Gaussian Random Variables: you need to play with the standard deviation of the gaussian distribution which works well for your network....
      * Xavier Initialization: It suggests that variance of the gaussian distribution of weights for each neuron should depend on the number of inputs to the layer. A recent research suggested that for ReLU neurons, the recommended update is, `np.random.randn(n_in, n_out)*sqrt(2/n_in)`
      * You might be surprised to know that 10-20% of the ReLUs might be dead at a particular time while training and even in the end.
    * CNN
      * The <b>first layer</b> will try to detect edges and form templates for <b>edge detection</b>
      * The subsequent layers will try to combine them into <b>simpler shapes</b> and eventually into <b>templates of different object positions, illumination, scales, etc</b>
      * The final layers will <b>match an input image with all the templates</b> and the <b>final prediction</b> is like a weighted sum of all of them.
    * CNN important layers
      * Convolution Layer
      * ReLU layer
      * Pooling Layer: Using padding in convolution layer, the image size remains same. So, pooling layers are used to reduce the size of image.
      * Fully Connected Layer: Each pixel is considered as a separate neuron just like a regular neural network. The last fully-connected layer will contain as many neurons as the number of classes to be predicted
    * In this article, I think the correct formula for calculating output size should be: `(W-F+2P)/S + 1`
    * To calculate zero-padding size: `(F-1)/2`
    * Filters might be called kernels sometimes
    * Alexnet
    ![AlexNet](https://www.analyticsvidhya.com/wp-content/uploads/2016/03/fig-8.png)
    * It has code example using GraphLab with pre-trained model. Pre-trained model may increase the accuracy but difficult to find and if you use CPU, may take very long time to run
    

* Digits recognition with <b>TensorFlow</b> [lower level library]: https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29
  * <b>Tensorflow Resources</b>: https://github.com/jtoy/awesome-tensorflow
  * Tensorflow Models: https://github.com/tensorflow/models
  * Google Object Detection API: https://github.com/tensorflow/models/tree/master/object_detection

* Digits recognition, using NN with <b>Keras</b> [higher level library]: https://www.analyticsvidhya.com/blog/2016/10/tutorial-optimizing-neural-networks-using-keras-with-image-recognition-case-study/
  * Keras has 3 backend you can use: https://keras.io/backend/
    * Tensorflow (from Google)
    * CNTK (from Microsoft)
    * Theano (from University of Montreal)
    * You can change between these backend
  * Install Keras: https://keras.io/#installation
  * <b>Keras Resources</b>: https://github.com/fchollet/keras-resources
  * <b>Keras Examples</b>: https://github.com/fchollet/keras/tree/master/examples
  * For installing TensorFlow, strongly recommend to use `virtualenv` (it's fast, simply and won't influence other installed python libraries): https://www.tensorflow.org/install/install_mac
  * After `virtualenv` installation and validaton, Commands to turn on and turn off virtual environment:
    * To activate the virtual environment, `$ source ~/Documents/Virtual_Env/bin/activate      # If using bash, sh, ksh, or zsh`, change "Documents/Virtual_Env" to your own virtual environment folder name
    * To activate the virtual environment, `$ source ~/Documents/Virtual_Env/bin/activate.csh  # If using csh or tcsh`, change "Documents/Virtual_Env" to your own virtual environment folder name
    * Then in your terminal, you will see `(Virtual_Env)$`
    * To deactivate your virtual envvironment, `(Virtual_Env)$ deactivate`<commit_msg>add virtual env commands<commit_after>
Time to try experiments in Neural Network, Deep Learning and any other AI methods :)
Neural Network is a universal approximator, which means you can use it to implment other machine learning algorithms


*****************************************************************

RESOURCES

* Database
  * ImageNet: http://www.image-net.org
    * Here, you can download images in different format
* Standford CNN for visual Recognition lectures: http://cs231n.stanford.edu/syllabus.html
  * I think, once you started to learn deep learning, you will really feel it's so deep to learn....
  * My notes: https://github.com/hanhanwu/Hanhan_Data_Science_Resources2/blob/master/Standford_CNN_Notes1.pdf

* Here is a bunch of libraries, tutorials you can try: https://www.analyticsvidhya.com/blog/2016/08/deep-learning-path/

* 10 Deep Learning Architectures: https://www.analyticsvidhya.com/blog/2017/08/10-advanced-deep-learning-architectures-data-scientists/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29

* Fundamentals of neural network: https://www.analyticsvidhya.com/blog/2016/03/introduction-deep-learning-fundamentals-neural-networks/
  * It's such a great feeling to review and to understand deeper
  * After learning NN a while, are you going to forget about the relationship between weights, activation function and EACH neuron like I did? This article is a great source to review. So! Each neuron has an activation function, the input for this single neuron can come from multiple sources, each input and bias could has weight.
  * I think, the activation function can be whatever you defined. The purpose for defining an activation is to guarantee it will give the output you want. For example, the input is x1, x2 and the output should be `x1 AND x2`, so your activation function should use mathematical method to guarantee the output will be `x1 AND x2` when the input is x1, x2.
  * The power of multiple layers of NN - you can break a complex problem into small problems, and each NN layer deals with one of the small problem, finally, you get what you finally want
  * The number of neurons in the output layer will depend on the type of problem. It can be 1 for regression or binary classification problem or multiple for multi-class classification problems.
  * <b>The ultimate objective is to update the weights of the model in order to minimize the loss function</b>. The weights are updated using a back-propogation algorithm.
  * `A XNOR B = (A' AND B') OR (A AND B) = NOT ((A OR B) AND (A' OR B'))`, `A' = NOT A`
  * Each optimization includes:
    * Select a network architecture, i.e. number of hidden layers,  number of neurons in each layer and activation function
    * Initialize weights randomly
    * Use forward propagation to determine the output node
    * Find the error of the model using the known labels
    * <b>Back-propogate the error</b> into the network and determine the error for each node
    * Update the weights to minimize the loss function
  * During back-propagation:
    * <b>The error at a node is based on weighted sum of errors on all the nodes of the next layer which take output of this node as input.</b>
  * Important params for optimizing neural network
    * Type of architecture
    * Number of Layers
    * Number of Neurons in a layer
    * Regularization parameters
    * Learning Rate
    * Type of optimization / backpropagation technique to use
    * Dropout rate
    * Weight sharing
  * Think about the activation function, the architecture, do you think Deep Learning can really generate infinite number of solutions?! So powerful isn't it? The only problem is, you still have to have ground truth first, and for Deep Learning, the ground truth has to be more accurate and informative
    
* Core concepts of neural network: https://www.analyticsvidhya.com/blog/2016/08/evolution-core-concepts-deep-learning-neural-networks/
  * The meaning of "sample", "epoch", "batch": https://keras.io/getting-started/faq/#what-does-sample-batch-epoch-mean
    * <b>Sample</b>: One element of a dataset. Such as, 1 image, 1 audio file
    * <b>Batch</b>: A set of N samples. The samples in a batch are processed independently, in parallel. If training, a batch results in only one update to the model. A batch generally approximates the distribution of the input data better than a single input. <b>The larger the batch, the better the approximation</b>, but also takes longer time.
    * <b>Epoch</b>: Epoch: an arbitrary cutoff, generally defined as "one pass over the entire dataset", used to separate training into <b>distinct phases</b>, which is useful for <b>logging and periodic evaluation</b>. When using Keras `evaluation_data` or `evaluation_split` with the `fit` method of Keras models, <b>evaluation will be run at the end of every epoch</b>.
    
* Activation functions and when to ues them: https://www.analyticsvidhya.com/blog/2017/10/fundamentals-deep-learning-activation-functions-when-to-use-them/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29
  * They help neural network choose the useful points and suppress irrelevant info. They can decide whether a neuron should be activated or not, whether the information the neuron is receiving is relevant or should be ignored
  * Without activation function, the weights, bias will simply do a linear transformation, and the neural network will work as a linear regression problem
  * Gradient helps back-propagation. <b>To check gradient, you just check derivate at each data point</b>
  * <b>Binary Step Function</b>
    * Threshold based classifier, decide whether or not to activate the neuron
    * Only serves for binary classification, cannot work for multi-class classification
    * The gradient is 0, and cannot help back-propagation. Because back-propagation expects to use gradients to update errors and improve the model, but this function will set gradients to 0 and cannot really make any improvement
  * <b>Linear Function</b>
    * When you have multiple classes, choose the one with max value
    * But for linear function, the derivate is constant, which means gradient will be the same each time, still cannot help back-propagation
  * <b>Sigmoid Function</b>
    * `f(x)=1/(1+e^-x)`
    * non-linear
    * y ranges between [0,1], x ranges between [-infinite, infinite]. But because of the y range, sigmoid function is not symmetric around the origin and the values received are all positive
    * When gradient is approaching 0 (sigmoid curve is flat), the neuron is not really learning
  * <b>Tanh</b>
    * Scaled version of sigmoid function
    * `tanh(x)=2sigmoid(2x)-1`
    * It works similar to the sigmoid function but is symmetric over the origin. it ranges from -1 to 1
    * Your choice of using sigmoid or tanh would basically depend on the requirement of gradient in the problem statement
    * But similar to the sigmoid function we still have the vanishing gradient problem. When the gradient approaches to 0, the neuron is not really learn
  * <b>ReLU</b>
    * `f(x)=max(0,x)`, f(x)= x if x>= 0, f(x)=0 if x<0
    * non-linear
    * The main advantage of using the ReLU function over other activation functions is that <b>it does not activate all the neurons at the same time</b>.
    * But the gradient is 0 for x<0, which made the neurons die for activations in that region. Because it's not learning but x is always negative, so it's never learn
    * ReLU function should <b>only be used in the hidden layers</b>
  * <b>Leaky ReLU</b>
    * f(x)=ax if x<0, f(x)=x if x>=0
    * Improved ReLU
    * By doing this, it removes 0 gradient and therefore no dead neurons
    * In case of a <b>parameterised ReLU function</b>, ‘a‘ is also a trainable parameter. The network also learns the value of ‘a‘ for faster and more optimum convergence. The parametrised ReLU function is used when the leaky ReLU function still fails to solve the problem of dead neurons and the relevant information is not successfully passed to the next layer.
  * <b>Softmax</b>
    * Idealy used we you want the output is showing probability, because all the output are in range between [0,1]
  * <b>Summarized Suggestions from the author</b>
    * Sigmoid functions and their combinations generally work better in the case of classifiers
    * Sigmoids and tanh functions are sometimes avoided due to the vanishing gradient problem
    * ReLU function is a general activation function and is used in most cases these days
    * If we encounter a case of dead neurons in our networks the leaky ReLU function is the best choice
    * Always keep in mind that ReLU function should only be used in the hidden layers
    * As <b>a rule of thumb</b>, you can begin with using ReLU function and then move over to other activation functions in case ReLU doesn’t provide with optimum results
    
* Deep Leaning for Computer Vision: https://www.analyticsvidhya.com/blog/2016/04/deep-learning-computer-vision-introduction-convolution-neural-networks/?lipi=urn%3Ali%3Apage%3Ad_flagship3_feed%3BxPn6EhynRquw3Evzrg79RA%3D%3D
  * Detailed analysis of activation functions
    * Sigmoid (not recommended for CNN)
      * Kill the gradient
      * Outputs are not zero-centered since it's between [0,1]
      * Taking the exp() is computationally expensive
    * Tanh (not recommended for CNN)
      * Still kill the gradient
      * But output can be zero-centered
    * ReLU (Rectified Linear Unit)
      * <b>most commonly used for CNN</b>
      * Gradient won’t saturate in the positive region
      * Computationally very efficient as simple thresholding is required
      * Empirically found to converge faster than sigmoid or tanh.
      * Drawbacks - Output is not zero-centered and always positive
      * Drawbacks - Gradient is killed for x<0. Few techniques like <b>leaky ReLU</b> and <b>parametric ReLU</b> are used to overcome this
      * Drawbacks - Gradient is not defined at x=0. But this can be easily catered using <b>sub-gradients</b> and posts less practical challenges as x=0 is generally a rare case
    * Data Preprocessing
      * Same image size
      * Mean centering: subtract mean value from each pixel
    * Weights Initialization
      * All zeros - a bad idea
      * Gaussian Random Variables: you need to play with the standard deviation of the gaussian distribution which works well for your network....
      * Xavier Initialization: It suggests that variance of the gaussian distribution of weights for each neuron should depend on the number of inputs to the layer. A recent research suggested that for ReLU neurons, the recommended update is, `np.random.randn(n_in, n_out)*sqrt(2/n_in)`
      * You might be surprised to know that 10-20% of the ReLUs might be dead at a particular time while training and even in the end.
    * CNN
      * The <b>first layer</b> will try to detect edges and form templates for <b>edge detection</b>
      * The subsequent layers will try to combine them into <b>simpler shapes</b> and eventually into <b>templates of different object positions, illumination, scales, etc</b>
      * The final layers will <b>match an input image with all the templates</b> and the <b>final prediction</b> is like a weighted sum of all of them.
    * CNN important layers
      * Convolution Layer
      * ReLU layer
      * Pooling Layer: Using padding in convolution layer, the image size remains same. So, pooling layers are used to reduce the size of image.
      * Fully Connected Layer: Each pixel is considered as a separate neuron just like a regular neural network. The last fully-connected layer will contain as many neurons as the number of classes to be predicted
    * In this article, I think the correct formula for calculating output size should be: `(W-F+2P)/S + 1`
    * To calculate zero-padding size: `(F-1)/2`
    * Filters might be called kernels sometimes
    * Alexnet
    ![AlexNet](https://www.analyticsvidhya.com/wp-content/uploads/2016/03/fig-8.png)
    * It has code example using GraphLab with pre-trained model. Pre-trained model may increase the accuracy but difficult to find and if you use CPU, may take very long time to run
    

* Digits recognition with <b>TensorFlow</b> [lower level library]: https://www.analyticsvidhya.com/blog/2016/10/an-introduction-to-implementing-neural-networks-using-tensorflow/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29
  * <b>Tensorflow Resources</b>: https://github.com/jtoy/awesome-tensorflow
  * Tensorflow Models: https://github.com/tensorflow/models
  * Google Object Detection API: https://github.com/tensorflow/models/tree/master/object_detection

* Digits recognition, using NN with <b>Keras</b> [higher level library]: https://www.analyticsvidhya.com/blog/2016/10/tutorial-optimizing-neural-networks-using-keras-with-image-recognition-case-study/
  * Keras has 3 backend you can use: https://keras.io/backend/
    * Tensorflow (from Google)
    * CNTK (from Microsoft)
    * Theano (from University of Montreal)
    * You can change between these backend
  * Install Keras: https://keras.io/#installation
  * <b>Keras Resources</b>: https://github.com/fchollet/keras-resources
  * <b>Keras Examples</b>: https://github.com/fchollet/keras/tree/master/examples
  * For installing TensorFlow, strongly recommend to use `virtualenv` (it's fast, simply and won't influence other installed python libraries): https://www.tensorflow.org/install/install_mac
    * `sudo pip install --upgrade virtualenv`
    * Then you create a new folder as virtual environment, such as folder myvirtual_env, then type `virtualenv --system-site-packages myvirtual_env`, of course, it should be thr path of your folder
    * Now you can activate it, in this case, type command `source ~/myvirtual_env/bin/activate `
  * After `virtualenv` installation and validaton, Commands to turn on and turn off virtual environment:
    * To activate the virtual environment, `$ source ~/Documents/Virtual_Env/bin/activate      # If using bash, sh, ksh, or zsh`, change "Documents/Virtual_Env" to your own virtual environment folder name
    * To activate the virtual environment, `$ source ~/Documents/Virtual_Env/bin/activate.csh  # If using csh or tcsh`, change "Documents/Virtual_Env" to your own virtual environment folder name
    * Then in your terminal, you will see `(Virtual_Env)$`
    * To deactivate your virtual envvironment, `(Virtual_Env)$ deactivate`
{'fraction_non_alphanumeric': 0.07666921420134788, 'fraction_numerical': 0.00972695060098659, 'mean_word_length': 3.9890814558058927, 'pattern_counts': {'":': 2, '<': 163, '<?xml version=': 0, '>': 159, 'https://': 38, 'lorem ipsum': 0, 'www.': 22, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '83577', 'n_tokens_mistral': 8054, 'n_tokens_neox': 7619, 'n_words': 3663}