<commit_before># Tools 

This foler contains useful tools for performing face detection, facial recognition, and setting up directories to be tested on. Feel free to contribute any tools which you find useful for running the FCT Facial Recognition software.

## FaceCrop

FaceCrop is a tool that takes an input image (.jpg, .png, .bmp, etc) and detects all faces within the image. Once detected, the tool crops the image to the minimum dimensions that contain the face. FaceCrop will allow for much more freedom when it comes to creating test sets. It is now not necessary to require a picture to be taken at a precise distance and with one subject. 

## OpenCV
To run FaceCrop, you must first download openCV on your machine. To download openCV, go to [this site](http://opencv.org/downloads.html) and select the appropriate file to download. Then, run the following commands in the directory openCV that gets created when you unzip the download:

    sudo apt-get install cmake
    sudo apt-get install libgtk2.0-dev
    sudo apt-get install python-numpy 

Make sure you are in the directory created by unzipping the downlaoded openCV package and then:

    cmake -D CMAKE_BUILD_TYPE=RELEASE -D BUILD_NEW_PYTHON_SUPPORT=ON -D CMAKE_INSTALL_PREFIX=/usr/local ./

You should see "Configuring done." and "Generating done." at the end of the make. Then:

    make

And finally:

    sudo make install

If you have any issues with the install, check out [this link](https://www.youtube.com/watch?v=MqQB5KKJCh0) for additional help.

## How To Run

Once you have openCV installed, running FaceCrop should not be a problem. Run "make" and then ./facedetect to crop photos. Currently the path to the photo is hard coded within facedetect.cpp, but future versions will create command line arguments and possible scripts to run this program on an entire directory. <commit_msg>update tools README<commit_after># Tools

This foler contains useful tools for performing face detection, facial recognition, and setting up directories to be tested on. Feel free to contribute any tools which you find useful for running the FCT Facial Recognition software.

## FaceCrop

FaceCrop is a tool that takes an input image (.jpg, .png, .bmp, etc) and detects all faces within the image. Once detected, the tool crops the image to the minimum dimensions that contain the face. FaceCrop will allow for much more freedom when it comes to creating test sets. It is now not necessary to require a picture to be taken at a precise distance and with one subject.

FaceCrop usage:
    ./crop-faces.sh [path-to-source-images]

FaceCrop will create a directory in the pwd called cropped_test_set that contains the subdirectories of all the cropped
faces from the source directory.

## OpenCV
To run FaceCrop, you must first download openCV on your machine. To download openCV, go to [this site](http://opencv.org/downloads.html) and select the appropriate file to download. Then, run the following commands in the directory openCV that gets created when you unzip the download:

    sudo apt-get install cmake
    sudo apt-get install libgtk2.0-dev
    sudo apt-get install python-numpy

Make sure you are in the directory created by unzipping the downlaoded openCV package and then:

    cmake -D CMAKE_BUILD_TYPE=RELEASE -D BUILD_NEW_PYTHON_SUPPORT=ON -D CMAKE_INSTALL_PREFIX=/usr/local ./

You should see "Configuring done." and "Generating done." at the end of the make. Then:

    make

And finally:

    sudo make install

If you have any issues with the install, check out [this link](https://www.youtube.com/watch?v=MqQB5KKJCh0) for additional help.

## How To Run

Once you have openCV installed, running FaceCrop should not be a problem. Run "make" and then ./facedetect to crop photos. Currently the path to the photo is hard coded within facedetect.cpp, but future versions will create command line arguments and possible scripts to run this program on an entire directory.
{'fraction_non_alphanumeric': 0.04981036662452592, 'fraction_numerical': 0.00202275600505689, 'mean_word_length': 4.151041666666667, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3898474', 'n_tokens_mistral': 1068, 'n_tokens_neox': 1022, 'n_words': 579}