<commit_before># classify-handwritten-characters
Classify handwritten Chinese characters

## Prerequisites
* HWDB1.1 (1.1 million image samples. [info][3], [train.zip (1873 MB)][1], [test.zip (471 MB)][2])
* OLHWDB1.1 (1.1 million stroke samples. [info][6], [train.zip (187 MB)][4], [test.zip (47 MB)][5])
* Tensorflow 1.1
* Python 3
* Python modules in requirements.txt

## Creating the model
1. Unzip the training and test sets
2. $ ./convert_gnt_to_records.py unzipped_hwdb1.1_folder
3. $ ./gnt_model
4. $ ./gnt_predict data/model-stepN.ckpt png_image_sample
5. $ ./write_pb_file data/model-stepN.ckpt output_dir

## Updating the android tensorflow library
This repository comes prepackaged with the android tensorflow library. This might require updating in the future
for a new version of tensorflow. Binaries are available on https://ci.tensorflow.org/view/Nightly/job/nightly-android/.
To update:
 * Replace android/app/libs/libandroid_tensorflow_inference_java.jar with the [nightly version][nightly_jar]
 * Replace android/app/src/main/jniLibs/armeabi-v7a/libtensorflow_inference.so with the [nightly version][nightly_so].

## TODO
 [x] Update the android app to allow drawing at a higher resolution that gets downsampled before inference
 [x] Update the android app to crop the drawing before sending to the model
 [ ] Make tapping on a character bring user to dictionary entry for that character
 [x] Add undo for strokes<commit_msg>update todo<commit_after># classify-handwritten-characters
Classify handwritten Chinese characters

## Prerequisites
* HWDB1.1 (1.1 million image samples. [info][3], [train.zip (1873 MB)][1], [test.zip (471 MB)][2])
* OLHWDB1.1 (1.1 million stroke samples. [info][6], [train.zip (187 MB)][4], [test.zip (47 MB)][5])
* Tensorflow 1.1
* Python 3
* Python modules in requirements.txt

## Creating the model
1. Unzip the training and test sets
2. $ ./convert_gnt_to_records.py unzipped_hwdb1.1_folder
3. $ ./gnt_model
4. $ ./gnt_predict data/model-stepN.ckpt png_image_sample
5. $ ./write_pb_file data/model-stepN.ckpt output_dir

## Updating the android tensorflow library
This repository comes prepackaged with the android tensorflow library. This might require updating in the future
for a new version of tensorflow. Binaries are available on https://ci.tensorflow.org/view/Nightly/job/nightly-android/.
To update:
 * Replace android/app/libs/libandroid_tensorflow_inference_java.jar with the [nightly version][nightly_jar]
 * Replace android/app/src/main/jniLibs/armeabi-v7a/libtensorflow_inference.so with the [nightly version][nightly_so].

## TODO
 [x] Update the android app to allow drawing at a higher resolution that gets downsampled before inference
 [x] Update the android app to crop the drawing before sending to the model
 [x] Make tapping on a character bring user to dictionary entry for that character
 [x] Add undo for strokes
{'fraction_non_alphanumeric': 0.0984060984060984, 'fraction_numerical': 0.02564102564102564, 'mean_word_length': 4.879837067209776, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1101592', 'n_tokens_mistral': 937, 'n_tokens_neox': 890, 'n_words': 317}