<commit_before>**How to build:**

***Prepare the source tree***

- git clone Xpiks repo
- git submodule init & git submodule update
- rename tinyAES/aes.c to tinyAES/aes.cpp
- ensure you have `git` available in your PATH

***Mac OS X / Windows : build third-parties***

- download **Hunspell 1.3.3** from http://hunspell.sourceforge.net/ and extract it in the `src/` directory
- open project `src/hunspell/hunspell.pro` in Qt Creator, execute `Run qmake`, execute `Build`
- copy built library (e.g. `libhunspell.a`) from the build directory to the `src/libs` directory
- follow instructions in `src/cpp-libface/README.md` to build libface
- copy built library to `src/libs` directory
- [windows] you will also need to download **zlib 1.2.8** and extract it in the `src/` directory
- [windows] open project `src/zlib-project/zlib.pro` in Qt Creator, execute `Run qmake`, execute `Build`
- [windows] copy built library (e.g. `z.lib`) from the build directory to the `src/libs` directory
- open project `src/quazip/quazip/quazip.pro` in Qt Creator, execute `Run qmake`, execute `Build`
- copy built library (e.g. `libquazip.a`) from the build directory to the `src/libs` directory
- [windows] do `git clone https://github.com/blackrosezy/build-libcurl-windows.git` and execute `build.bat`
- [windows] copy contests of `lib/dll-debug-x64` contents to the `src/libs/` directory
- [windows] build exiv2 library (see steps below)

***Visual Studio 2015 build of exiv2 library***

- download **exiv2 v0.25** library from http://www.exiv2.org/ and extract it somewhere
- open `msvc2005\exiv2.sln` and upgrade all projects
- from each project remove file `localtime.c`
- open file `include\exiv2\exv_msvs.h` and enable unicode support, disable png, curl and ssh (alternatively, you can overwrite this file with the one in the _Xpiks_ repository via path `src/exiv2-0.25/include/exiv2/exv_msvs.h`)
- in Visual Studio select `"Build" -> "Batch build"` and mark all _DebugDLL_ and _ReleaseDLL_ builds for your platform (x64 or x86)
- press "Build"
- copy build artifacts (`libexiv2.dll`, `libexiv2.lib`, `libexpat.dll`, `libexpat.lib`) to `src/libs` directory in _Xpiks_ repository

***Manual build***
- install Qt 5.4 or later from official website (or 5.2 or later from repository in case of Linux)
- [linux] install development packages for `libcurl`, `hunspell`, `quazip`, `zlib`, `libexiv`
- [linux] install `exiftool` (this **is not** necessary since Xpiks 1.3.1) and `expat` (this **is** necessary since Xpiks 1.3.1)
- open main project in Qt Creator, execute `Run qmake`, execute `Build`
- you can open, build and run tests project (`src/xpiks-tests/xpiks-tests.pro`) additionally

If you want to create Xpiks executable for everyday usage, before each `Run qmake` step described below, select `Release` in Qt Creator. Select `Debug` (the default) in all other cases.

**Linux package build**

This chapter assumes that you have installed generic development packages for your OS.

***Additional requirements***

In some cases you neet to install `libcurl-devel`, `hunspell-devel`, `quazip-devel`, `libexiv-devel` packages (they can have slightly different names in your distribution) if it is not installed as a build requirement.

***Debian package build***

- Install build dependecies from repository (QT)
- cd to _xpiks-qt_ subdirectory
- `debuild -i -us -uc -b`

***OpenSUSE RPM build***

- Install rpmbuild.
- `mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,BUILD,BUILDROOT,SOURCES}`
- `tar -czf xpiks-qt.tar.gz xpiks-qt/ tiny-aes/`
- Drop this source tarball in `~/rpmbuild/SOURCES`
- Drop the the spec file found in `xpiks-qt/RPM` folder to `~/rpmbuild/SPECS`
- `rpmbuild -ba ~/rpmbuild/SPECS/xpiks.spec`

***More***

More instructions can be found at this page http://ribtoks.github.io/xpiks/blog/2014/building-xpiks-alone

Xpiks uses English dictionaries from http://cgit.freedesktop.org/libreoffice/dictionaries/tree/. In Windows they are stored in the `dict/` subdirectory, in OS X they are in the `Resources/` directory of a bundle.
<commit_msg>Added instructions about expat [ci skip]<commit_after>**How to build:**

***Prepare the source tree***

- git clone Xpiks repo
- git submodule init & git submodule update
- rename tinyAES/aes.c to tinyAES/aes.cpp
- ensure you have `git` available in your PATH

***Mac OS X / Windows : build third-parties***

- download **Hunspell 1.3.3** from http://hunspell.sourceforge.net/ and extract it in the `src/` directory
- open project `src/hunspell/hunspell.pro` in Qt Creator, execute `Run qmake`, execute `Build`
- copy built library (e.g. `libhunspell.a`) from the build directory to the `src/libs` directory
- follow instructions in `src/cpp-libface/README.md` to build libface
- copy built library to `src/libs` directory
- [windows] you will also need to download **zlib 1.2.8** and extract it in the `src/` directory
- [windows] open project `src/zlib-project/zlib.pro` in Qt Creator, execute `Run qmake`, execute `Build`
- [windows] copy built library (e.g. `z.lib`) from the build directory to the `src/libs` directory
- open project `src/quazip/quazip/quazip.pro` in Qt Creator, execute `Run qmake`, execute `Build`
- copy built library (e.g. `libquazip.a`) from the build directory to the `src/libs` directory
- [windows] do `git clone https://github.com/blackrosezy/build-libcurl-windows.git` and execute `build.bat`
- [windows] copy contests of `lib/dll-debug-x64` contents to the `src/libs/` directory
- [windows] build exiv2 library (see steps below)

***Visual Studio 2015 build of exiv2 library***

- download **exiv2 v0.25** library from http://www.exiv2.org/ and extract it somewhere
- download **expat-2.1.1** library from https://sourceforge.net/projects/expat/ and extract it on same level as exiv2
- rename directory `expat-2.1.1/` to `expat\`
- open `msvc2005\exiv2.sln` and upgrade all projects
- from each project remove file `localtime.c`
- open file `include\exiv2\exv_msvs.h` and enable unicode support, disable png, curl and ssh (alternatively, you can overwrite this file with the one in the _Xpiks_ repository via path `src/exiv2-0.25/include/exiv2/exv_msvs.h`)
- in Visual Studio select `"Build" -> "Batch build"` and mark all _DebugDLL_ and _ReleaseDLL_ builds for your platform (x64 or x86)
- press "Build"
- copy build artifacts (`libexiv2.dll`, `libexiv2.lib`, `libexpat.dll`, `libexpat.lib`) to `src/libs` directory in _Xpiks_ repository

***Manual build***
- install Qt 5.4 or later from official website (or 5.2 or later from repository in case of Linux)
- [linux] install development packages for `libcurl`, `hunspell`, `quazip`, `zlib`, `libexiv`
- [linux] install `exiftool` (this **is not** necessary since Xpiks 1.3.1) and `expat` (this **is** necessary since Xpiks 1.3.1)
- open main project in Qt Creator, execute `Run qmake`, execute `Build`
- you can open, build and run tests project (`src/xpiks-tests/xpiks-tests.pro`) additionally

If you want to create Xpiks executable for everyday usage, before each `Run qmake` step described below, select `Release` in Qt Creator. Select `Debug` (the default) in all other cases.

**Linux package build**

This chapter assumes that you have installed generic development packages for your OS.

***Additional requirements***

In some cases you neet to install `libcurl-devel`, `hunspell-devel`, `quazip-devel`, `libexiv-devel` packages (they can have slightly different names in your distribution) if it is not installed as a build requirement.

***Debian package build***

- Install build dependecies from repository (QT)
- cd to _xpiks-qt_ subdirectory
- `debuild -i -us -uc -b`

***OpenSUSE RPM build***

- Install rpmbuild.
- `mkdir -p ~/rpmbuild/{RPMS,SRPMS,SPECS,BUILD,BUILDROOT,SOURCES}`
- `tar -czf xpiks-qt.tar.gz xpiks-qt/ tiny-aes/`
- Drop this source tarball in `~/rpmbuild/SOURCES`
- Drop the the spec file found in `xpiks-qt/RPM` folder to `~/rpmbuild/SPECS`
- `rpmbuild -ba ~/rpmbuild/SPECS/xpiks.spec`

***More***

More instructions can be found at this page http://ribtoks.github.io/xpiks/blog/2014/building-xpiks-alone

Xpiks uses English dictionaries from http://cgit.freedesktop.org/libreoffice/dictionaries/tree/. In Windows they are stored in the `dict/` subdirectory, in OS X they are in the `Resources/` directory of a bundle.

{'fraction_non_alphanumeric': 0.12039608742905446, 'fraction_numerical': 0.012921144789276657, 'mean_word_length': 4.3432258064516125, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 3, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2505735', 'n_tokens_mistral': 2814, 'n_tokens_neox': 2663, 'n_words': 1027}