<commit_before># Utf8ConvAtlStl
### C++ helper functions to convert between Unicode UTF-8 (`std::string`) and UTF-16 (`CStringW`)

<NAME>

This C++ project contains a _header-only_ module for UTF-8 (`std::string`) <-> UTF-16 (`CStringW`) conversions.

It makes sense to use `CStringW` at the Windows ATL/MFC level for UTF-16 text,
and `std::string` e.g. in cross-platform C++ code storing UTF-8 text.

This reusable header-only module ([`Utf8Conv.h`](https://github.com/GiovanniDicanio/Utf8ConvAtlStl/blob/master/Utf8ConvAtlStl/Utf8ConvAtlStl/Utf8Conv.h)) 
provides a couple of convenient conversion functions between
these two types of strings.
 
Conversion errors are signaled using `AtlThrow()`, which [by default](https://msdn.microsoft.com/en-us/library/z325eyx0.aspx) translates
to throwing `CAtlException` instances.
 
For more details, please read [my MSDN Magazine article](https://msdn.microsoft.com/magazine/mt763237):
 
>  C++ - Unicode Encoding Conversions with STL Strings and Win32 APIs

The code presented here is based on that article, but uses `CStringW` instead
of `std::wstring` for UTF-16 strings at the Windows platform level.

This project contains also a unit-test for the reusable Unicode conversion module.

Code developed using Visual Studio 2015.  
Compiles cleanly at `/W4` in both 32-bit builds and 64-bit builds.

**Note for Older VC++ Compilers**  
If you are using Visual Studio 2010, which doesn't support the C++11 `constexpr` keyword, you can still include this C++ code in your projects, simply substituting every instance of `constexpr` with `const`; this code will work just fine.
<commit_msg>Added unit test link and few changes<commit_after># Utf8ConvAtlStl
### C++ helper functions to convert between Unicode UTF-8 (`std::string`) and UTF-16 (`CStringW`)

<NAME>

This C++ project contains a _header-only_ module for UTF-8 (`std::string`) <-> UTF-16 (`CStringW`) conversions.

It makes sense to use `CStringW` at the Windows ATL/MFC level for UTF-16 text,
and `std::string` e.g. in cross-platform C++ code storing UTF-8 text.

This **reusable header-only module** ([`Utf8Conv.h`](https://github.com/GiovanniDicanio/Utf8ConvAtlStl/blob/master/Utf8ConvAtlStl/Utf8ConvAtlStl/Utf8Conv.h)) 
provides a couple of convenient conversion functions between
these two types of strings.
 
Conversion errors are signaled using `AtlThrow()`, which [by default](https://msdn.microsoft.com/en-us/library/z325eyx0.aspx) translates
to throwing `CAtlException` instances.
 
For more details, please read [my MSDN Magazine article](https://msdn.microsoft.com/magazine/mt763237):
 
>  C++ - Unicode Encoding Conversions with STL Strings and Win32 APIs

The code presented here is based on that article, but uses `CStringW` instead
of `std::wstring` for UTF-16 strings at the Windows platform level.

This project contains also a **unit-test** ([`Utf8ConvTest.cpp`](https://github.com/GiovanniDicanio/Utf8ConvAtlStl/blob/master/Utf8ConvAtlStl/Utf8ConvAtlStl/Utf8ConvTest.cpp)) for the reusable Unicode conversion module.

Code developed using **Visual Studio 2015**.  
Compiles cleanly at `/W4` in both 32-bit builds and 64-bit builds.

**Note for Older VC++ Compilers**  
If you are using Visual Studio 2010, which doesn't support the C++11 `constexpr` keyword, you can still include this C++ code in your projects, simply substituting every instance of `constexpr` with `const`; this code will work just fine.

{'fraction_non_alphanumeric': 0.10285547785547786, 'fraction_numerical': 0.027097902097902096, 'mean_word_length': 4.980836236933798, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 9, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '29970', 'n_tokens_mistral': 1170, 'n_tokens_neox': 1036, 'n_words': 403}