<commit_before>= Flows
:toc:

About manipulation of flows (including byte streams, character flows, files) in Java.

NOTE: The reader should understand https://github.com/oliviercailloux/java-course/blob/master/Shell.adoc#Paths[Relative and absolute paths]. The reader should know the https://docs.oracle.com/javase/tutorial/java/javaOO/arguments.html[Varargs] construct. (The last exercice uses https://github.com/oliviercailloux/java-course/blob/master/Maven/README.adoc[Maven].)

== Overview
There are four essential kinds of objects that represent flows of information in Java.

* https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html[InputStream]: to read flows of bytes.
* https://docs.oracle.com/javase/8/docs/api/java/io/OutputStream.html[OutputStream]: to write flows of bytes.
* https://docs.oracle.com/javase/8/docs/api/java/io/Reader.html[Reader]: to read flows of characters.
* https://docs.oracle.com/javase/8/docs/api/java/io/Writer.html[Writer]: to write flows of characters.

It is typical for a flow of character to be encoded as a flow of bytes. For example, to store the string `Hello, world` in a text file on your hard disk, using the https://en.wikipedia.org/wiki/UTF-8[UTF-8] encoding standard, the string would be first https://en.wikipedia.org/wiki/UTF-8#Codepage_layout[encoded] as the bytes _48_, _65_, _6C_, _6C_, _6E_, _2C_, _20_, _77_, _6F_, _72_, _6C_, _64_ in hexadecimal (or equivalently in decimal, _72_, _101_, _108_, _108_, _110_, _44_, _32_, _119_, _111_, _114_, _108_, _100_), then written (as a bit stream) on the disk. In Java, a mapping between characters and bytes is called a _character set_. You should always opt for the https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/charset/StandardCharsets.html#UTF_8[`UTF-8`] character set when you have a choice, as it is the standard general purpose character set.

The most common way of obtaining a flow is to read from a file. To designate the file, you typically use a path relative to your project directory (the place where `pom.xml` is stored, when you use Maven). 

Two essential classes help you deal with files: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Path.html[`Path`] and https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Files.html[`Files`]. For example, use `Files#newBufferedWriter()` to obtain a `Writer` that writes to a file.

Classes that represent flows (such as the four above) implement `Closeable`. This signals that they must be _closed_ after use. (This serves to release resources that might have been reserved by the operating system, or to proceed with finalizing operations, such as effectively writing to disk by flushing the cache.) Do not forget to close your flows after use by calling `close()`. Use the https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html[try-with-resources] statement for this, as it is a simple way of making sure `close()` is duly called.

NOTE: Many examples on the internet still use the `File` class. This approach should now be considered https://docs.oracle.com/javase/tutorial/essential/io/legacy.html[deprecated]. Using `Path` and `Files`, as illustrated here, should be favored: among other advantages, it readily adapts to other kinds of file systems such as https://github.com/google/jimfs[in-memory file systems], as explained below.

== Exercice: read using a byte flow
* Create a file `in.txt` in your project directory, containing `Hello, world`. 
* In Java, create a `Path` object designating that file.
* Use the appropriate static method from the `Files` class to obtain an `InputStream` representing the content of that file.
* Read and print the bytes from that stream using a loop, one by one.
* Check that the bytes are as expected.

Here is a https://github.com/oliviercailloux/sample-flows/tree/master/src/main/java/io/github/oliviercailloux/sample_flows[solution]. Don’t cheat! Try it for yourself before looking.

* Move the file around in your hard disk (for example, put in into the `src/main` sub-directory, or put in into a directory higher up the hierarchy, one that is not included in your project). Adapt the code.

In this example, the specific encoding used for encoding characters to bytes does not matter much, because all encodings used in practice encode the basic latin alphabet in the same way. Problems can arise when using “more exotic” characters. For example, the character `é` is encoded as the bytes https://www.fileformat.info/info/unicode/char/00e9/index.htm[_C3_, _A9_] under `UTF-8`, but as https://en.wikipedia.org/wiki/%C3%89#Character_mappings[_E9_] under `ISO-8859-1`.

== Exercice<commit_msg>conversion error "o"<commit_after>= Flows
:toc:

About manipulation of flows (including byte streams, character flows, files) in Java.

NOTE: The reader should understand https://github.com/oliviercailloux/java-course/blob/master/Shell.adoc#Paths[Relative and absolute paths]. The reader should know the https://docs.oracle.com/javase/tutorial/java/javaOO/arguments.html[Varargs] construct. (The last exercice uses https://github.com/oliviercailloux/java-course/blob/master/Maven/README.adoc[Maven].)

== Overview
There are four essential kinds of objects that represent flows of information in Java.

* https://docs.oracle.com/javase/8/docs/api/java/io/InputStream.html[InputStream]: to read flows of bytes.
* https://docs.oracle.com/javase/8/docs/api/java/io/OutputStream.html[OutputStream]: to write flows of bytes.
* https://docs.oracle.com/javase/8/docs/api/java/io/Reader.html[Reader]: to read flows of characters.
* https://docs.oracle.com/javase/8/docs/api/java/io/Writer.html[Writer]: to write flows of characters.

It is typical for a flow of character to be encoded as a flow of bytes. For example, to store the string `Hello, world` in a text file on your hard disk, using the https://en.wikipedia.org/wiki/UTF-8[UTF-8] encoding standard, the string would be first https://en.wikipedia.org/wiki/UTF-8#Codepage_layout[encoded] as the bytes _48_, _65_, _6C_, _6C_, _6F_, _2C_, _20_, _77_, _6F_, _72_, _6C_, _64_ in hexadecimal (or equivalently in decimal, _72_, _101_, _108_, _108_, _111_, _44_, _32_, _119_, _111_, _114_, _108_, _100_), then written (as a bit stream) on the disk. In Java, a mapping between characters and bytes is called a _character set_. You should always opt for the https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/charset/StandardCharsets.html#UTF_8[`UTF-8`] character set when you have a choice, as it is the standard general purpose character set.

The most common way of obtaining a flow is to read from a file. To designate the file, you typically use a path relative to your project directory (the place where `pom.xml` is stored, when you use Maven). 

Two essential classes help you deal with files: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Path.html[`Path`] and https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/nio/file/Files.html[`Files`]. For example, use `Files#newBufferedWriter()` to obtain a `Writer` that writes to a file.

Classes that represent flows (such as the four above) implement `Closeable`. This signals that they must be _closed_ after use. (This serves to release resources that might have been reserved by the operating system, or to proceed with finalizing operations, such as effectively writing to disk by flushing the cache.) Do not forget to close your flows after use by calling `close()`. Use the https://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html[try-with-resources] statement for this, as it is a simple way of making sure `close()` is duly called.

NOTE: Many examples on the internet still use the `File` class. This approach should now be considered https://docs.oracle.com/javase/tutorial/essential/io/legacy.html[deprecated]. Using `Path` and `Files`, as illustrated here, should be favored: among other advantages, it readily adapts to other kinds of file systems such as https://github.com/google/jimfs[in-memory file systems], as explained below.

== Exercice: read using a byte flow
* Create a file `in.txt` in your project directory, containing `Hello, world`. 
* In Java, create a `Path` object designating that file.
* Use the appropriate static method from the `Files` class to obtain an `InputStream` representing the content of that file.
* Read and print the bytes from that stream using a loop, one by one.
* Check that the bytes are as expected.

Here is a https://github.com/oliviercailloux/sample-flows/tree/master/src/main/java/io/github/oliviercailloux/sample_flows[solution]. Don’t cheat! Try it for yourself before looking.

* Move the file around in your hard disk (for example, put in into the `src/main` sub-directory, or put in into a directory higher up the hierarchy, one that is not included in your project). Adapt the code.

In this example, the specific encoding used for encoding characters to bytes does not matter much, because all encodings used in practice encode the basic latin alphabet in the same way. Problems can arise when using “more exotic” characters. For example, the character `é` is encoded as the bytes https://www.fileformat.info/info/unicode/char/00e9/index.htm[_C3_, _A9_] under `UTF-8`, but as https://en.wikipedia.org/wiki/%C3%89#Character_mappings[_E9_] under `ISO-8859-1`.

== Exercice
{'fraction_non_alphanumeric': 0.09880207781193681, 'fraction_numerical': 0.017173751722675712, 'mean_word_length': 4.61882072662299, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 36, 'lorem ipsum': 0, 'www.': 2, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7672878', 'n_tokens_mistral': 2848, 'n_tokens_neox': 2682, 'n_words': 1123}