<commit_before>Java Direct I/O (Jaydio)
========================

Jaydio is a Java library for giving the programmer finer control over
file I/O, in part by bypassing the OS buffer cache. **Currently only Linux is
supported**, but Jaydio has big dreams of expanding to other platforms
and adding new functionality other than the vanilla direct I/O currently
offered. Jaydio serves as as a starting point for asynchronous / non-blocking direct I/O,
custom file page caching layers, and even advanced cache replacement policies
like [ARC](http://dbs.uni-leipzig.de/file/ARC.pdf)
or [2Q](http://www.inf.fu-berlin.de/lehre/WS06/DBS-Tech/Reader/2QBufferManagement.pdf).
Currently, it is useful for preventing the operating system from evicting
memory pages which need to stay "warm" in favor of non-critical file pages.



Adding Jaydio to Your Project
-----------------------------

[jaydio-0.1.jar](https://oss.sonatype.org/service/local/repositories/releases/content/net/smacke/jaydio/0.1/jaydio-0.1.jar)

Jaydio has dependencies on [JNA](https://github.com/twall/jna) and log4j.
If you add Jaydio to your project using Maven (groupId `net.smacke`,
artifactId `jaydio`, version `0.1`), these will of course be handled
automatically. Alternatively, if you don't want to deal with Maven or
with tracking down dependencies manually, the following are also available:

[jaydio-0.1-jar-with-dependencies.jar](http://smacke.net/jaydio/jaydio-0.1-jar-with-dependencies.jar)

[jaydio-0.1-jar-with-dependencies.jar.asc](http://smacke.net/jaydio/jaydio-0.1-jar-with-dependencies.jar.asc)

The API reference for Jaydio is located in the [javadoc](http://smacke.net/jaydio/javadoc/index.html).

How it works
------------

Jaydio works by passing the `O_DIRECT` flag to the Linux `open()` system call.
Unfortunately, talking directly to disk imposes some nasty rules about how
many bytes can be written at a time, and where they can be written. In particular,
all writes must be in multiples of the file system block size, and they must
originate from chunks of memory aligned to the memory page size, and they must
also occur at file system block boundaries.<commit_msg>add code for maven dep in readme<commit_after>Java Direct I/O (Jaydio)
========================

Jaydio is a Java library for giving the programmer finer control over
file I/O, in part by bypassing the OS buffer cache. **Currently only Linux is
supported**, but Jaydio has big dreams of expanding to other platforms
and adding new functionality other than the vanilla direct I/O currently
offered. Jaydio serves as as a starting point for asynchronous / non-blocking direct I/O,
custom file page caching layers, and even advanced cache replacement policies
like [ARC](http://dbs.uni-leipzig.de/file/ARC.pdf)
or [2Q](http://www.inf.fu-berlin.de/lehre/WS06/DBS-Tech/Reader/2QBufferManagement.pdf).
Currently, it is useful for preventing the operating system from evicting
memory pages which need to stay "warm" in favor of non-critical file pages.



Adding Jaydio to Your Project
-----------------------------

[jaydio-0.1.jar](https://oss.sonatype.org/service/local/repositories/releases/content/net/smacke/jaydio/0.1/jaydio-0.1.jar)

Jaydio has dependencies on [JNA](https://github.com/twall/jna) and log4j.

If you use Maven, you can add the following to your `pom.xml`:

```xml
<dependencies>
  ... <!-- dots indicate other dependencies you may have -->
  <dependency>
    <groupId>net.smacke</groupId>
    <artifactId>jaydio</artifactId>
    <version>0.1</version>
  </dependency>
  ...
</dependencies>
```

Alternatively, if you don't want to deal with Maven or
with tracking down dependencies manually, the following are also available:

[jaydio-0.1-jar-with-dependencies.jar](http://smacke.net/jaydio/jaydio-0.1-jar-with-dependencies.jar)

[jaydio-0.1-jar-with-dependencies.jar.asc](http://smacke.net/jaydio/jaydio-0.1-jar-with-dependencies.jar.asc)

The API reference for Jaydio is located in the [javadoc](http://smacke.net/jaydio/javadoc/index.html).

How it works
------------

Jaydio works by passing the `O_DIRECT` flag to the Linux `open()` system call.
Unfortunately, talking directly to disk imposes some nasty rules about how
many bytes can be written at a time, and where they can be written. In particular,
all writes must be in multiples of the file system block size, and they must
originate from chunks of memory aligned to the memory page size, and they must
also occur at file system block boundaries.
{'fraction_non_alphanumeric': 0.12028565052443652, 'fraction_numerical': 0.009372907833072975, 'mean_word_length': 4.075877689694225, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3446072', 'n_tokens_mistral': 1437, 'n_tokens_neox': 1320, 'n_words': 510}