<commit_before>This tool will let you to test how much active/inactive memory is used for Page Cache. It supports **ONLY LINUX**, because it uses `/proc` filesystem.  
I was inspired by for experiment by <NAME> and his presentation https://habrahabr.ru/company/yandex/blog/231957/https://habrahabr.ru/company/yandex/blog/231957/  
For demonstration, I recommend you to have VM with no load.  
To make a proper demonstration I recommend you to do following steps:  
# Preparations:
1) Clone and compile `vmtouch` from here: https://github.com/hoytech/vmtouch
2) Generate files (1GB, 4GB). The rule is simple: first file should be less, than total amount of memory, second - more.
```
terminal1$ dd if=/dev/zero of=./large count=10240 bs=102400
terminal1$ dd if=/dev/zero of=./huge count=40960 bs=102400
```
3) Flush all caches:
```
terminal1$ echo 3 > /proc/sys/vm/drop_caches
```
4) Run script demonstating mapping these file in Page Cache (in separate terminal):
```
terminal2$ watch "echo HUGE; vmtouch ./huge ; echo LARGE ; vmtouch ./large"
```
5) Run in separate terminal
```
terminal3$ mem.sh 
```
# Demonstration:
After you flushed all caches, you shold see active and INACTIVE = 0
1) Read large file. You will to see INACTIVE growing. Also `vmtouch`  
```
terminal1$ cat ./large > /dev/null
```
2) Read huge file. You will see on terminal with `vmstat`, that file large was moved away from inactive memory
```
terminal1$ cat ./huge > /dev/null
```
3) Read large file again. You will see on terminal with `vmstat`, how part of huge file is moving away from inactive memory
4) Read large file again and you will see, that it moved to an Active memory
5) Read huge file again. You will see, that large file was moved away partially, or was not moved at all<commit_msg>Fixed formatting<commit_after>This tool will let you to test how much active/inactive memory is used for Page Cache. It supports **ONLY LINUX**, because it uses `/proc` filesystem.  
I was inspired by for experiment by <NAME> and his presentation https://habrahabr.ru/company/yandex/blog/231957/https://habrahabr.ru/company/yandex/blog/231957/  
For demonstration, I recommend you to have VM with no load.  
To make a proper demonstration I recommend you to do following steps:  
# Preparations:
1) Clone and compile `vmtouch` from here: https://github.com/hoytech/vmtouch
2) Generate files (1GB, 4GB). The rule is simple: first file should be less, than total amount of memory, second - more.
```
terminal1$ dd if=/dev/zero of=./large count=10240 bs=102400
terminal1$ dd if=/dev/zero of=./huge count=40960 bs=102400
```
3) Flush all caches:
```
terminal1$ echo 3 > /proc/sys/vm/drop_caches
```
4) Run script demonstating mapping these file in Page Cache (in separate terminal):
```
terminal2$ watch "echo HUGE; vmtouch ./huge ; echo LARGE ; vmtouch ./large"
```
5) Run in separate terminal
```
terminal3$ mem.sh 
```
# Demonstration:
After you flushed all caches, you shold see active and INACTIVE = 0  
1) Read large file. You will to see INACTIVE growing. Also `vmtouch`  
```
terminal1$ cat ./large > /dev/null
```
2) Read huge file. You will see on terminal with `vmstat`, that file large was moved away from inactive memory
```
terminal1$ cat ./huge > /dev/null
```
3) Read large file again. You will see on terminal with `vmstat`, how part of huge file is moving away from inactive memory
4) Read large file again and you will see, that it moved to an Active memory
5) Read huge file again. You will see, that large file was moved away partially, or was not moved at all
{'fraction_non_alphanumeric': 0.09861932938856016, 'fraction_numerical': 0.030994646379261765, 'mean_word_length': 4.144927536231884, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 11, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6550076', 'n_tokens_mistral': 1174, 'n_tokens_neox': 1060, 'n_words': 506}