<commit_before># Notes for Reading Code
by yuanmu.lb

* ./autogen.sh to generate the complete code

* true or false: 
	- for testing state of some component, true-Yes, false-No
	- for executing some function, true-1-Failed, false-0-Success

* get offset of struct member:
```
(int)&(((struct type *)NULL)->member)
```

* avoid the deadlock of multi-thread and fork
	- prefork/postfork is to solve this problem
	(http://mail-index.netbsd.org/tech-userlevel/2013/01/07/msg007117.html)
```
                           malloc-lock
                                |
            +---- thread-1 ----<1>--------------------    
            +---- thread-2 ---------------------------  +--------+
    main ---+-- main thread ---(a)--<2>---(b)---------  | Parent |
                                     |                  +--------+
    ---------------------------------|----------------------------
                                     +-- child --(c)--  +--------+
                                                        | Child  |
                                                        +--------+
```
**problem:**
parent has multi threads and thread-1 holds the malloc-lock at <1>.
then main thread fork a child process. the child copy content of 
main thread of parent, including the lock held by thread-1.
But child process does not have thread-1. so the lock in child 
process will never be released and child will be hang up.
(fork just copy the content of thread calling it. it not copys
 all the threads of parent process)

**solution:**
main thread get all locks at (a) before fork and release all 
locks at (b) of parent and (c) of child. This is what prefork/postfork
do.

* GDB with jemalloc

jemalloc is compiled default in gcc debug mode.
(configure.ac line 146 : -g3)
(-g means -g2; -g3 means more details)
 but original jemalloc is compiled in gcc -O3 in default. this may reorder the 
code, expand the loop, etc that makes you confused at gdb.
and, ./autogen.sh --enable-debug could disable the -O3 option. But it will 
enable some debug options, like opt.junk, etc.
So, I modified the -O3 to -O0 in configure.ac in line 750~760
```
	[prepare jemalloc]
	make
	(./autogen.sh, ./configure are not recommended. It will regenerate 
	some files and the comments in the files will be lost)
	(if you want to enable debug mode, run './autogen.sh --enable-debug')
	(to clean the files generated by make, run 'make clean')
	[compile gdb test]
	cd gdb-it
	gcc -g3 -o malloc malloc.c
	[run gdb]
	gdb malloc
	(gdb) set environment LD_PRELOAD=../lib/libjemalloc.so
	(gdb) b jemalloc_constructor
	(gdb) r
```

* initialize a struct
{...} could be used to initialize/assign a struct, for example :
	```
		struct person{
			int id;
			char * name;
		};
		struct person me={1, "yuanmu"};
	```

* TSD/TLS : thread specific data / thread local storage

we can use pthread APIs(pthread_key_create,...) to control tsd/tls.
use pthread_key_create to create tsd in one place 
and then every thread will have this tsd in its own space. 
```
                                  +--- thread 1 --- get tsd-A ---
    main thread -- create tsd-A --+-- main thread - get tsd-A ---<commit_msg>update notes.md<commit_after># Notes for Reading Code
by yuanmu.lb

* ./autogen.sh to generate the complete code
	(make distclean -- clean the files generated by autogen.sh)

* true or false: 
	- for testing state of some component, true-Yes, false-No
	- for executing some function, true-1-Failed, false-0-Success

* get offset of struct member:
```
(int)&(((struct type *)NULL)->member)
```

* avoid the deadlock of multi-thread and fork
	- prefork/postfork is to solve this problem
	(http://mail-index.netbsd.org/tech-userlevel/2013/01/07/msg007117.html)
```
                           malloc-lock
                                |
            +---- thread-1 ----<1>--------------------    
            +---- thread-2 ---------------------------  +--------+
    main ---+-- main thread ---(a)--<2>---(b)---------  | Parent |
                                     |                  +--------+
    ---------------------------------|----------------------------
                                     +-- child --(c)--  +--------+
                                                        | Child  |
                                                        +--------+
```
**problem:**
parent has multi threads and thread-1 holds the malloc-lock at <1>.
then main thread fork a child process. the child copy content of 
main thread of parent, including the lock held by thread-1.
But child process does not have thread-1. so the lock in child 
process will never be released and child will be hang up.
(fork just copy the content of thread calling it. it not copys
 all the threads of parent process)

**solution:**
main thread get all locks at (a) before fork and release all 
locks at (b) of parent and (c) of child. This is what prefork/postfork
do.

* GDB with jemalloc

jemalloc is compiled default in gcc debug mode.
(configure.ac line 146 : -g3)
(-g means -g2; -g3 means more details)
but original jemalloc is compiled in gcc -O3 in default. this may reorder the 
code, expand the loop, etc that makes you confused at gdb.
and, ./autogen.sh --enable-debug could disable the -O3 option. But it will 
enable some debug options, like opt.junk, etc.
So, I modified the -O3 to -O0 in configure.ac in line 750~760
```
	[prepare jemalloc]
	./autogen.sh
	make
	(if you want to enable debug mode, run './autogen.sh --enable-debug')
	(to clean the files generated by make, run 'make clean')
	[compile gdb test]
	cd gdb-it
	gcc -g3 -o malloc malloc.c
	[run gdb]
	gdb malloc
	(gdb) set environment LD_PRELOAD=../lib/libjemalloc.so
	(gdb) b jemalloc_constructor
	(gdb) r
```

* initialize a struct
{...} could be used to initialize/assign a struct, for example :
	```
		struct person{
			int id;
			char * name;
		};
		struct person me={1, "yuanmu"};
	```

* TSD/TLS : thread specific data / thread local storage

we can use pthread APIs(pthread_key_create,...) to control tsd/tls.
use pthread_key_create to create tsd in one place 
and then every thread will have this tsd in its own space. 
```
                                  +--- thread 1 --- get tsd-A ---
    main thread -- create tsd-A --+-- main thread - get tsd-A ---
{'fraction_non_alphanumeric': 0.14997611845247572, 'fraction_numerical': 0.013692087247253622, 'mean_word_length': 1.8170403587443946, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4340070', 'n_tokens_mistral': 1896, 'n_tokens_neox': 1772, 'n_words': 745}