<commit_before># ELF Database system
In 2008, Elfdbs is a prouct for project of Lanzhou University.

## Structure
![structure](https://github.com/unanao/Elfdbs/blob/master/document/12.png.png)

## Implement
### ELF analysis and how to read it
ELF File Format
![ELF Format](https://github.com/unanao/Elfdbs/blob/master/document/img/elf_formate.JPG)

#### Read ELF file header
The begin of the file is ELF header, describ the organization of the file."section" include information of "link view", for example command, data, symbol, relocate symbol and so on.

The structure of "ELF header table"   
```c
typedef struct
{
	unsigned char e_ident[EI_NIDENT];     /* Magic number and other info */
	Elf32_Half    e_type;                 /* Object file type */
	Elf32_Half    e_machine;              /* Architecture */
	Elf32_Word    e_version;              /* Object file version */
	Elf32_Addr    e_entry;                /* Entry point virtual address */
	Elf32_Off     e_phoff;                /* Program header table file offset */
	Elf32_Off     e_shoff;                /* Section header table file offset */
	Elf32_Word    e_flags;                /* Processor-specific flags */
	Elf32_Half    e_ehsize;               /* ELF header size in bytes */
	Elf32_Half    e_phentsize;            /* Program header table entry size */
	Elf32_Half    e_phnum;                /* Program header table entry count */
	Elf32_Half    e_shentsize;            /* Section header table entry size */
	Elf32_Half    e_shnum;                /* Section header table entry count */
	Elf32_Half    e_shstrndx;             /* Section header string table index */
} Elf32_Ehdr;
```

Procedure to read ELF header:  
1. Jump to begin of file header

```c
lseek(fd, 0, SEEK_SET)
```

2.Calculate the size of ELF header  
```c 
size = sizeof(Elf32_Ehdr) 
```

3.read ELF header to the buffer  
```c
read(fd, buf, size) 
```

#### Read Program Header table
Data structure of "program header table"  
```c
typedef struct { 
	Elf32_Word p_type; 
	Elf32_Off  p_offset; 
	Elf32_Addr p_vaddr; 
	Elf32_Addr p_paddr; 
	Elf32_Word p_filesz; 
	Elf32_Word p_memsz; 
	Elf32_Word p_flags; 
	Elf32_Word p_align; 
} Elf32_phdr; 
```


We can get size and offset from ELF Header
* e_phentsize and  e_phnum: size of program header
* e_phoff : offset of program header

Howto read "program header table"  

1.Jump to program header  
```c
lseek(fd,Elf32_Ehdr->e_phoff,SEEK_SET)
```

2.Calculate size of program header table  
```c
size=Elf32_e_phentsize * Elf32_e_phnum;
```

3.Read to buffer  
```c
read(fd,buf,size);
```  

#### Read "Section Header table"
Data structure of "section header table"
```c
typedef struct{ 
	Elf32_Word sh_name; 
	Elf32_Word sh_type; 
	Elf32_Word sh_flags; 
	Elf32_Addr sh_addr; 
	Elf32_Off sh_offset; 
	Elf32_Word sh_size; 
	Elf32_Word sh_link; 
	Elf32_Word sh_info; 
	Elf32_Word sh_addralign; 
	Elf32_Word sh_entsize; 
}Elf32_Shdr; 
```

Get size and offset from ELF header
* e_shentsize and e_shnum : size 
* e_shoff: offset

Procedure for reading "Section header table"  
1. Jump to Section header  
```c
lseek(fd, Elf32_Ehdr->e_shoff, SEEK_SET)
```  
2. Calculate size of "Section header table"  
```c  
size=Elf32_e_shentsize * Elf32_e_shnum;
```  
3. Read "Section Header Table"  
```c
read(fd,buf,size);
```  

#### Read ELF Section<commit_msg>format adjust<commit_after># ELF Database system
In 2008, Elfdbs is a prouct for project of Lanzhou University.

## Structure
![structure](https://github.com/unanao/Elfdbs/blob/master/document/12.png.png)

## Implement
### ELF analysis and how to read it
ELF File Format
![ELF Format](https://github.com/unanao/Elfdbs/blob/master/document/img/elf_formate.JPG)

#### Read ELF file header
The begin of the file is ELF header, describ the organization of the file."section" include information of "link view", for example command, data, symbol, relocate symbol and so on.

The structure of "ELF header table"   
```c
typedef struct
{
	unsigned char e_ident[EI_NIDENT];     /* Magic number and other info */
	Elf32_Half    e_type;                 /* Object file type */
	Elf32_Half    e_machine;              /* Architecture */
	Elf32_Word    e_version;              /* Object file version */
	Elf32_Addr    e_entry;                /* Entry point virtual address */
	Elf32_Off     e_phoff;                /* Program header table file offset */
	Elf32_Off     e_shoff;                /* Section header table file offset */
	Elf32_Word    e_flags;                /* Processor-specific flags */
	Elf32_Half    e_ehsize;               /* ELF header size in bytes */
	Elf32_Half    e_phentsize;            /* Program header table entry size */
	Elf32_Half    e_phnum;                /* Program header table entry count */
	Elf32_Half    e_shentsize;            /* Section header table entry size */
	Elf32_Half    e_shnum;                /* Section header table entry count */
	Elf32_Half    e_shstrndx;             /* Section header string table index */
} Elf32_Ehdr;
```

Procedure to read ELF header:  
1.Jump to begin of file header

```c
lseek(fd, 0, SEEK_SET)
```

2.Calculate the size of ELF header  
```c 
size = sizeof(Elf32_Ehdr) 
```

3.read ELF header to the buffer  
```c
read(fd, buf, size) 
```

#### Read Program Header table
Data structure of "program header table"  
```c
typedef struct { 
	Elf32_Word p_type; 
	Elf32_Off  p_offset; 
	Elf32_Addr p_vaddr; 
	Elf32_Addr p_paddr; 
	Elf32_Word p_filesz; 
	Elf32_Word p_memsz; 
	Elf32_Word p_flags; 
	Elf32_Word p_align; 
} Elf32_phdr; 
```


We can get size and offset from ELF Header
* e_phentsize and  e_phnum: size of program header
* e_phoff : offset of program header

Howto read "program header table"  

1.Jump to program header  
```c
lseek(fd,Elf32_Ehdr->e_phoff,SEEK_SET)
```

2.Calculate size of program header table  
```c
size=Elf32_e_phentsize * Elf32_e_phnum;
```

3.Read to buffer  
```c
read(fd,buf,size);
```  

#### Read "Section Header table"
Data structure of "section header table"
```c
typedef struct{ 
	Elf32_Word sh_name; 
	Elf32_Word sh_type; 
	Elf32_Word sh_flags; 
	Elf32_Addr sh_addr; 
	Elf32_Off sh_offset; 
	Elf32_Word sh_size; 
	Elf32_Word sh_link; 
	Elf32_Word sh_info; 
	Elf32_Word sh_addralign; 
	Elf32_Word sh_entsize; 
}Elf32_Shdr; 
```

Get size and offset from ELF header
* e_shentsize and e_shnum : size 
* e_shoff: offset

Procedure for reading "Section header table"  
1.Jump to Section header  
```c
lseek(fd, Elf32_Ehdr->e_shoff, SEEK_SET)
```  
2.Calculate size of "Section header table"  
```c  
size=Elf32_e_shentsize * Elf32_e_shnum;
```  
1.Read "Section Header Table"  
```c
read(fd,buf,size);
```  

#### Read ELF Section
{'fraction_non_alphanumeric': 0.0957957957957958, 'fraction_numerical': 0.02942942942942943, 'mean_word_length': 3.076499388004896, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 7, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1127981', 'n_tokens_mistral': 2586, 'n_tokens_neox': 2359, 'n_words': 702}