<commit_before>header:
0x00 4 bytes 0003
0x04 2 bytes uint16 width of grid
0x06 2 bytes uint16 length of grid
0x08 4 bytes signed int32, x coordinate of start of grid 
0x0C 4 bytes signed int32, z coordinate of start of grid 
0x10 4 bytes signed(or unsigned) int32, grid cell x size
0x14 4 bytes signed(or unsigned) int32, grid cell z size
0x18 2 bytes uint16 entries count for last section
0x1A 2 bytes padding? always 0
0x1C 4 bytes offset into triangle indices 
0x20 4 bytes offset into triangles 
0x24 4 bytes offset into vertices
0x28 4 bytes offset into ??

header length: 0x2C bytes 
0x8,0xC is the smallest coordinate of the level at which the grid starts
0x10, 0x14 are the sizes of each cell in the grid along the x and z axis

-----
at 0x2C some sort of grid data structure starts with 8 bytes per grid entry. It's a grid with each cell of the grid being a quadtree
8 bytes per entry 

[int8 or uint8 triangles count] [unknown byte, always 0?] [uint16 grid entry index] [uint32 triangle indices index]

grid entry index points to the children of the entry. index+0 to index+3 are its children if index is not 0.
When triangles count is not 0, [triangles count] entries starting at the indices index belong to the grid entry
-----

grid section: 2 bytes per entry? Is multiplied by 0x24 to get offset into triangles 

----- 

triangles: 0x24 bytes per entry 

0x00 4 bytes vertex index 
0x04 4 bytes vertex index  
0x08 4 bytes vertex index 
0x0C float, affects wall
..
0x18 1 byte flags?
..
0x20 1 byte used for position stuff?
.. 

-----

vertices: 0xC bytes per entry (probably XYZ)<commit_msg>Add triangle data info<commit_after>header:
0x00 4 bytes 0003
0x04 2 bytes uint16 width of grid
0x06 2 bytes uint16 length of grid
0x08 4 bytes signed int32, x coordinate of start of grid 
0x0C 4 bytes signed int32, z coordinate of start of grid 
0x10 4 bytes signed(or unsigned) int32, grid cell x size
0x14 4 bytes signed(or unsigned) int32, grid cell z size
0x18 2 bytes uint16 entries count for last section
0x1A 2 bytes padding? always 0
0x1C 4 bytes offset into triangle indices 
0x20 4 bytes offset into triangles 
0x24 4 bytes offset into vertices
0x28 4 bytes offset into ??

header length: 0x2C bytes 
0x8,0xC is the smallest coordinate of the level at which the grid starts
0x10, 0x14 are the sizes of each cell in the grid along the x and z axis

-----
at 0x2C some sort of grid data structure starts with 8 bytes per grid entry. It's a grid with each cell of the grid being a quadtree
8 bytes per entry 

[int8 or uint8 triangles count] [unknown byte, always 0?] [uint16 grid entry index] [uint32 triangle indices index]

grid entry index points to the children of the entry. index+0 to index+3 are its children if index is not 0.
When triangles count is not 0, [triangles count] entries starting at the indices index belong to the grid entry
-----

grid section: 2 bytes per entry? Is multiplied by 0x24 to get offset into triangles 

----- 

triangles: 0x24 bytes per entry 

0x00 4 bytes vertex index 
0x04 4 bytes vertex index  
0x08 4 bytes vertex index 
0x0C float, affects wall, positioning of triangles?
0x10 signed short, X direction of normal vector of length ~1 
0x12 signed short, Y direction of normal vector of length ~1 
0x14 signed short, Z direction of normal vector of length ~1 
0x16 ushort collision type 
0x18 byte, messes with collision if changed
0x19 byte (always 0x01? Doesn't matter if set to 0)
0x1A ushort, can be set to 0
0x1C ushort, can be set to 0
0x1E ushort, can be set to 0
0x20-0x24 padding 
..
0x18 1 byte flags?
..
0x20 1 byte used for position stuff?
.. 

-----

vertices: 0xC bytes per entry (probably XYZ)
{'fraction_non_alphanumeric': 0.04391707583196945, 'fraction_numerical': 0.07855973813420622, 'mean_word_length': 3.7747395833333335, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3583279', 'n_tokens_mistral': 1265, 'n_tokens_neox': 1097, 'n_words': 640}