<commit_before>n64split: N64 ROM splitter, texture ripper, recursive MIPS disassembler
--------

The idea behind n64split is to be able to split a ROM into various components:
assembly code, textures, level data, geometry data, behavior data, audio data;
and add on a build system that is able to take changes made to those files and 
reconstruct a ROM.  The resulting ROM is almost bit for bit compatible, with 
exception of some macro instruction translation and relocation of MIO0 data.

The disassembler is recursive, so it will disassemble all procedures it 
discovers through JALs. It also somewhat intelligently decodes instruction 
pairs to macro instructions (like LUI/ADDIU to LA and LUI/LW pairs to LW) and 
adds proper labels JALs and local branches.

Usage:
n64split -c configs/sm64.u.config "Super Mario 64 (U).z64"

This will split the ROM based on the contents of the configuration file 
"sm64.u.config".  See the comments in that file if you want to adapt it.

Output:
gen/
 +- behavior_data.s - behavior command bank
 +- sm64.s - top level assembly
 +- bin/ - raw binary data from undecoded sections and MIO0 blocks
 +- geo/ - decoded geo layout data
 +- levels/ - decoded level data
 +- textures/ - all ripped textures
 +- Makefile.gen - generated Makefile with texture and level dependencies
 
Build System: build system to reconstruct ROM after making changes
--------
The build system relies on GNU make to detect changes in resources and 
mips64-elf assembler and linker to rebuild the ROM. You could of course 
circumvent 'make' and just build everything every time with a batch script, 
but this is incredibly slow due to all the texture encoding and a terribly 
inefficient MIO0 compressor. A win32 build of make is included, but you'll 
need to either build or obtain** mips64-elf binutils on your own if you want 
to rebuild the image.

**  The developer of cen64 also makes n64tool which provides prebuilt
    mips64 toolchains.  The current list is below, but you can find more info
    on his website: http://git.cen64.com/?p=n64tool.git
  Windows (IA-32) : http://downloads.cen64.com/n64tool-win32-latest.zip
  Windows (x86_64) : http://downloads.cen64.com/n64tool-win64-latest.zip
  Linux (x86_64): http://downloads.cen64.com/n64tool-linux64-latest.tgz

Usage: With mips64-elf bin directory in your PATH, just run 'make' from the
command line, and it will handle the rest.
$ make
mips64-elf-as -mtune=vr4300 -march=vr4300 -o build/sm64.o gen/sm64.s
mips64-elf-ld -Tn64.ld -Map build/sm64.map -o build/sm64.elf build/sm64.o 
mips64-elf-objcopy build/sm64.elf build/sm64.bin -O binary
./tools/n64cksum build/sm64.bin sm64.gen.z64

Output:
build artifacts will go in build directory.
Complete ROM is sm64.gen.z64

Source is MIT licensed and available at:
https://github.com/queueRAM/sm64tools/

n64split makes use of the following libraries. See the licenses directory for
more information:
 - libpng: PNG decoding and encoding
 - capstone: raw MIPS disassembler
 - libconfig: config file reading

Changelog:
v0.3.1a: Bugfixes and cleanup
 - fix major bug introduced in n64graphics due to texture renaming
 - update INFO and stats messages from n64split
 - add ROM validation and support for .v64 (BADC ordered) ROMs
 - add hello world example
v0.3a: Complete disassembly and decoding of scripts
 - 100% of behavior scripts decoded and most behavior references detected
 - ~99% of asm sections disassembled
 - update geo layout commands 0x0A, 0x0E, 0x0F, 0x12, 0x18, 0x19, 0x1C
 - generate assembly files for standalone geo layout sections
 - detect more LA that are split across JALs
 - add option to configure number of columns in data ptr tables
 - decode interaction table and functions
 - decode cut scene tables and functions
 - decode camera change table and functions
 - decode camera preset tables and functions
 - automatic detection and disassembly of dummy stub functions
 - add examples to skip startup screens and start right in level
 - switch HUD toggle example based off of Kaze's method from Green Stars
 - removed individual texture subdirectories
v0.2a: Most scripts decoded
 - supports decoding level scripts, geometry layout, behavior scripts and assembly routines<commit_msg>Prepare README for n64split v0.4 release<commit_after>n64split: N64 ROM splitter, texture ripper, recursive MIPS disassembler
--------

The idea behind n64split is to be able to split a ROM into various components:
assembly code, textures, level data, geometry data, behavior data, audio data;
and add on a build system that is able to take changes made to those files and 
reconstruct a ROM.  The resulting ROM is almost bit for bit compatible, with 
exception of some macro instruction translation and relocation of MIO0 data.

The disassembler is recursive, so it will disassemble all procedures it 
discovers through JALs. It also somewhat intelligently decodes instruction 
pairs to macro instructions (like LUI/ADDIU to LA and LUI/LW pairs to LW) and 
adds proper labels JALs and local branches.

Usage:
$ n64split [-c CONFIG] [-o OUTPUT_DIR] [-s SCALE] [-t] [-v] ROM

Optional arguments:
 -c CONFIG     ROM configuration file (default: determine from checksum)
 -o OUTPUT_DIR output directory (default: {CONFIG.basename}.split)
 -s SCALE      amount to scale models by (default: 1024.000000)
 -p            generate procedure table for analysis
 -t            generate large texture for MIO0 blocks
 -v            verbose progress output

File arguments:
 ROM        input ROM file

Example usage:
$ n64split "Super Mario 64 (U).z64"

This will split the ROM based on the contents of the configuration file 
"sm64.u.config".  See the comments in that file if you want to adapt it.

Output:
sm64.split/
 +- behavior_data.s - behavior command bank
 +- sm64.s - top level assembly
 +- bin/ - raw binary data from undecoded sections and MIO0 blocks
 +- geo/ - decoded geo layout data
 +- levels/ - decoded level data
 +- textures/ - all ripped textures
 +- Makefile.split - generated Makefile with texture and level dependencies
 
Build System: build system to reconstruct ROM after making changes
--------
The build system relies on GNU make to detect changes in resources and 
mips64-elf assembler and linker to rebuild the ROM. You could of course 
circumvent 'make' and just build everything every time with a batch script, 
but this is incredibly slow due to all the texture encoding and a terribly 
inefficient MIO0 compressor. A win32 build of make is included, but you'll 
need to either build or obtain** mips64-elf binutils on your own if you want 
to rebuild the image.

**  The developer of cen64 also makes n64tool which provides prebuilt
    mips64 toolchains.  The current list is below, but you can find more info
    on his website: http://git.cen64.com/?p=n64tool.git
  Windows (IA-32) : http://downloads.cen64.com/n64tool-win32-latest.zip
  Windows (x86_64) : http://downloads.cen64.com/n64tool-win64-latest.zip
  Linux (x86_64): http://downloads.cen64.com/n64tool-linux64-latest.tgz

Usage: With mips64-elf bin directory in your PATH, just run 'make' from the
command line, and it will handle the rest.
$ make
mips64-elf-as -mtune=vr4300 -march=vr4300 -o build/sm64.o gen/sm64.s
mips64-elf-ld -Tn64.ld -Map build/sm64.map -o build/sm64.elf build/sm64.o 
mips64-elf-objcopy build/sm64.elf build/sm64.bin -O binary
./tools/n64cksum build/sm64.bin sm64.gen.z64

Output:
build artifacts will go in 'build' directory.
Complete ROM is sm64.z64

Source is MIT licensed and available at:
https://github.com/queueRAM/sm64tools/

n64split makes use of the following libraries. See the licenses directory for
more information:
 - libpng: PNG decoding and encoding
 - capstone: raw MIPS disassembler
 - libconfig: config file reading

Changelog:
v0.4: More data: music, collision, more configs
 - new config files: SM64 (J), SM64 (E), MK64 (U)
 - add IA4 and IA1 graphics types
 - add command line options:
   - generate large texture from MIO0 block
   - generate procedure table
   - specify output directory
 - disassembler changes:
   - add more address detection and %hi/%lo macro insertion
   - handle BAL instructions
 - add automatic config file detection based on header checksum
 - corrected a lot of textures in the SM64 configs
 - add music sequence bank and instrument set parsing
 - add collision data parser and export to Wavefront OBJ
 - add sm64walk tool to walk through SM64 level scripts
 - correct SM64 behavior script lengths
v0.3.1a: Bugfixes and cleanup
 - fix major bug introduced in n64graphics due to texture renaming
 - update INFO and stats messages from n64split
 - add ROM validation and support for .v64 (BADC ordered) ROMs
 - add hello world example
v0.3a: Complete disassembly and decoding of scripts
 - 100% of behavior scripts decoded and most behavior references detected
 - ~99% of asm sections disassembled
 - update geo layout commands 0x0A, 0x0E, 0x0F, 0x12, 0x18, 0x19, 0x1C
 - generate assembly files for standalone geo layout sections
 - detect more LA that are split across JALs
 - add option to configure number of columns in data ptr tables
 - decode interaction table and functions
 - decode cut scene tables and functions
 - decode camera change table and functions
 - decode camera preset tables and functions
 - automatic detection and disassembly of dummy stub functions
 - add examples to skip startup screens and start right in level
 - switch HUD toggle example based off of Kaze's method from Green Stars
 - removed individual texture subdirectories
v0.2a: Most scripts decoded
 - supports decoding level scripts, geometry layout, behavior scripts and assembly routines
{'fraction_non_alphanumeric': 0.06054020490530891, 'fraction_numerical': 0.034357859877884714, 'mean_word_length': 3.8175473579262214, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '895990', 'n_tokens_mistral': 2952, 'n_tokens_neox': 2675, 'n_words': 1319}