<commit_before>I want to set a standard for precompiled vCPU programs.  There are
many emulators getting born, and Loader.ino will need to adopt
something as well.  Both users of real Gigatrons and users of
emulators benefit from a single standard.

Here is my proposal (made while very tired, but please feedback and
I want to consolidate this in a few days from now. So please shoot):

Marcel

=== DRAFT ===

My proposal is to use the same byte sequence that the ROM-to-RAM
routine sees.  That is essentially the in-ROM sequence, but skipping
the trampolines in the top 5 bytes of every ROM page. Those bytes
are not part of the program but part of the mechanism for accessing
the data.

Something like this (from the top of my head):

n * [ <hiAddress> + <loAddress> + <segmentSize&255> + ( segmentSize * <dataByte> ) ] + <0>

<..> represents a single byte. The rest is meta. Hope the idea is
clear. To be specific: The number "n" itself will not appear at the
start of the file.

The first two bytes are the RAM address of the first byte to be
loaded, in big-endian order.  (I'm a little-endian guy, but this
order is useful in the ROM-to-RAM loader).  The end of the sequence
is implied by the terminating zero-byte.

Programs are fragmented into RAM, in n>0 segments of 1 to 256 bytes
each, because the RAM organisation fragmented. Both Snake and Racer
extend in the unused bytes of the video area.  A segment isn't
allowed to cross a page boundary. Segments may be in any order in
the file.  They are loaded sequentially. So in case of overlap, the
later one overwrites the earlier one. (We can make movies with
that!) (There is no encoding for a zero-length program. Use an
zero-length file for that use case, for all I care... or encode a
known constant, such as [0]=0)

This implies that there is a translation to do in Loader.ino when
loading such files into a Gigatron, but I think that is ok: The
limitations on the Loader packets (60 bytes) are influenced by what
the video signal looks like. I suggest we keep that kind of
entanglement out of the file format.

The ROM sequence ends with a terminating zero. This acts as an EOF
for the ROM file".  But I dont have a strong argument in favour of
it or against.  I feel it helps files readers to know when to expect
the real EOF.  So my idea is to keep it in the file format: hiAddress=0
indicates the end of the program stream.  And about loading code/data
into zero page with such a file: do so in the first segment.

File name extension: .gt1 (For Gigatron-1)
<commit_msg>File format: include exection address<commit_after>I want to set a standard for precompiled vCPU programs.  There are
many emulators getting born, and Loader.ino will need to adopt
something as well.  Both users of real Gigatrons and users of
emulators benefit from a single standard.

Here is my proposal (made while very tired, but please feedback and
I want to consolidate this in a few days from now. So please shoot):

Marcel

=== DRAFT ===

My proposal is to use the same byte sequence that the ROM-to-RAM
routine sees.  That is essentially the in-ROM sequence, but skipping
the trampolines in the top 5 bytes of every ROM page. Those bytes
are not part of the program but part of the mechanism for accessing
the data.

Gt1File :=
        n * ( <hiAddress>
            + <loAddress>
            + <segmentSize&255>
            + segmentSize * <dataByte> )
        + <0>
        + <hiStart>
        + <loStart>
        EOF

<..> represents a single byte. The rest is meta. Hope the idea is
clear. To be specific: The number "n" itself will not appear at the
start of the file.

The first two bytes are the RAM address of the first byte to be
loaded, in big-endian order.  (I'm a little-endian guy, but this
order is useful in the ROM-to-RAM loader).  The end of the sequence
is implied by the terminating zero-byte.

Additionally, 2 bytes are appended that indicate the start address
for execution. The ROM format doesn't have these bytes. Zero means
"no execution". Although the Gigagron ROM is little-endian where
possible, in this case all 16-bit values are stored in big-endian.

Programs are fragmented into RAM, in n>0 segments of 1 to 256 bytes
each, because the RAM organisation fragmented. Both Snake and Racer
extend in the unused bytes of the video area.  A segment isn't
allowed to cross a page boundary. Segments may be in any order in
the file.  They are loaded sequentially. So in case of overlap, the
later one overwrites the earlier one. (We can make movies with
that!) (There is no encoding for a zero-length program. Use an
zero-length file for that use case, for all I care... or encode a
known constant, such as [0]=0)

This implies that there is a translation to do in Loader.ino when
loading such files into a Gigatron, but I think that is ok: The
limitations on the Loader packets (60 bytes) are influenced by what
the video signal looks like. I suggest we keep that kind of
entanglement out of the file format.

The ROM sequence ends with a terminating zero. This acts as an EOF
for the ROM file".  But I dont have a strong argument in favour of
it or against.  I feel it helps files readers to know when to expect
the real EOF.  So my idea is to keep it in the file format: hiAddress=0
indicates the end of the program stream.  And about loading code/data
into zero page with such a file: do so in the first segment.

File name extension: .gt1 (For Gigatron-1)

{'fraction_non_alphanumeric': 0.04861878453038674, 'fraction_numerical': 0.006998158379373849, 'mean_word_length': 3.5908706677937445, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5645720', 'n_tokens_mistral': 1532, 'n_tokens_neox': 1459, 'n_words': 928}