<commit_before><commit_msg>Proto definition.<commit_after># Protobuf protocol

This protocol was created by adsb-tools. This specification is official.

## Format

[Protocol buffer](https://developers.google.com/protocol-buffers/docs/overview) encoding.
Proto definition is [here](https://github.com/flamingcowtv/adsb-tools/blob/master/proto/adsb.proto).
Stream is a series of Adsb records with length encoded in a prefix (protobuf isn't self
delimiting). The prefix is structured such that an entire stream is a valid AdsbStream
record.

First frame must always be an AdsbHeader; frames can otherwise appear in any order, including
additional headers.

## Prefix
* `0x0a`: in protobuf encoding, field #1, type 2 (length-encoded bytes); (1 << 3) | 2
* Length of packet not including prefix, encoded as a [base 128 varint](https://developers.google.com/protocol-buffers/docs/encoding#varints)
  
## Packet
See [definition file](https://github.com/flamingcowtv/adsb-tools/blob/master/proto/adsb.proto)
for details.

## Tips

To decode a stream file:
`$ protoc-c --proto_path=adsb-tools/proto --decode=AdsbStream adsb-tools/proto/adsb.proto < streamfile`
{'fraction_non_alphanumeric': 0.10860121633362294, 'fraction_numerical': 0.008688097306689836, 'mean_word_length': 4.9689119170984455, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3348675', 'n_tokens_mistral': 353, 'n_tokens_neox': 343, 'n_words': 113}