<commit_before>=begin pod

=TITLE role Blob

=SUBTITLE Immutable buffer for binary data ('Binary Large OBject')

    role Blob[::T = uint8] does Positional[T] does Stringy { }

The C<Blob> role is an immutable interface to binary types, and offers a
list-like interface to lists of integers, typically unsigned integers.

=head1 Methods

=head2 method new

Defined as:

    method new(*@codes)

Usage:

    $blob = Blob.new(LIST)

Creates a C<Blob> from a list of integers.

=head2 method  Bool

Defined as:

    multi method Bool(Blob:D:)

Usage:

    BLOB.Bool

Returns C<False> if and only if the buffer is empty.

=head2 method elems

Defined as:

    multi method elems(Blob:D:) returns Int:D

Usage:

    BLOB.elems

Returns the number of elements of the buffer.

=head2 method decode

Defined as:

    multi method decode(Blob:D: Str:D $encoding = 'UTF-8') returns Str:D

Usage:

    BLOB.decode(ENCODING?)

Applies an encoding to turn the blob into a L<Str|/type/Str>.

=head2 method subbuf

Defined as:

    method subbuf(Int $from, Int $len = self.elems) returns Blob:D

Usage:

    BLOB.subbuf(FROM, LENGTH?)

Extracts a part of the invocant buffer, starting from the index with
elements C<$from>, and taking C<$len> elements (or less if the buffer is
shorter), and creates a new buffer as the result.

    say Blob.new(1..10).subbuf(2, 4);    # Blob:0x<03 04 05 06>

=head2 method unpack

Defined as:

    method unpack(Blob:D: $template) returns List:D

Usage:

    BLOB.unpack(TEMPLATE)

Extracts features from the blob according to the template string, and
returns them as a list.

The template string consists of zero or more units that begin with an ASCII
letter, and are optionally followed by a quantifier. The quantifier can be
C<*> (which typically stands for "use up the rest of the Blob here"), or a
positive integer (without a C<+>).

Whitespace between template units is ignored.

Examples of valid templates include C<"A4 C n*"> and C<"A*">.

The following letters are recognized:

=begin table

    Letter  Meaning
    ======  =======
    A       Extract a string, where each element of the Blob maps to a codepoint
    a       Same as A
    C       Extract an element from the blob as an integer
    H       Extracts a hex string
    L       Extracts four elements and returns them as a single unsigned integer
    n       Extracts two elements and combines them in "network" (big-endian) byte order into a single integer
    N       Extracts four elements and combines them in "network" (big-endian) byte order into a single integer
    S       Extracts two elements and returns them as a single unsigned integer
    v       Same as S
    V       Same as L
    x       Drop an element from the blob (that is, ignore it)
    Z       Same as A

=end table

=head2 sub pack

    sub pack(Str $template, *@items) returns Buf

Packs the given items according to the template and returns a buffer
containing the packed bytes.

The template string consists of zero or more units that begin with an ASCII
letter, and are optionally followed by a quantifier.  For details, see
L<unpack|/routine/unpack>.

=end pod
<commit_msg>Mention Blob.subbuf can also take a Range<commit_after>=begin pod

=TITLE role Blob

=SUBTITLE Immutable buffer for binary data ('Binary Large OBject')

    role Blob[::T = uint8] does Positional[T] does Stringy { }

The C<Blob> role is an immutable interface to binary types, and offers a
list-like interface to lists of integers, typically unsigned integers.

=head1 Methods

=head2 method new

Defined as:

    method new(*@codes)

Usage:

    $blob = Blob.new(LIST)

Creates a C<Blob> from a list of integers.

=head2 method  Bool

Defined as:

    multi method Bool(Blob:D:)

Usage:

    BLOB.Bool

Returns C<False> if and only if the buffer is empty.

=head2 method elems

Defined as:

    multi method elems(Blob:D:) returns Int:D

Usage:

    BLOB.elems

Returns the number of elements of the buffer.

=head2 method decode

Defined as:

    multi method decode(Blob:D: Str:D $encoding = 'UTF-8') returns Str:D

Usage:

    BLOB.decode(ENCODING?)

Applies an encoding to turn the blob into a L<Str|/type/Str>.

=head2 method subbuf

Defined as:

    method subbuf(Int $from, Int $len = self.elems) returns Blob:D
    method subbuf(Range $range) returns Blob:D

Usage:

    BLOB.subbuf(FROM, LENGTH?)
    BLOB.subbuf(RANGE)

Extracts a part of the invocant buffer, starting from the index with
elements C<$from>, and taking C<$len> elements (or less if the buffer is
shorter), and creates a new buffer as the result.

    say Blob.new(1..10).subbuf(2, 4);    # Blob:0x<03 04 05 06>
    say Blob.new(1..10).subbuf(*-2);     # Blob:0x<09 0a>
    say Blob.new(1..10).subbuf(*-5,2);   # Blob:0x<06 07>

For convenience, also allows a C<Range> to be specified to indicate which
part of the invocant buffer you would like:

    say Blob.new(1..10).subbuf(2..5);    # Blob:0x<03 04 05 06>

=head2 method unpack

Defined as:

    method unpack(Blob:D: $template) returns List:D

Usage:

    BLOB.unpack(TEMPLATE)

Extracts features from the blob according to the template string, and
returns them as a list.

The template string consists of zero or more units that begin with an ASCII
letter, and are optionally followed by a quantifier. The quantifier can be
C<*> (which typically stands for "use up the rest of the Blob here"), or a
positive integer (without a C<+>).

Whitespace between template units is ignored.

Examples of valid templates include C<"A4 C n*"> and C<"A*">.

The following letters are recognized:

=begin table

    Letter  Meaning
    ======  =======
    A       Extract a string, where each element of the Blob maps to a codepoint
    a       Same as A
    C       Extract an element from the blob as an integer
    H       Extracts a hex string
    L       Extracts four elements and returns them as a single unsigned integer
    n       Extracts two elements and combines them in "network" (big-endian) byte order into a single integer
    N       Extracts four elements and combines them in "network" (big-endian) byte order into a single integer
    S       Extracts two elements and returns them as a single unsigned integer
    v       Same as S
    V       Same as L
    x       Drop an element from the blob (that is, ignore it)
    Z       Same as A

=end table

=head2 sub pack

    sub pack(Str $template, *@items) returns Buf

Packs the given items according to the template and returns a buffer
containing the packed bytes.

The template string consists of zero or more units that begin with an ASCII
letter, and are optionally followed by a quantifier.  For details, see
L<unpack|/routine/unpack>.

=end pod

{'fraction_non_alphanumeric': 0.07384155455904334, 'fraction_numerical': 0.012257100149476832, 'mean_word_length': 3.23213156230234, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4072466', 'n_tokens_mistral': 2162, 'n_tokens_neox': 2040, 'n_words': 946}