<commit_before>=begin pod

=TITLE Sets, Bags, and Mixes

=SUBTITLE Unique collections and weighted lists in Perl 6

Often, one wants to work with lists of values that are unique. While
calling L<C<.unique>|unique> on the list repeatedly is certainly an option,
better still is to use a L<B<C<Set>>|Set> or L<B<C<SetHash>>|SetHash>,
whose elements are guaranteed to be unique.

=begin comment
=defn  Set or SetHash
Collection of distinct objects
=end comment

Other times, one wants to keep track of the number of occurrences of an
item. One could use a hash E<emdash> or use the built-in L<B<C<Bag>>|Bag> or
L<B<C<BagHash>>|BagHash> types (for integer numbers) or the
L<B<C<Mix>>|Mix> or L<B<C<MixHash>>|MixHash> types (for arbitrary
numbers).

=begin comment
=defn  Bag or BagHash
Collection of distinct objects mapped to integer weights

=defn  Mix or MixHash
Collection of distinct objects mapped to real-number weights
=end comment

=for comment
TODO: Partial type graph showing only QuantHash, Setty, Baggy, Mixy, Set,
      SetHash, Bag, BagHash, Mix, MixHash.

The six collection classes C<Set>, C<SetHash>, C<Bag>, C<BagHash>, C<Mix>,
C<MixHash>, all share similar semantics:

For one thing, as far as they are concerned, identical objects refer to the same
element - where identity is determined using the L<WHICH> methods (i.e. the same
way that the L<===> operator checks identity). For value types like C<Str>, this
means having the same value; for reference types like C<Array>, it means
referring to the same object instance.

Secondly, they provide a Hash-like interface where the actual elements of the
collection (which can be objects of any type) are the 'keys', and the associated
weights are the 'values':

=table
                      value of $a{$b} if $b       value of $a{$b} if $b
    type of $a        is an element               is not an element
    -------------     ----------------------      ---------------------
    Set / SetHash     True                        False
    Bag / BagHash     a positive integer          0
    Mix / MixHash     a non-zero real number      0

=head1 Set/Bag Operators

=comment TODO: Update this after ab5tract's set/bag/mix operator redesign.

There are many infixes devoted to preforming common operations on
L<C<Set>s|Set>, such as unions and set differences. Other operations
include boolean checks, like whether an object is an element of a
C<Set>, or whether one C<Set> is a subset of another C<Set>.

These infixes can be written using the UTF-8 character that represents
the function (like L<C<∈>|∈>, or L<C<∪>|∪>), or they can be written with
an equivalent ASCII version (like L<C<(elem)>|(elem)> or L<C<(|)>|(|)>).

Most of the time, explicitly using C<Set> objects with these infixes is
unnecessary. All of the infix operators will work on any objects of type
L<C<Any>|Any> for its arguments (e.g., L<C<List>s|List>,
L<C<Parcel>s|Parcel>, L<C<Mix>es|Mix>, etc.) and coerce them to C<Set>s
where needed.

In some cases, if the type of an argument is a L<Bag>, the
infix operator will behave in a different but analogous way to the way
it would behave with only C<Set> arguments.

=head2 Operators that return C<Bool>

=head3 infix (elem)

    multi sub infix:<(elem)>($a, Any $b --> Bool)
    multi sub infix:<(elem)>($a, Set $b --> Bool)

Returns C<True> if C<$a> is an B<element> of C<$b> as an element.

=head4 infix ∈

    only sub infix:<<"\x2208">>($a, $b --> Bool)

Equivalent to L«(elem)».

=head4 infix ∉

    only sub infix:<<"\x2208">>($a, $b --> Bool)

Equivalent to C<!(elem)>, i.e., returns C<True> if C<$a> is not an
element of C<$b>.

=head3 infix (cont)

    multi sub infix:<(cont)>(Any $a, $b --> Bool)
    multi sub infix:<(cont)>(Set $a, $b --> Bool)

Returns C<True> if C<$a> B<contains> C<$b> as an element.

=head4 infix ∋

    only sub infix:<<"\x220B">>($a, $b --> Bool)<commit_msg>Correct unicode codepoint value for ∉<commit_after>=begin pod

=TITLE Sets, Bags, and Mixes

=SUBTITLE Unique collections and weighted lists in Perl 6

Often, one wants to work with lists of values that are unique. While
calling L<C<.unique>|unique> on the list repeatedly is certainly an option,
better still is to use a L<B<C<Set>>|Set> or L<B<C<SetHash>>|SetHash>,
whose elements are guaranteed to be unique.

=begin comment
=defn  Set or SetHash
Collection of distinct objects
=end comment

Other times, one wants to keep track of the number of occurrences of an
item. One could use a hash E<emdash> or use the built-in L<B<C<Bag>>|Bag> or
L<B<C<BagHash>>|BagHash> types (for integer numbers) or the
L<B<C<Mix>>|Mix> or L<B<C<MixHash>>|MixHash> types (for arbitrary
numbers).

=begin comment
=defn  Bag or BagHash
Collection of distinct objects mapped to integer weights

=defn  Mix or MixHash
Collection of distinct objects mapped to real-number weights
=end comment

=for comment
TODO: Partial type graph showing only QuantHash, Setty, Baggy, Mixy, Set,
      SetHash, Bag, BagHash, Mix, MixHash.

The six collection classes C<Set>, C<SetHash>, C<Bag>, C<BagHash>, C<Mix>,
C<MixHash>, all share similar semantics:

For one thing, as far as they are concerned, identical objects refer to the same
element - where identity is determined using the L<WHICH> methods (i.e. the same
way that the L<===> operator checks identity). For value types like C<Str>, this
means having the same value; for reference types like C<Array>, it means
referring to the same object instance.

Secondly, they provide a Hash-like interface where the actual elements of the
collection (which can be objects of any type) are the 'keys', and the associated
weights are the 'values':

=table
                      value of $a{$b} if $b       value of $a{$b} if $b
    type of $a        is an element               is not an element
    -------------     ----------------------      ---------------------
    Set / SetHash     True                        False
    Bag / BagHash     a positive integer          0
    Mix / MixHash     a non-zero real number      0

=head1 Set/Bag Operators

=comment TODO: Update this after ab5tract's set/bag/mix operator redesign.

There are many infixes devoted to preforming common operations on
L<C<Set>s|Set>, such as unions and set differences. Other operations
include boolean checks, like whether an object is an element of a
C<Set>, or whether one C<Set> is a subset of another C<Set>.

These infixes can be written using the UTF-8 character that represents
the function (like L<C<∈>|∈>, or L<C<∪>|∪>), or they can be written with
an equivalent ASCII version (like L<C<(elem)>|(elem)> or L<C<(|)>|(|)>).

Most of the time, explicitly using C<Set> objects with these infixes is
unnecessary. All of the infix operators will work on any objects of type
L<C<Any>|Any> for its arguments (e.g., L<C<List>s|List>,
L<C<Parcel>s|Parcel>, L<C<Mix>es|Mix>, etc.) and coerce them to C<Set>s
where needed.

In some cases, if the type of an argument is a L<Bag>, the
infix operator will behave in a different but analogous way to the way
it would behave with only C<Set> arguments.

=head2 Operators that return C<Bool>

=head3 infix (elem)

    multi sub infix:<(elem)>($a, Any $b --> Bool)
    multi sub infix:<(elem)>($a, Set $b --> Bool)

Returns C<True> if C<$a> is an B<element> of C<$b> as an element.

=head4 infix ∈

    only sub infix:<<"\x2208">>($a, $b --> Bool)

Equivalent to L«(elem)».

=head4 infix ∉

    only sub infix:<<"\x2209">>($a, $b --> Bool)

Equivalent to C<!(elem)>, i.e., returns C<True> if C<$a> is not an
element of C<$b>.

=head3 infix (cont)

    multi sub infix:<(cont)>(Any $a, $b --> Bool)
    multi sub infix:<(cont)>(Set $a, $b --> Bool)

Returns C<True> if C<$a> B<contains> C<$b> as an element.

=head4 infix ∋

    only sub infix:<<"\x220B">>($a, $b --> Bool)
{'fraction_non_alphanumeric': 0.12442218798151002, 'fraction_numerical': 0.005906522855675398, 'mean_word_length': 3.455949656750572, 'pattern_counts': {'":': 0, '<': 161, '<?xml version=': 0, '>': 175, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1974006', 'n_tokens_mistral': 2568, 'n_tokens_neox': 2477, 'n_words': 1110}