<commit_before>=head0 The Basics

Perl has traditionally been very strong in the area of gathering information
from text files, and report it. In fact that is what Perl was written for
originally.

A typical such problem might look like this: You host a table tennis
tournament, and the referees tell you the results of each game in the format
C<Player 1 vs Player 2 | 3:2>, which means that C<Player 1> won against
C<Player 2> by 3 to 2 sets. You need a script that sums up how many games and
sets each player has won, and thus determines the overall winner.

The input data looks like this:

    <NAME> <NAME>
    Ana vs Dave | 3:0
    Charlie vs Beth | 3:1
    Ana vs Beth | 2:3
    Dave vs Charlie | 3:0
    Ana vs Charlie | 3:1
    Beth vs Dave | 0:3

Where the first line is just the list of players, and every line after that is
a result of a match.

Here's one way to solve that problem in Perl 6:

    use v6;

    my $file = open 'scores';

    my @names = $file.get.split(' ');
    my %games;
    my %sets;

    for $file.lines -> $line {
        my ($pairing, $result) = $line.split(' | ');
        my ($p1, $p2) = $pairing.split(' vs ');
        my ($r1, $r2) = $result.split(':');
        %sets{$p1} += $r1;
        %sets{$p2} += $r2;
        if $r1 > $r2 {
            %games{$p1}++;
        } else {
            %games{$p2}++;
        }
    }

    my @sorted = @names.sort({ %sets{$_} }).sort({ %games{$_} }).reverse;
    for @sorted -> $n {
        say "$n has won { %games{$n} } games and { %sets{$n} } sets";
    }

Every Perl 6 program should begin with C<use v6;>. This line tells the
compiler which version of Perl it is written in, and if you accidentally use
the Perl 5 interpreter, it gives you a rather helpful error message.

A Perl 6 program consists of a number of statements separated by semicolons,
with the small exception that the semicolon is optional if the statement ends
in a curly bracket and stands at the end of a line.

    my $file = open 'scores';

C<my> declares a lexical variable, that is a variable that is visible in the
current block, or in the rest of the file if it's not in a block.

A variable name begins with a I<sigil>, which is a non-word character like
C<$>, C<@>, C<%>, C<&> or sometimes even more characters, like the double
colon C<::>. After the sigil comes an identifier, which may consist of
letters, digits and the underscore. Between letters you can also use a dash
C<-> or a hyphen C<'>, so C<isn't> and C<very-cool> are valid identifiers.

Each sigil carries a meaning. Variables starting with a dollar can hold
arbitrary values. Here the built-in function C<open> is called, which opens a
file of name C<scores>, and returns an object describing that file, a
I<file handle>. The equality sign C<=> I<assigns> that file handle to the
variable on the left, which means it takes care the C<$file> stores that
file handle.

C<'scores'> is a I<string literal>. A string is a piece of text, or sequence
of characters. Writing it directly after the function name passes it as an
argument to that function. A different way to write that is C<open('scores')>,
which is more familiar to programmers of C-style languages.

    my @names = $file.get.split(' ');

Another variable declaration; the C<@> sigil makes that variable an C<Array>.
Arrays store ordered lists of things and can be manipulated later. The
right-hand side uses the previously declared and initialized variable
C<$file>, and calls the C<get> method on it, which reads one line from the
file, and returns it (with the line ending removed). We call the C<split>
method on the result string, passing a string of a single blank to it.
C<split> decomposes the string it was called on into a list of strings, so it
turns C<"<NAME>"> into C<"Beth", "Ana", "Charlie", "Dave">.
This list is stored into the array C<@names>.

    my %games;
    my %sets;

Here two variables are declared as hashes. A C<Hash> is an unordered
collection of pairs of keys and values. Other programming languages call that
a I<hash table>, I<dictionary> or I<map>. You can ask a hash table for the
value that corresponds to a certain C<$key> with C<%hash{$key}>.

Here in the score counting program C<%games> stores the number of won games
per player, and C<%sets> the number of won sets per player.

    for $file.lines -> $line {
        ...
    }

C<$file.lines> produces a list of lines that are read from the file
C<scores>.
C<for> introduces a loop that runs the block indicated by C<...> once for
each item of the list, setting the variable C<$line> to current value.

During the first iteration C<$line> contains the string
C<Ana vs Dave | 3:0>, during the second C<Charlie vs Beth | 3:1> and so on.

        my ($pairing, $result) = $line.split(' | ');

Again C<my> declares variables, this time a two of them at once. The
right-hand side of the assignment is again a call to C<split>, this time
splitting up on a vertical bar surrounded by spaces. C<$pairing> gets the
first item of the returned list, C<$result> the second.

So while processing the first line, C<$pairing> holds the string
C<Ana vs Dave> and C<$result> is set to C<3:0>.

The next two lines follow the same pattern:

        my ($p1, $p2) = $pairing.split(' vs ');
        my ($r1, $r2) = $result.split(':');

The names of the two players are extracted and stored in C<$p1> and C<$p2>,
and the results for each player are stored in C<$r1> and C<$r2>.

So for the first line the variables are set up as follows:

    Variable    Contents
    $line       '<NAME> | 3:0'
    $pairing    'Ana vs Dave'
    $result     '3:0'
    $p1         'Ana'
    $p2         'Dave'
    $r1         '3'
    $r2         '0'

Then the number of won sets are counted:

        %sets{$p1} += $r1;
        %sets{$p2} += $r2;

This is a shortcut for

        %sets{$p1} = %sets{$p1} + $r1;
        %sets{$p2} = %sets{$p2} + $r2;

So C<+= $r1> means I<increase the thing on the left by $r1>.
In the first iteration C<%sets{$p1}> is not yet set, so it defaults to a
special value called C<undef>. The plus sign treats it as a number, the result
of which is zero. The plus sign also treats C<$r1> as a number.

Before these two lines are execute, C<%sets> is empty. The assignment
automatically populates the hash; after these two lines run for the first
time, C<%sets> is C<< 'Ana' => 3, 'Dave' => 0 >>.
(The fat arrow C<< => >> separates key and value in a C<Pair>.)

=for vim: spell<commit_msg>more [basics] and build system hacking<commit_after>=head0 The Basics

Perl has traditionally been very strong in the area of gathering information
from text files, and report it. In fact that is what Perl was written for
originally.

A typical such problem might look like this: You host a table tennis
tournament, and the referees tell you the results of each game in the format
C<Player 1 vs Player 2 | 3:2>, which means that C<Player 1> won against
C<Player 2> by 3 to 2 sets. You need a script that sums up how many games and
sets each player has won, and thus determines the overall winner.

The input data looks like this:

    <NAME> <NAME>
    Ana vs Dave | 3:0
    Charlie vs Beth | 3:1
    Ana vs Beth | 2:3
    Dave vs Charlie | 3:0
    Ana vs Charlie | 3:1
    Beth vs Dave | 0:3

Where the first line is just the list of players, and every line after that is
a result of a match.

Here's one way to solve that problem in Perl 6:

    use v6;

    my $file = open 'scores';

    my @names = $file.get.split(' ');
    my %games;
    my %sets;

    for $file.lines -> $line {
        my ($pairing, $result) = $line.split(' | ');
        my ($p1, $p2) = $pairing.split(' vs ');
        my ($r1, $r2) = $result.split(':');
        %sets{$p1} += $r1;
        %sets{$p2} += $r2;
        if $r1 > $r2 {
            %games{$p1}++;
        } else {
            %games{$p2}++;
        }
    }

    my @sorted = @names.sort({ %sets{$_} }).sort({ %games{$_} }).reverse;
    for @sorted -> $n {
        say "$n has won { %games{$n} } games and { %sets{$n} } sets";
    }

Every Perl 6 program should begin with C<use v6;>. This line tells the
compiler which version of Perl it is written in, and if you accidentally use
the Perl 5 interpreter, it gives you a rather helpful error message.

A Perl 6 program consists of a number of statements separated by semicolons,
with the small exception that the semicolon is optional if the statement ends
in a curly bracket and stands at the end of a line.

    my $file = open 'scores';

C<my> declares a lexical variable, that is a variable that is visible in the
current block, or in the rest of the file if it's not in a block.

A variable name begins with a I<sigil>, which is a non-word character like
C<$>, C<@>, C<%>, C<&> or sometimes even more characters, like the double
colon C<::>. After the sigil comes an identifier, which may consist of
letters, digits and the underscore. Between letters you can also use a dash
C<-> or a hyphen C<'>, so C<isn't> and C<very-cool> are valid identifiers.

Each sigil carries a meaning. Variables starting with a dollar can hold
arbitrary values. Here the built-in function C<open> is called, which opens a
file of name C<scores>, and returns an object describing that file, a
I<file handle>. The equality sign C<=> I<assigns> that file handle to the
variable on the left, which means it takes care the C<$file> stores that
file handle.

C<'scores'> is a I<string literal>. A string is a piece of text, or sequence
of characters. Writing it directly after the function name passes it as an
argument to that function. A different way to write that is C<open('scores')>,
which is more familiar to programmers of C-style languages.

    my @names = $file.get.split(' ');

Another variable declaration; the C<@> sigil makes that variable an C<Array>.
Arrays store ordered lists of things and can be manipulated later. The
right-hand side uses the previously declared and initialized variable
C<$file>, and calls the C<get> method on it, which reads one line from the
file, and returns it (with the line ending removed). We call the C<split>
method on the result string, passing a string of a single blank to it.
C<split> decomposes the string it was called on into a list of strings, so it
turns C<"<NAME> <NAME>"> into C<"Beth", "Ana", "Charlie", "Dave">.
This list is stored into the array C<@names>.

    my %games;
    my %sets;

Here two variables are declared as hashes. A C<Hash> is an unordered
collection of pairs of keys and values. Other programming languages call that
a I<hash table>, I<dictionary> or I<map>. You can ask a hash table for the
value that corresponds to a certain C<$key> with C<%hash{$key}>.

Here in the score counting program C<%games> stores the number of won games
per player, and C<%sets> the number of won sets per player.

    for $file.lines -> $line {
        ...
    }

C<$file.lines> produces a list of lines that are read from the file
C<scores>.
C<for> introduces a loop that runs the block indicated by C<...> once for
each item of the list, setting the variable C<$line> to current value.

During the first iteration C<$line> contains the string
C<<NAME> | 3:0>, during the second C<Charlie vs Beth | 3:1> and so on.

        my ($pairing, $result) = $line.split(' | ');

Again C<my> declares variables, this time a two of them at once. The
right-hand side of the assignment is again a call to C<split>, this time
splitting up on a vertical bar surrounded by spaces. C<$pairing> gets the
first item of the returned list, C<$result> the second.

So while processing the first line, C<$pairing> holds the string
C<Ana vs Dave> and C<$result> is set to C<3:0>.

The next two lines follow the same pattern:

        my ($p1, $p2) = $pairing.split(' vs ');
        my ($r1, $r2) = $result.split(':');

The names of the two players are extracted and stored in C<$p1> and C<$p2>,
and the results for each player are stored in C<$r1> and C<$r2>.

So for the first line the variables are set up as follows:

=begin table Contents of Variables

=headrow

=row

=cell Variable

=cell Contents

=bodyrows

=row

=cell $line

=cell '<NAME> | 3:0'

=row

=cell $pairing

=cell 'Ana vs Dave'

=row

=cell $result 

=cell '3:0'

=row

=cell $p1

=cell 'Ana'

=row

=cell $p2

=cell 'Dave'

=row

=cell $r1

=cell '3'

=row

=cell $r2

=cell '0'

=end table

Then the number of won sets are counted:

        %sets{$p1} += $r1;
        %sets{$p2} += $r2;

This is a shortcut for

        %sets{$p1} = %sets{$p1} + $r1;
        %sets{$p2} = %sets{$p2} + $r2;

So C<+= $r1> means I<increase the thing on the left by $r1>.
In the first iteration C<%sets{$p1}> is not yet set, so it defaults to a
special value called C<undef>. The plus sign treats it as a number, the result
of which is zero. The plus sign also treats C<$r1> as a number.

Before these two lines are execute, C<%sets> is empty. The assignment
automatically populates the hash; after these two lines have run for the
first time, C<%sets> is C<< 'Ana' => 3, 'Dave' => 0 >>.
(The fat arrow C<< => >> separates key and value in a C<Pair>.)

        if $r1 > $r2 {
            %games{$p1}++;
        } else {
            %games{$p2}++;
        }

If C<$r1> has a larger value than C<$r2>, C<%games{$p1}> is incremented by
one, if that hash item did not exist previously it springs into existence
automatically If C<$r1> is not larger than C<$r2>, C<%games{$p2}> is
incremented.

C<$thing++> is short for C<$thing += 1> or
C<$thing = $thing + 1>, with the small exception that the return value of the
expression is C<$thing>, not the incremented value. Just like in the C
programming language you can also use C<++> as a prefix, in which case it
returns the increment value; C<my $x = 1; say ++$x> prints C<2>.

    my @sorted = @names.sort({ %sets{$_} }).sort({ %games{$_} }).reverse;

This might look a bit scary at first, but it consists of three relatively
simple steps. An array knows how to sort itself with the C<sort> method.
However to print out the winner first, we have to sort by the score of the
player, not by its name. You can achieve that by passing a I<block> to the
sort method, which transforms the array elements (which are the names of
players) to the thing you want to sort by. The array items are passed in
through the I<topic variable> C<$_>.

You have seen blocks before: The C<for> loop worked on a block
C<< -> $line { ... } >>, the C<if> statement worked on the blocks
C<{ %games{$p1}++ }> and C<{ %games{$p1} }>. A block is just a piece of normal
Perl 6 code, optionally with a signature (the C<< -> $line >> part).
More about that in the next chapter. (TODO: write that)

So the simplest way to sort the players by score would be
C<@names.sort({ %games{$_} })>, which sorts by number of won games. However
Ana and Dave have both won two games, and that simple sort doesn't account for
the number of won sets, which is the secondary criterion to decide who has won
the tournament.

When two array items are transformed to the same number, C<sort> leaves their
relative order unchanged. Computer scientists call that a I<stable> sort. We
can use this fact to achieve our goal by two sorting twice: first sorting by
the number of won sets (the secondary criterion), then by the number of won
games.

=for vim: spell
{'fraction_non_alphanumeric': 0.1012649931179131, 'fraction_numerical': 0.011470144851543554, 'mean_word_length': 3.3606744784224065, 'pattern_counts': {'":': 0, '<': 187, '<?xml version=': 0, '>': 204, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3985431', 'n_tokens_mistral': 4859, 'n_tokens_neox': 4645, 'n_words': 2422}