<commit_before>=pod

=head0 Parrot Intermediate Representation

Z<CHP-3>

X<Parrot Intermediate Representation;;(see PIR)>
X<PIR (Parrot intermediate representation)>
The Parrot intermediate representation (PIR) is an overlay on top of the
Parrot assembly language (PASM). PIR provides some simplifications and
high-level constructs to simplify programming tasks above what an
assembly language like PASM typically would have. It has many
high-level features that will be familiar to programmers, but
it still isn't considered to be a high-level language by itself. In fact,
the Parrot developers specifically want to keep in that way for a
number of reasons. PASM, the Parrot assembly language on which PIR is
based, is discussed in more detail in A<CHP-5>Chapter 5.

X<.pir files> As a convention, files containing pure PIR code generally
have a F<.pir> extension. Parrot will treat a file with any other
extension, besides F<.pasm> as a PIR file in mixed mode. This means that
the file can contain PIR and PASM code interchangably, with a few caveats
that will be discussed later.

X<PIR (Parrot intermediate representation);documentation>
PIR is well documented, both in traditional documentation and in
instructional code examples. The documentation for the PIR compiler IMCC
in F<docs/imcc/> or the project documentation in F<docs/> are good
sources for information about the current syntax, semantics, and
implementation. The other PIR compiler, PIRC, has its own documentation
that is slowly maturing. This is a useful source of information too. The
test suite in F<t/compilers/imcc/> shows examples of proper working code. In fact,
the test suite is the definitive PIR resource, because it shows how PIR
I<actually works>, even when the documentation may be out of date.

=head1 Statements

Z<CHP-3-SECT-1>

X<statements (PIR)>
X<PIR (Parrot intermediate representation);statements>
The syntax of statements in PIR is much more flexible then is commonly
found in assembly languages, but is more rigid and "close to the machine"
then some higher-level languages like C are. PIR has a very close
relationship with the Parrot assembly language, PASM. All PASM instructions
are valid PIR instructions. PIR does add some extra syntactic options to
help improve readability and programmability, however. The statement
delimiter for both PIR and PASM is a newline C<\n>. Each statement has to
be on its own line N<This isn't entirely true when you consider things like
macros and heredocs, but we'll tackle those issues when we come to them.>,
but empty whitespace lines between statements are okay. Statements may also
start with a label, for use with jumps and branches. Comments are marked by
a hash sign (C<#>), and continue until the end of the line. POD blocks may
be used for multi-line documentation.

To help with readability, PIR has some high-level constructs, including
symbol operators:

  $I1 = 5                       # set $I1, 5

named variables:

  count = 5

and complex statements built from multiple keywords and symbol
operators:

  if $I1 <= 5 goto LABEL        # le $I1, 5, LABEL

We will get into all of these in more detail as we go. Notice that PIR
does not, and will not, have high-level looping structures like C<while>
or C<for> loops and C<if>/C<then>/C<else> branch structures. Because of
these omissions PIR can become a little bit messy and unweildy for large
programs. Luckily, there are a large group of high-level languages (HLL)
that can be used to program Parrot instead. PIR is used primarily to
write the compilers and libraries for these languages.

=head1 Variables and Constants

Z<CHP-3-SECT-2>

=head2 Parrot Registers

Z<CHP-3-SECT-2.1>

PIR code has a variety of ways to store values while you work with
them. Actually, the only place to store values is in a Parrot register,
but there are multiple ways to work with these registers. Parrot's
register names always start with a dollar sign, followed by a single
character that shows whether it is an integer (I), numeric (N), string
(S), or PMC (P) register, and then the number of the register:

  $S0 = "Hello, Polly.\n"
  print $S0

You can have as many registers of each type as you need, Parrot will
automatically allocate new ones for you. The process is transparent, and
programmers should never have to worry about it.

Parrot registers are allocated in a linear array, and register numbers
are indices into this array. Having more registers means Parrot must
allocate more storage space for them, which can decrease memory efficiency
and register allocation/fetch performance. In general, it's better to
keep the number of registers small. However, the number of the register
does not necessarily correspond to the actual storage location where the
register data is held. A memory allocator unit translates register names
in the form "$S0" into an actual fixed memory location. This allocator
can also help to optimize register usage so that existing registers are
reused instead of allocating new ones in memory. The short version is that
the programmer should never have to worry about register allocation, and
should feel free to use as many as she wants. As with any system, it's
a good idea to be mindful of the things that might impact performance
anyway.

=head2 Constants

X<constants (PIR)>
X<PIR (Parrot intermediate representation);constants>
X<strings;in PIR>
Parrot has four primary data types: integers, floating-point numbers,
strings, and PMCs. Integers and floating-point numbers can be specified
in the code with numeric constants.

  $I0 = 42       # Integers are regular numeric constants
  $I1 = -1       # They can be negative or positive
  $I2 = 0xA5     # They can also be hexadecimal
  $I3 = 046      # ...or octal

  $N0 = 3.14     # Numbers can have a decimal point
  $N1 = 4        # ...or they don't
  $N2 = -1.2e+4  # Numbers can also use scientific notation.

String literals are enclosed in single or double-quotes:

  $S0 = "This is a valid literal string"
  $S1 = 'This is also a valid literal string'

Strings in double-quotes accept all sorts of escape sequences using
backslashes. Strings in single-quotes only allow escapes for nested
quotes:

  $S0 = "This string is \n on two lines"
  $S0 = 'This is a \n one-line string with a slash in it'

Or, if you need more flexibility, you can use a heredoc:

  $S2 = << "End_Token"

  This is a multi-line string literal. Notice that
  it doesn't use quotation marks. The string continues
  until the ending token (the thing in quotes next to
  the << above) is found.

  End_Token

=head2 Named Variables

Z<CHP-3-SECT-2.3>

X<named variables (PIR)>
X<PIR (Parrot intermediate representation);named variables>
Calling a value "$S0" isn't very descriptive, and usually it's a lot
nicer to be able to refer to values using a helpful name. For this
reason Parrot allows registers to be given temporary variable names to
use instead. These named variables can be used anywhere a register
would be used normally  N<...because they actually are registers, but
with fancier names>. They're declared with the C<.local> statement
which requires a variable type and a name:

  .local string hello
  set hello, "Hello, Polly.\n"
  print hello

This snippet defines a string variable named C<hello>, assigns it the
value "Hello, Polly.\n", and then prints the value.

X<types;variable (PIR)><commit_msg>[Book] Add some missing information about encoding: and charset: in string literals<commit_after>=pod

=head0 Parrot Intermediate Representation

Z<CHP-3>

X<Parrot Intermediate Representation;;(see PIR)>
X<PIR (Parrot intermediate representation)>
The Parrot intermediate representation (PIR) is an overlay on top of the
Parrot assembly language (PASM). PIR provides some simplifications and
high-level constructs to simplify programming tasks above what an
assembly language like PASM typically would have. It has many
high-level features that will be familiar to programmers, but
it still isn't considered to be a high-level language by itself. In fact,
the Parrot developers specifically want to keep in that way for a
number of reasons. PASM, the Parrot assembly language on which PIR is
based, is discussed in more detail in A<CHP-5>Chapter 5.

X<.pir files> As a convention, files containing pure PIR code generally
have a F<.pir> extension. Parrot will treat a file with any other
extension, besides F<.pasm> as a PIR file in mixed mode. This means that
the file can contain PIR and PASM code interchangably, with a few caveats
that will be discussed later.

X<PIR (Parrot intermediate representation);documentation>
PIR is well documented, both in traditional documentation and in
instructional code examples. The documentation for the PIR compiler IMCC
in F<docs/imcc/> or the project documentation in F<docs/> are good
sources for information about the current syntax, semantics, and
implementation. The other PIR compiler, PIRC, has its own documentation
that is slowly maturing. This is a useful source of information too. The
test suite in F<t/compilers/imcc/> shows examples of proper working code. In fact,
the test suite is the definitive PIR resource, because it shows how PIR
I<actually works>, even when the documentation may be out of date.

=head1 Statements

Z<CHP-3-SECT-1>

X<statements (PIR)>
X<PIR (Parrot intermediate representation);statements>
The syntax of statements in PIR is much more flexible then is commonly
found in assembly languages, but is more rigid and "close to the machine"
then some higher-level languages like C are. PIR has a very close
relationship with the Parrot assembly language, PASM. All PASM instructions
are valid PIR instructions. PIR does add some extra syntactic options to
help improve readability and programmability, however. The statement
delimiter for both PIR and PASM is a newline C<\n>. Each statement has to
be on its own line N<This isn't entirely true when you consider things like
macros and heredocs, but we'll tackle those issues when we come to them.>,
but empty whitespace lines between statements are okay. Statements may also
start with a label, for use with jumps and branches. Comments are marked by
a hash sign (C<#>), and continue until the end of the line. POD blocks may
be used for multi-line documentation.

To help with readability, PIR has some high-level constructs, including
symbol operators:

  $I1 = 5                       # set $I1, 5

named variables:

  count = 5

and complex statements built from multiple keywords and symbol
operators:

  if $I1 <= 5 goto LABEL        # le $I1, 5, LABEL

We will get into all of these in more detail as we go. Notice that PIR
does not, and will not, have high-level looping structures like C<while>
or C<for> loops and C<if>/C<then>/C<else> branch structures. Because of
these omissions PIR can become a little bit messy and unweildy for large
programs. Luckily, there are a large group of high-level languages (HLL)
that can be used to program Parrot instead. PIR is used primarily to
write the compilers and libraries for these languages.

=head1 Variables and Constants

Z<CHP-3-SECT-2>

=head2 Parrot Registers

Z<CHP-3-SECT-2.1>

PIR code has a variety of ways to store values while you work with
them. Actually, the only place to store values is in a Parrot register,
but there are multiple ways to work with these registers. Parrot's
register names always start with a dollar sign, followed by a single
character that shows whether it is an integer (I), numeric (N), string
(S), or PMC (P) register, and then the number of the register:

  $S0 = "Hello, Polly.\n"
  print $S0

You can have as many registers of each type as you need, Parrot will
automatically allocate new ones for you. The process is transparent, and
programmers should never have to worry about it.

Parrot registers are allocated in a linear array, and register numbers
are indices into this array. Having more registers means Parrot must
allocate more storage space for them, which can decrease memory efficiency
and register allocation/fetch performance. In general, it's better to
keep the number of registers small. However, the number of the register
does not necessarily correspond to the actual storage location where the
register data is held. A memory allocator unit translates register names
in the form "$S0" into an actual fixed memory location. This allocator
can also help to optimize register usage so that existing registers are
reused instead of allocating new ones in memory. The short version is that
the programmer should never have to worry about register allocation, and
should feel free to use as many as she wants. As with any system, it's
a good idea to be mindful of the things that might impact performance
anyway.

=head2 Constants

X<constants (PIR)>
X<PIR (Parrot intermediate representation);constants>
X<strings;in PIR>
Parrot has four primary data types: integers, floating-point numbers,
strings, and PMCs. Integers and floating-point numbers can be specified
in the code with numeric constants.

  $I0 = 42       # Integers are regular numeric constants
  $I1 = -1       # They can be negative or positive
  $I2 = 0xA5     # They can also be hexadecimal
  $I3 = 046      # ...or octal

  $N0 = 3.14     # Numbers can have a decimal point
  $N1 = 4        # ...or they don't
  $N2 = -1.2e+4  # Numbers can also use scientific notation.

String literals are enclosed in single or double-quotes:

  $S0 = "This is a valid literal string"
  $S1 = 'This is also a valid literal string'

Strings in double-quotes accept all sorts of escape sequences using
backslashes. Strings in single-quotes only allow escapes for nested
quotes:

  $S0 = "This string is \n on two lines"
  $S0 = 'This is a \n one-line string with a slash in it'

Or, if you need more flexibility, you can use a heredoc:

  $S2 = << "End_Token"

  This is a multi-line string literal. Notice that
  it doesn't use quotation marks. The string continues
  until the ending token (the thing in quotes next to
  the << above) is found.

  End_Token

=head3 Strings: Encodings and Charsets

Strings are complicated. It used to be that all that was needed was to
support the ASCII charset, which only contained a handful of common
symbols and English characters. Now we need to worry about character
encodings and charsets in order to make sense out of all the string data
in the world.

Parrot has a very flexible system for handling and manipulating strings.
Every string is associated with an encoding and a character set (charset).
The default for Parrot is 8-bit ASCII, which is simple to use and is almost
universally supported. However, support is built in to have other formats as
well.

String constants, like the ones we've seen above, can have an optional
prefix specifying the encoding and the charset to be used by the string.
Parrot will maintain these values internally, and will automatically convert
strings when necessary to preserve the information. String prefixes are
specified as C<encoding:charset:> at the front of the string. Here are some
examples:

  $S0 = utf8:unicode:"Hello UTF8 Unicode World!"
  $S1 = utf16:unicode:"Hello UTF16 Unicode World!"
  $S2 = ascii:"This is 8-bit ASCII"
  $S3 = binary:"This is treated as raw unformatted binary"

The C<binary:> encoding treats the string as a buffer of raw unformatted
binary data. It isn't really a "string" per se because binary data isn't
treated as if it contains any readable characters. These kinds of strings
are useful for library routines that return large amounts of binary data
that doesn't easily fit into any other primitive data type.

When two types of strings are combined together in some way, such as through
concatenation, they must both use the same character set an encoding.
Parrot will automatically upgrade one or both of the strings to use the next
highest compatible format, if they aren't equal. ASCII strings will
automatically upgrade to UTF-8 strings if needed, and UTF-8 will upgrade
to UTF-16. Handling and maintaining these data and conversions all happens
automatically inside Parrot, and you the programmer don't need to worry
about the details.

=head2 Named Variables

Z<CHP-3-SECT-2.3>

X<named variables (PIR)>
X<PIR (Parrot intermediate representation);named variables>
Calling a value "$S0" isn't very descriptive, and usually it's a lot
nicer to be able to refer to values using a helpful name. For this
reason Parrot allows registers to be given temporary variable names to
use instead. These named variables can be used anywhere a register
would be used normally  N<...because they actually are registers, but
with fancier names>. They're declared with the C<.local> statement
which requires a variable type and a name:

  .local string hello
  set hello, "Hello, Polly.\n"
  print hello

This snippet defines a string variable named C<hello>, assigns it the
value "Hello, Polly.\n", and then prints the value.

X<types;variable (PIR)>
{'fraction_non_alphanumeric': 0.05165289256198347, 'fraction_numerical': 0.007969303423848878, 'mean_word_length': 4.223866790009251, 'pattern_counts': {'":': 0, '<': 85, '<?xml version=': 0, '>': 75, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '3924694', 'n_tokens_mistral': 4670, 'n_tokens_neox': 4421, 'n_words': 2602}