<commit_before># Copyright (C) 2007, The Perl Foundation.
# $Id$

=head1 NAME

docs/pdds/pdd19_pir.pod - Parrot Intermediate Representation

=head1 VERSION

$Revision$


=head1 ABSTRACT

This document outlines the architecture and core syntax of Parrot
Intermediate Representation (PIR).

=head1 DESCRIPTION

PIR is a stable, middle-level language intended both as a target for the
generated output from high-level language compilers, and for human use
developing core features and extensions for Parrot.

=head2 Basic Syntax

A valid PIR program consists of a sequence of statements, directives, comments
and empty lines.

=head3 Statements

A statement starts with an optional label, contains an instruction, and is
terminated by a newline (<NL>). Each statement must be on its own line.

  [label:] [instruction] <NL>

An instruction may be either a low-level opcode or a higher-level PIR
operation, such as a subroutine call, a method call, a directive, or PIR
syntactic sugar.

=head3 Directives

A directive provides information for the PIR compiler that is outside the
normal flow of executable statements. Directives are all prefixed with a ".",
as in C<.local> or C<.sub>.

=head3 Comments

Comments start with C<#> and last until the following newline. PIR also allows
comments in Pod format. Comments, Pod content, and empty lines are ignored.

=head3 Identifiers

Identifiers start with a letter or underscore, then may contain additionally
letters, digits, and underscores. Identifiers don't have any limit on length
at the moment, but some sane-but-generous length limit may be imposed in the
future (256 chars, 1024 chars?). The following examples are all valid
identifiers.

    a
    _a
    A42

Opcode names are not reserved words in PIR, and may be used as variable names.
For example, you can define a local variable named C<print>.  [See RT #24251]
Note that by using an opcode name as a local variable name, the variable will
I<hide> the opcode name, effectively making the opcode unusable.

In contrast to opcode names, PIR keywords I<are> reserved, and cannot be used
as identifiers. Some opcode names are, in fact, PIR keywords, which therefore
cannot be used as identifiers.

The following are PIR keywords, and cannot be used as identifiers:

 goto      if       int         null
 num       pmc      string      unless


{{ NOTE: compilers/pirc does not have any reserved words; all keywords and ops
   can be used as identifiers.
}}

=head3 Labels

A label declaration consists of a label name followed by a colon. A label name
conforms to the standard requirements for identifiers. A label declaration may
occur at the start of a statement, or stand alone on a line, but always within
a subroutine.

A reference to a label consists of only the label name, and is generally used
as an argument to an instruction or directive.

A PIR label is accessible only in the subroutine where it's defined. A label
name must be unique within a subroutine, but it can be reused in other
subroutines.

  goto label1
     ...
  label1:

=head3 Registers and Variables

There are two ways of referencing Parrot's registers. The first is
through named local variables declared with C<.local>.

  .local pmc foo

The type of a named variable can be C<int>, C<num>, C<string> or C<pmc>,
corresponding to the types of registers. No other types are used. [See
RT#42769]

The second way of referencing a register is through a register variable
C<$In>, C<$Sn>, C<$Nn>, or C<$Pn>. The capital letter indicates the type
of the register (integer, string, number, or PMC). I<n> consists of
digit(s) only. There is no limit on the size of I<n>. There is no direct
correspondence between the value of I<n> and the position of the
register in the register set, C<$P42> may be stored in the zeroth PMC
register, if it is the only register in the subroutine.

{{DEPRECATION NOTE: PIR will no longer support the old PASM-style syntax
for registers without dollar signs: C<In>, C<Sn>, C<Nn>, C<Pn>. RT#57638}}

=head2 Constants

Constants may be used in place of registers or variables. A constant is not
allowed on the left side of an assignment, or in any other context where the
variable would be modified.

=over 4

=item 'single-quoted string constant'

Are delimited by single-quotes (C<'>). They are taken to be ASCII encoded. No
escape sequences are processed.

=item "double-quoted string constants"

Are delimited by double-quotes (C<">). A C<"> inside a string must be escaped
by C<\>. The default encoding for a double-quoted string constant is 7-bit
ASCII, other character sets and encodings must be marked explicitly using a
charset or encoding flag.

=item <<"heredoc",  <<'heredoc'

Heredocs work like single or double quoted strings. All lines up to
the terminating delimiter are slurped into the string. The delimiter
has to be on its own line, at the beginning of the line and with no
trailing whitespace.

Assignment of a heredoc:

  $S0 = <<"EOS"
  ...
 EOS

A heredoc as an argument:

  function(<<"END_OF_HERE", arg)
  ...
 END_OF_HERE

  .return(<<'EOS')
  ...
 EOS

  .yield(<<'EOS')
  ...
 EOS

You may have multiple heredocs within a single statement or directive:

   function(<<'INPUT', <<'OUTPUT', 'some test')
   ...
 INPUT
   ...
 OUTPUT

=item charset:"string constant"

Like above with a character set attached to the string. Valid character
sets are currently: C<ascii> (the default), C<binary>, C<unicode>
(with UTF-8 as the default encoding), and C<iso-8859-1>.

=back

=head2 String escape sequences

Inside double-quoted strings the following escape sequences are processed.

  \xhh        1..2 hex digits
  \ooo        1..3 oct digits
  \cX         control char X
  \x{h..h}    1..8 hex digits
  \uhhhh      4 hex digits
  \Uhhhhhhhh  8 hex digits
  \a, \b, \t, \n, \v, \f, \r, \e, \\, \"

=over 4

=item encoding:charset:"string constant"

Like above with an extra encoding attached to the string. For example:

  set S0, utf8:unicode:"«"

The encoding and charset are attached to the string constant, and
adopted by any string container the constant is assigned to.

The standard escape sequences are honored within strings with an
alternate encoding, so in the example above, you can include a
particular Unicode character as either a literal sequence of bytes, or
as an escape sequence.

=item numeric constants

Both integers (C<42>) and numbers (C<3.14159>) may appear as constants.
C<0x> and C<0b> denote hex and binary constants respectively.

=back

=head2 Directives

=over 4

=item .local <type> <identifier> [:unique_reg]

Define a local name I<identifier> within a subroutine with the given
I<type>. You can define multiple identifiers of the same type by
separating them with commas:

  .local int i, j

The optional C<:unique_reg> modifier will force the register allocator to
associate the identifier with a unique register for the duration of the
subroutine. If the register allocator is thought of as an optimization
tool for allowing fewer registers to be used in a register frame by reusing
unused registers, then the C<:unique_reg> directive forces this
optimization to be turned off. This can be important in a number of
situations:

=over 4

=item * When a subroutine has a small fixed number of registers

=item * When a named variable or named register is used throughout the entire
subroutine

=item * When a reference needs to be made to a register

=back

=item .lex <string constant>, <reg>

Declare a lexical variable that is an alias for a PMC register. For example,
given this preamble:

    .lex '$a', $P0
    $P1 = new 'Integer'

    These two opcodes have an identical effect:

    $P0 = $P1
    store_lex '$a', $P1

    And these two opcodes also have an identical effect:

    $P1 = $P0
    $P1 = find_lex '$a'

=item .const <type> <identifier> = <const>

Define a constant named I<identifier> of type I<type> and assign value
I<const> to it. The I<type> may be either an integer value or a string
constant. The constant is stored in the constant table of the current
bytecode file.

=item .globalconst <type> <identifier> = <const>

As C<.const> above, but the defined constant is globally accessible.

=item .sub

  .sub <identifier> [:<flag> ...]
  .sub <quoted string> [:<flag> ...]

Define a subroutine. All code in a PIR source file must be defined in a
subroutine. See the section L<Subroutine flags> for available flags.
Optional flags are a list of I<flag>, separated by  spaces.

The name of the sub may be either a bare identifier or a quoted string
constant. Bare identifiers must be valid PIR identifiers (see L<Identifiers>
above), but string sub names can contain any characters, including characters
from different character sets (see L<Constants> above).

Always paired with C<.end>.

=item .end

End a subroutine. Always paired with C<.sub>.

=item .namespace [ <identifier> ; <identifier> ]

   .namespace [ <key>? ]

   key: <identifier> [';' <identifier>]*

Defines the namespace from this point onwards.  By default the program is not
in any namespace.  If you specify more than one, separated by semicolons, it
creates nested namespaces, by storing the inner namespace object in the outer
namespace's global pad.

You can specify the root namespace by using empty brackets, such as:

    .namespace [ ]

The brackets are not optional, although the key inside them is.

=item .loadlib 'lib_name'

Load the given library at compile time, that is, as soon that line is
parsed.  See also the C<loadlib> opcode, which does the same at run time.

A library loaded this way is also available at runtime, as if it has been
loaded again in C<:load>, so there is no need to call C<loadlib> at runtime.

=item .HLL <hll_name>

Define the HLL for the current file. Takes one string constant, the name
of the HLL.

=item .HLL_map <core_type> = <user_type> [unimplemented: RT #57430]

Whenever Parrot has to create PMCs inside C code on behalf of the running
user program, it consults the current type mapping for the executing HLL
and creates a PMC of type I<user_type> instead of I<core_type>, if such
a mapping is defined. I<core_type> and I<user_type> may be any valid string
constant.

For example, with this code snippet:

  .loadlib 'dynlexpad'

  .HLL 'Foo'
  .HLL_map 'LexPad' = 'DynLexPad'

  .sub main :main
    ...

all subroutines for language I<Foo> would use a dynamic lexpad pmc.

=item .line <integer>, <string>

Set the line number and filename to the value specified. This is useful in
case the PIR code is generated from some source file, and error messages
should print the source file, not the line number and filename of the
generated file.

{{ DEPRECATION NOTE: was C<<#line <integer> <string>>>. See [RT#45857],
[RT#43269], and [RT#47141]. }}

=item .namespace <identifier> [deprecated: See RT #48737]

{{ DEPRECATION NOTE: this variation of C<.namespace> and
C<.endnamespace> are deprecated.  They were a hackish attempt at
implementing scopes in Parrot, but didn't actually turn out to be
useful.}}<commit_msg>[pdd] Document .annotate and .file and update .line directives in the PIR PDD.<commit_after># Copyright (C) 2007, The Perl Foundation.
# $Id$

=head1 NAME

docs/pdds/pdd19_pir.pod - Parrot Intermediate Representation

=head1 VERSION

$Revision$


=head1 ABSTRACT

This document outlines the architecture and core syntax of Parrot
Intermediate Representation (PIR).

=head1 DESCRIPTION

PIR is a stable, middle-level language intended both as a target for the
generated output from high-level language compilers, and for human use
developing core features and extensions for Parrot.

=head2 Basic Syntax

A valid PIR program consists of a sequence of statements, directives, comments
and empty lines.

=head3 Statements

A statement starts with an optional label, contains an instruction, and is
terminated by a newline (<NL>). Each statement must be on its own line.

  [label:] [instruction] <NL>

An instruction may be either a low-level opcode or a higher-level PIR
operation, such as a subroutine call, a method call, a directive, or PIR
syntactic sugar.

=head3 Directives

A directive provides information for the PIR compiler that is outside the
normal flow of executable statements. Directives are all prefixed with a ".",
as in C<.local> or C<.sub>.

=head3 Comments

Comments start with C<#> and last until the following newline. PIR also allows
comments in Pod format. Comments, Pod content, and empty lines are ignored.

=head3 Identifiers

Identifiers start with a letter or underscore, then may contain additionally
letters, digits, and underscores. Identifiers don't have any limit on length
at the moment, but some sane-but-generous length limit may be imposed in the
future (256 chars, 1024 chars?). The following examples are all valid
identifiers.

    a
    _a
    A42

Opcode names are not reserved words in PIR, and may be used as variable names.
For example, you can define a local variable named C<print>.  [See RT #24251]
Note that by using an opcode name as a local variable name, the variable will
I<hide> the opcode name, effectively making the opcode unusable.

In contrast to opcode names, PIR keywords I<are> reserved, and cannot be used
as identifiers. Some opcode names are, in fact, PIR keywords, which therefore
cannot be used as identifiers.

The following are PIR keywords, and cannot be used as identifiers:

 goto      if       int         null
 num       pmc      string      unless


{{ NOTE: compilers/pirc does not have any reserved words; all keywords and ops
   can be used as identifiers.
}}

=head3 Labels

A label declaration consists of a label name followed by a colon. A label name
conforms to the standard requirements for identifiers. A label declaration may
occur at the start of a statement, or stand alone on a line, but always within
a subroutine.

A reference to a label consists of only the label name, and is generally used
as an argument to an instruction or directive.

A PIR label is accessible only in the subroutine where it's defined. A label
name must be unique within a subroutine, but it can be reused in other
subroutines.

  goto label1
     ...
  label1:

=head3 Registers and Variables

There are two ways of referencing Parrot's registers. The first is
through named local variables declared with C<.local>.

  .local pmc foo

The type of a named variable can be C<int>, C<num>, C<string> or C<pmc>,
corresponding to the types of registers. No other types are used. [See
RT#42769]

The second way of referencing a register is through a register variable
C<$In>, C<$Sn>, C<$Nn>, or C<$Pn>. The capital letter indicates the type
of the register (integer, string, number, or PMC). I<n> consists of
digit(s) only. There is no limit on the size of I<n>. There is no direct
correspondence between the value of I<n> and the position of the
register in the register set, C<$P42> may be stored in the zeroth PMC
register, if it is the only register in the subroutine.

{{DEPRECATION NOTE: PIR will no longer support the old PASM-style syntax
for registers without dollar signs: C<In>, C<Sn>, C<Nn>, C<Pn>. RT#57638}}

=head2 Constants

Constants may be used in place of registers or variables. A constant is not
allowed on the left side of an assignment, or in any other context where the
variable would be modified.

=over 4

=item 'single-quoted string constant'

Are delimited by single-quotes (C<'>). They are taken to be ASCII encoded. No
escape sequences are processed.

=item "double-quoted string constants"

Are delimited by double-quotes (C<">). A C<"> inside a string must be escaped
by C<\>. The default encoding for a double-quoted string constant is 7-bit
ASCII, other character sets and encodings must be marked explicitly using a
charset or encoding flag.

=item <<"heredoc",  <<'heredoc'

Heredocs work like single or double quoted strings. All lines up to
the terminating delimiter are slurped into the string. The delimiter
has to be on its own line, at the beginning of the line and with no
trailing whitespace.

Assignment of a heredoc:

  $S0 = <<"EOS"
  ...
 EOS

A heredoc as an argument:

  function(<<"END_OF_HERE", arg)
  ...
 END_OF_HERE

  .return(<<'EOS')
  ...
 EOS

  .yield(<<'EOS')
  ...
 EOS

You may have multiple heredocs within a single statement or directive:

   function(<<'INPUT', <<'OUTPUT', 'some test')
   ...
 INPUT
   ...
 OUTPUT

=item charset:"string constant"

Like above with a character set attached to the string. Valid character
sets are currently: C<ascii> (the default), C<binary>, C<unicode>
(with UTF-8 as the default encoding), and C<iso-8859-1>.

=back

=head2 String escape sequences

Inside double-quoted strings the following escape sequences are processed.

  \xhh        1..2 hex digits
  \ooo        1..3 oct digits
  \cX         control char X
  \x{h..h}    1..8 hex digits
  \uhhhh      4 hex digits
  \Uhhhhhhhh  8 hex digits
  \a, \b, \t, \n, \v, \f, \r, \e, \\, \"

=over 4

=item encoding:charset:"string constant"

Like above with an extra encoding attached to the string. For example:

  set S0, utf8:unicode:"«"

The encoding and charset are attached to the string constant, and
adopted by any string container the constant is assigned to.

The standard escape sequences are honored within strings with an
alternate encoding, so in the example above, you can include a
particular Unicode character as either a literal sequence of bytes, or
as an escape sequence.

=item numeric constants

Both integers (C<42>) and numbers (C<3.14159>) may appear as constants.
C<0x> and C<0b> denote hex and binary constants respectively.

=back

=head2 Directives

=over 4

=item .local <type> <identifier> [:unique_reg]

Define a local name I<identifier> within a subroutine with the given
I<type>. You can define multiple identifiers of the same type by
separating them with commas:

  .local int i, j

The optional C<:unique_reg> modifier will force the register allocator to
associate the identifier with a unique register for the duration of the
subroutine. If the register allocator is thought of as an optimization
tool for allowing fewer registers to be used in a register frame by reusing
unused registers, then the C<:unique_reg> directive forces this
optimization to be turned off. This can be important in a number of
situations:

=over 4

=item * When a subroutine has a small fixed number of registers

=item * When a named variable or named register is used throughout the entire
subroutine

=item * When a reference needs to be made to a register

=back

=item .lex <string constant>, <reg>

Declare a lexical variable that is an alias for a PMC register. For example,
given this preamble:

    .lex '$a', $P0
    $P1 = new 'Integer'

    These two opcodes have an identical effect:

    $P0 = $P1
    store_lex '$a', $P1

    And these two opcodes also have an identical effect:

    $P1 = $P0
    $P1 = find_lex '$a'

=item .const <type> <identifier> = <const>

Define a constant named I<identifier> of type I<type> and assign value
I<const> to it. The I<type> may be either an integer value or a string
constant. The constant is stored in the constant table of the current
bytecode file.

=item .globalconst <type> <identifier> = <const>

As C<.const> above, but the defined constant is globally accessible.

=item .sub

  .sub <identifier> [:<flag> ...]
  .sub <quoted string> [:<flag> ...]

Define a subroutine. All code in a PIR source file must be defined in a
subroutine. See the section L<Subroutine flags> for available flags.
Optional flags are a list of I<flag>, separated by  spaces.

The name of the sub may be either a bare identifier or a quoted string
constant. Bare identifiers must be valid PIR identifiers (see L<Identifiers>
above), but string sub names can contain any characters, including characters
from different character sets (see L<Constants> above).

Always paired with C<.end>.

=item .end

End a subroutine. Always paired with C<.sub>.

=item .namespace [ <identifier> ; <identifier> ]

   .namespace [ <key>? ]

   key: <identifier> [';' <identifier>]*

Defines the namespace from this point onwards.  By default the program is not
in any namespace.  If you specify more than one, separated by semicolons, it
creates nested namespaces, by storing the inner namespace object in the outer
namespace's global pad.

You can specify the root namespace by using empty brackets, such as:

    .namespace [ ]

The brackets are not optional, although the key inside them is.

=item .loadlib 'lib_name'

Load the given library at compile time, that is, as soon that line is
parsed.  See also the C<loadlib> opcode, which does the same at run time.

A library loaded this way is also available at runtime, as if it has been
loaded again in C<:load>, so there is no need to call C<loadlib> at runtime.

=item .HLL <hll_name>

Define the HLL for the current file. Takes one string constant, the name
of the HLL.

=item .HLL_map <core_type> = <user_type> [unimplemented: RT #57430]

Whenever Parrot has to create PMCs inside C code on behalf of the running
user program, it consults the current type mapping for the executing HLL
and creates a PMC of type I<user_type> instead of I<core_type>, if such
a mapping is defined. I<core_type> and I<user_type> may be any valid string
constant.

For example, with this code snippet:

  .loadlib 'dynlexpad'

  .HLL 'Foo'
  .HLL_map 'LexPad' = 'DynLexPad'

  .sub main :main
    ...

all subroutines for language I<Foo> would use a dynamic lexpad pmc.

=item .line <integer>

Set the current PIR line number to the value specified. This is useful in
case the PIR code is generated from some source PIR files, and error messages
should print the source file's line number, not the line number of the
generated file. Note that line numbers increment per line of PIR; if you
are trying to store High Level Language debug information, you should instead
be using the C<.annotate> directive.

=item .file <quoted_string>

Set the current PIR file name to the value specified. This is useful in case
the PIR code is generated from some source PIR files, and error messages
should print the source file's name, not the name of the generated file.

{{ DEPRECATION NOTE: was C<<#line <integer> <string>>>. See [RT#45857],
[RT#43269], and [RT#47141]. }}

=item .annotate <key> <value>

Makes an entry in the bytecode annotations table. This is used to store high
level language debug information. Examples:

  .annotate "file" "aardvark.p6"
  .annotate "line" 5
  .annotate "column" 24

An annotation stays in effect until the next annotation with the same key or
the end of the current compilation unit (that is, if you use a tool such as
C<pbc_merge> to link multiple bytecode files, then annotations will not spill
over from one mergee's bytecode to another).

One annotation covers many PIR instructions. If the result of compiling one
line of HLL code is 15 lines of PIR, you only need to emit one annotation
before the first of those 15 lines to set the line number.

  .annotate "line" 42

The key must always be a quoted string. The value may be an integer, a number
or a quoted string. Note that integer values are stored most compactly; should
you instead of the above annotate directive emit:

  .annotate "line" "42"

Then you're asking for a bigger bytecode file as a result.

=item .namespace <identifier> [deprecated: See RT #48737]

{{ DEPRECATION NOTE: this variation of C<.namespace> and
C<.endnamespace> are deprecated.  They were a hackish attempt at
implementing scopes in Parrot, but didn't actually turn out to be
useful.}}
{'fraction_non_alphanumeric': 0.06902699949057566, 'fraction_numerical': 0.01010358295126507, 'mean_word_length': 3.9118015012510425, 'pattern_counts': {'":': 0, '<': 221, '<?xml version=': 0, '>': 189, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5751480', 'n_tokens_mistral': 7094, 'n_tokens_neox': 6688, 'n_words': 3603}