<commit_before># Source Generators


## Summary

Source generators aim to enable _compile time metaprogramming_, that is, code that can be created
at compile time and added to the compilation. Source generators will be able to read the contents of 
the compilation before running, as well as access any _additional files_, enabling generators to
introspect both user C# code and generator specific files.

> **Note**: This proposal is separate from the [previous generator design](generators.md)

### High Level Design Goals

- Generators produce one or more strings that represent C# source code to be added to the compilation.
- Explicitly _additive_ only. Generators can add new source code to a compilation but may **not** modify existing user code.
- Can produce diagnostics. When unable to generate source, the generator can inform the user of the problem.
- May access _additional files_, that is, non-C# source texts.
- Run _un-ordered_, each generator will see the same input compilation, with no access to files created by other source generators.
- A user specifies the generators to run via list of assemblies, much like analyzers.

## Implementation

At the simplest level source generators are an implementation of `Microsoft.CodeAnalysis.ISourceGenerator`

```csharp
namespace Microsoft.CodeAnalysis
{
    public interface ISourceGenerator
    {
        void Initialize(InitializationContext context);
        void Execute(SourceGeneratorContext context);
    }
}
```

Generator implementations are defined in external assemblies passed to the compiler
using the same `-analyzer:` option used for diagnostic analyzers. Implementations are required to
be annotated with a `Microsoft.CodeAnalysis.GeneratorAttribute` attribute.

An assembly can contain a mix of diagnostic analyzers and source generators.
Since generators are loaded from external assemblies, a generator cannot be used to build
the assembly in which it is defined.

`ISourceGenerator` has an `Initialize` method that is called by the host (either the IDE or
the command-line compiler) exactly once. `Initialize` passes an instance of `InitializationContext`
which can be used by the generator to register a set of callbacks that affect how future generation
passes will occur.

The main generation pass occurs via the `Execute` method. `Execute` passes an instance of `SourceGeneratorContext`
that provides access to the `Compilation` and allows the generator to alter the `Compilation` by adding source
and reporting diagnostics.

```csharp
namespace Microsoft.CodeAnalysis
{
    public readonly struct SourceGeneratorContext
    {
        public ImmutableArray<AdditionalText> AdditionalFiles { get; }

        public CancellationToken CancellationToken { get; }

        public Compilation Compilation { get; }

        public ISyntaxReceiver? SyntaxReceiver { get; }

        public void ReportDiagnostic(Diagnostic diagnostic) { throw new NotImplementedException(); }

        public void AddSource(string fileNameHint, SourceText sourceText) { throw new NotImplementedException(); }
    }
}
```

It is assumed that some generators will want to generate more than one `SourceText`, for example in a 1:1 mapping
for additional files. The `fileNameHint` parameter of `AddSource` is intended to address this:

1. If the generated files are emitted to disk, having some ability to put some distinguishing text might be useful.
For example, if you have two `.resx` files, generating the files with simply names of `ResxGeneratedFile1.cs` and
`ResxGeneratedFile2.cs` wouldn't be terribly useful -- you'd want it to be something like
`ResxGeneratedFile-Strings.cs` and `ResxGeneratedFile-Icons.cs` if you had two `.resx` files
 named "Strings" and "Icons" respectively.<commit_msg>PR feedback<commit_after># Source Generators


## Summary

Source generators aim to enable _compile time metaprogramming_, that is, code that can be created
at compile time and added to the compilation. Source generators will be able to read the contents of 
the compilation before running, as well as access any _additional files_, enabling generators to
introspect both user C# code and generator specific files.

> **Note**: This proposal is separate from the [previous generator design](generators.md)

### High Level Design Goals

- Generators produce one or more strings that represent C# source code to be added to the compilation.
- Explicitly _additive_ only. Generators can add new source code to a compilation but may **not** modify existing user code.
- Can produce diagnostics. When unable to generate source, the generator can inform the user of the problem.
- May access _additional files_, that is, non-C# source texts.
- Run _un-ordered_, each generator will see the same input compilation, with no access to files created by other source generators.
- A user specifies the generators to run via list of assemblies, much like analyzers.

## Implementation

At the simplest level source generators are an implementation of `Microsoft.CodeAnalysis.ISourceGenerator`

```csharp
namespace Microsoft.CodeAnalysis
{
    public interface ISourceGenerator
    {
        void Initialize(InitializationContext context);
        void Execute(SourceGeneratorContext context);
    }
}
```

Generator implementations are defined in external assemblies passed to the compiler
using the same `-analyzer:` option used for diagnostic analyzers. Implementations are required to
be annotated with a `Microsoft.CodeAnalysis.GeneratorAttribute` attribute.

An assembly can contain a mix of diagnostic analyzers and source generators.
Since generators are loaded from external assemblies, a generator cannot be used to build
the assembly in which it is defined.

`ISourceGenerator` has an `Initialize` method that is called by the host (either the IDE or
the command-line compiler) exactly once. `Initialize` passes an instance of `InitializationContext`
which can be used by the generator to register a set of callbacks that affect how future generation
passes will occur.

The main generation pass occurs via the `Execute` method. `Execute` passes an instance of `SourceGeneratorContext`
that provides access to the current `Compilation` and allows the generator to alter the resulting output `Compilation`
by adding source and reporting diagnostics.

The generator is also able to access any `AnalyzerAdditionalFiles` passed to the compiler via the `AdditionalFiles`
collection, allowing for generation decisions to based on more than just the user's C# code.

```csharp
namespace Microsoft.CodeAnalysis
{
    public readonly struct SourceGeneratorContext
    {
        public ImmutableArray<AdditionalText> AdditionalFiles { get; }

        public CancellationToken CancellationToken { get; }

        public Compilation Compilation { get; }

        public ISyntaxReceiver? SyntaxReceiver { get; }

        public void ReportDiagnostic(Diagnostic diagnostic) { throw new NotImplementedException(); }

        public void AddSource(string fileNameHint, SourceText sourceText) { throw new NotImplementedException(); }
    }
}
```

It is assumed that some generators will want to generate more than one `SourceText`, for example in a 1:1 mapping
for additional files. The `fileNameHint` parameter of `AddSource` is intended to address this:

1. If the generated files are emitted to disk, having some ability to put some distinguishing text might be useful.
For example, if you have two `.resx` files, generating the files with simply names of `ResxGeneratedFile1.cs` and
`ResxGeneratedFile2.cs` wouldn't be terribly useful -- you'd want it to be something like
`ResxGeneratedFile-Strings.cs` and `ResxGeneratedFile-Icons.cs` if you had two `.resx` files
 named "Strings" and "Icons" respectively.
{'fraction_non_alphanumeric': 0.05288275506255643, 'fraction_numerical': 0.0012898232942086935, 'mean_word_length': 4.65985401459854, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '953649', 'n_tokens_mistral': 1879, 'n_tokens_neox': 1799, 'n_words': 984}