<commit_before># Markdig [![Build status](https://ci.appveyor.com/api/projects/status/hk391x8jcskxt1u8?svg=true)](https://ci.appveyor.com/project/xoofx/markdig) [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)

<img align="right" width="160px" height="160px" src="img/markdig.png">

Markdig is a fast, powerfull, [CommonMark](http://commonmark.org/) compliant, extensible Markdown processor for .NET.

> NOTE: The repository is under construction. There will be a dedicated website and proper documentation at some point!

You can **try Markdig online** and compare it to other implementations on [babelmark3](http://babelmark.github.io/) 

## Features

- **Very fast parser** (no-regexp), very lightweight in terms of GC pressure. See benchmarks
- **Abstract Syntax Tree**
- Converter to **HTML**
- Passing more than **600+ tests** from the latest [CommonMark specs](http://spec.commonmark.org/)
- Includes all the core elements of CommonMark:
  - including GFM fenced code blocks. 
- **Extensible** architecture
  - Even the core Markdown/CommonMark parsing is pluggable, so it allows to disable builtin Markdown/Commonmark parsing (e.g [Disable HTML parsing](https://github.com/lunet-io/markdig/blob/7964bd0160d4c18e4155127a4c863d61ebd8944a/src/Markdig/MarkdownExtensions.cs#L306)) or change behaviour (e.g change matching `#` of a headers with `@`)   
- Built-in with **20+ extensions**, including:
  - 2 kind of tables:
    - **Pipe tables** (inspired from Github tables and [PanDoc - Pipe Tables](http://pandoc.org/README.html#extension-pipe_tables))
    - **Grid tables** (inspired from [Pandoc - Grid Tables](http://pandoc.org/README.html#extension-grid_tables)) 
  - **Extra emphasis** (inspired from [Pandoc - Emphasis](http://pandoc.org/README.html#strikeout) and [Markdown-it](https://markdown-it.github.io/)) 
    - strike through `~~`,
    - Subscript `~`
    - Superscript `^` 
    - Inserted `++`
    - Marked `==`
  - **Special attributes** or attached HTML attributes (inspired from [PHP Markdown Extra - Special Attributes](https://michelf.ca/projects/php-markdown/extra/#spe-attr))
  - **Definition lists** (inspired from [PHP Markdown Extra - Definitions Lists](https://michelf.ca/projects/php-markdown/extra/#def-list))
  - **Footnotes** (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes))
  - **Auto-identifiers** for headings (similar to [Pandoc - Auto Identifiers](http://pandoc.org/README.html#extension-auto_identifiers))
  - **Task Lists** inspired from [Github Task lists](https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments).
  - **Extra bullet lists**, supporting alpha bullet `a.` `b.` and roman bullet (`i`, `ii`...etc.)
  - **Media support** for media url (youtube, vimeo, mp4...etc.) (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/embedded-audio-and-video/441))
  - **Abbreviations** (inspired from [PHP Markdown Extra - Abbreviations](https://michelf.ca/projects/php-markdown/extra/#abbr))
  - **Citation** text by enclosing `""...""` (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/referencing-creative-works-with-cite/892))
  - **Custom containers** similar to fenced code block `:::` for generating a proper `<div>...</div>` instead (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/custom-container-for-block-and-inline/2051))
  - **Figures** (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/image-tag-should-expand-to-figure-when-used-with-title/265/5))
  - **Footers** (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/syntax-for-footer/2070))
  - **Mathematics**/Latex extension by enclosing `$$` for block and `$` for inline math (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/mathematics-extension/457/31))
  - **Soft lines as hard lines**
  - **Emoji** support (inspired from [Markdown-it](https://markdown-it.github.io/))
  - **SmartyPants** (inspired from [Daring Fireball - SmartyPants](https://daringfireball.net/projects/smartypants/))
  - **Bootstrap** class (to output bootstrap class)
- Compatible with .NET 3.5, 4.0+ and .NET Core (`netstandard1.1+`)
	
## Download

Markdig is available as a NuGet package: [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)

## Usage

The main entry point for the API is the `Markdig.Markdown` class:

By default, without any options, Markdig is using the plain CommonMark parser:

```csharp
var result = Markdown.ToHtml("This is a text with some *emphasis*");
Console.WriteLine(result);   // prints: <p>This is a text with some <em>emphasis</em></p>
```

In order to activate most of all advanced extensions (except Emoji, SoftLine as HarLine and SmartyPants)

```csharp
// Configure the pipeline with all advanced extensions active
var pipeline = new MarkdownPipelineBuilder().UseAdvancedExtensions().Build();
var result = Markdown.ToHtml("This is a text with some *emphasis*", pipeline);
```

You can have a look at the [MarkdownExtensions](https://github.com/lunet-io/markdig/blob/master/src/Markdig/MarkdownExtensions.cs) that describes all actionable extensions (by modifying the MarkdownPipeline)

## License

This software is released under the [BSD-Clause 2 license](https://github.com/lunet-io/markdig/blob/master/license.txt).


## Benchmarking

This is an early preview of the benchmarking against various implementations:

**C implementations**:

- [cmark](https://github.com/jgm/cmark): Reference C implementation of CommonMark, no support for extensions
- [Moonshine](https://github.com/brandonc/moonshine): popular C Markdown processor

**.NET implementations**:

- [Markdig](https://github.com/lunet-io/markdig): itself
- [CommonMark.NET(master)](https://github.com/Knagis/CommonMark.NET): CommonMark implementation for .NET, no support for extensions, port of cmark
  - [CommonMark.NET(pipe_tables)](https://github.com/AMDL/CommonMark.NET/tree/pipe-tables): An evolution of CommonMark.NET, supports extensions, not released yet
- [MarkdownDeep](https://github.com/toptensoftware/markdowndeep) another .NET implementation
- [MarkdownSharp](https://github.com/Kiri-rin/markdownsharp): Open source C# implementation of Markdown processor, as featured on Stack Overflow, regexp based.
- [Marked.NET](https://github.com/T-Alex/MarkedNet) port of original [marked.js](https://github.com/chjj/marked) project

**JavaScript/V8 implementations**:

- [Strike.V8](https://github.com/SimonCropp/Strike) [marked.js](https://github.com/chjj/marked) running in Google V8 (not .NET based)

### Analysis of the results:

- Markdig is roughly **x100 times faster than MarkdownSharp**
- **Among the best in CPU**, Extremelly competitive and often faster than other implementations (not feature wise equivalent) 
- **15% to 30% less allocations** and GC pressure

Because Marked.NET and MarkdownSharp are way too slow, we couldn't include them in the following charts:

![BenchMark CPU Time](img/BenchmarkCPU.png)

![BenchMark Memory](img/BenchmarkMemory.png)


### Performance in x86:

```
BenchmarkDotNet-Dev=v0.9.7.0+
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-4770 CPU 3.40GHz, ProcessorCount=8
Frequency=3319351 ticks, Resolution=301.2637 ns, Timer=TSC
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE
JitModules=clrjit-v4.6.1080.0

Type=Program  Mode=SingleRun  LaunchCount=2
WarmupCount=2  TargetCount=10

                     Method |      Median |    StdDev |Scaled | Gen 0  | Gen 1 | Gen 2  |Bytes Allocated/Op |
--------------------------- |------------ |---------- |------ |------  |------ |------  |------------------ |
                    Markdig |   5.5316 ms | 0.0372 ms |  0.71 |  56.00 | 21.00 |  49.00 |      1,285,917.31 |
     CommonMark.NET(master) |   4.7035 ms | 0.0422 ms |  0.60 | 113.00 |  7.00 |  49.00 |      1,502,404.60 |
CommonMark.NET(pipe_tables) |   5.6164 ms | 0.0298 ms |  0.72 | 111.00 | 56.00 |  49.00 |      1,863,128.13 |
               MarkdownDeep |   7.8193 ms | 0.0334 ms |  1.00 | 120.00 | 56.00 |  49.00 |      1,884,854.85 |
                      cmark |   4.2698 ms | 0.1526 ms |  0.55 |      - |     - |      - |                NA |
                  Moonshine |   6.0929 ms | 0.1053 ms |  1.28 |      - |     - |      - |                NA |
                  Strike.V8 |  10.5895 ms | 0.0492 ms |  1.35 |      - |     - |      - |                NA |
                 Marked.NET | 207.3169 ms | 5.2628 ms | 26.51 |  0.00  |  0.00 |  0.00  |    303,125,228.65 |
              MarkdownSharp | 675.0185 ms | 2.8447 ms | 86.32 | 40.00  | 27.00 | 41.00  |      2,413,394.17 |
```<commit_msg>Fix typo<commit_after># Markdig [![Build status](https://ci.appveyor.com/api/projects/status/hk391x8jcskxt1u8?svg=true)](https://ci.appveyor.com/project/xoofx/markdig) [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)

<img align="right" width="160px" height="160px" src="img/markdig.png">

Markdig is a fast, powerfull, [CommonMark](http://commonmark.org/) compliant, extensible Markdown processor for .NET.

> NOTE: The repository is under construction. There will be a dedicated website and proper documentation at some point!

You can **try Markdig online** and compare it to other implementations on [babelmark3](http://babelmark.github.io/) 

## Features

- **Very fast parser** (no-regexp), very lightweight in terms of GC pressure. See benchmarks
- **Abstract Syntax Tree**
- Converter to **HTML**
- Passing more than **600+ tests** from the latest [CommonMark specs](http://spec.commonmark.org/)
- Includes all the core elements of CommonMark:
  - including GFM fenced code blocks. 
- **Extensible** architecture
  - Even the core Markdown/CommonMark parsing is pluggable, so it allows to disable builtin Markdown/Commonmark parsing (e.g [Disable HTML parsing](https://github.com/lunet-io/markdig/blob/7964bd0160d4c18e4155127a4c863d61ebd8944a/src/Markdig/MarkdownExtensions.cs#L306)) or change behaviour (e.g change matching `#` of a headers with `@`)   
- Built-in with **20+ extensions**, including:
  - 2 kind of tables:
    - **Pipe tables** (inspired from Github tables and [PanDoc - Pipe Tables](http://pandoc.org/README.html#extension-pipe_tables))
    - **Grid tables** (inspired from [Pandoc - Grid Tables](http://pandoc.org/README.html#extension-grid_tables)) 
  - **Extra emphasis** (inspired from [Pandoc - Emphasis](http://pandoc.org/README.html#strikeout) and [Markdown-it](https://markdown-it.github.io/)) 
    - strike through `~~`,
    - Subscript `~`
    - Superscript `^` 
    - Inserted `++`
    - Marked `==`
  - **Special attributes** or attached HTML attributes (inspired from [PHP Markdown Extra - Special Attributes](https://michelf.ca/projects/php-markdown/extra/#spe-attr))
  - **Definition lists** (inspired from [PHP Markdown Extra - Definitions Lists](https://michelf.ca/projects/php-markdown/extra/#def-list))
  - **Footnotes** (inspired from [PHP Markdown Extra - Footnotes](https://michelf.ca/projects/php-markdown/extra/#footnotes))
  - **Auto-identifiers** for headings (similar to [Pandoc - Auto Identifiers](http://pandoc.org/README.html#extension-auto_identifiers))
  - **Task Lists** inspired from [Github Task lists](https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments).
  - **Extra bullet lists**, supporting alpha bullet `a.` `b.` and roman bullet (`i`, `ii`...etc.)
  - **Media support** for media url (youtube, vimeo, mp4...etc.) (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/embedded-audio-and-video/441))
  - **Abbreviations** (inspired from [PHP Markdown Extra - Abbreviations](https://michelf.ca/projects/php-markdown/extra/#abbr))
  - **Citation** text by enclosing `""...""` (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/referencing-creative-works-with-cite/892))
  - **Custom containers** similar to fenced code block `:::` for generating a proper `<div>...</div>` instead (inspired by this [CommonMark discussion ](https://talk.commonmark.org/t/custom-container-for-block-and-inline/2051))
  - **Figures** (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/image-tag-should-expand-to-figure-when-used-with-title/265/5))
  - **Footers** (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/syntax-for-footer/2070))
  - **Mathematics**/Latex extension by enclosing `$$` for block and `$` for inline math (inspired from this [CommonMark discussion](https://talk.commonmark.org/t/mathematics-extension/457/31))
  - **Soft lines as hard lines**
  - **Emoji** support (inspired from [Markdown-it](https://markdown-it.github.io/))
  - **SmartyPants** (inspired from [Daring Fireball - SmartyPants](https://daringfireball.net/projects/smartypants/))
  - **Bootstrap** class (to output bootstrap class)
- Compatible with .NET 3.5, 4.0+ and .NET Core (`netstandard1.1+`)
	
## Download

Markdig is available as a NuGet package: [![NuGet](https://img.shields.io/nuget/v/Markdig.svg)](https://www.nuget.org/packages/Markdig/)

## Usage

The main entry point for the API is the `Markdig.Markdown` class:

By default, without any options, Markdig is using the plain CommonMark parser:

```csharp
var result = Markdown.ToHtml("This is a text with some *emphasis*");
Console.WriteLine(result);   // prints: <p>This is a text with some <em>emphasis</em></p>
```

In order to activate most of all advanced extensions (except Emoji, SoftLine as HarLine and SmartyPants)

```csharp
// Configure the pipeline with all advanced extensions active
var pipeline = new MarkdownPipelineBuilder().UseAdvancedExtensions().Build();
var result = Markdown.ToHtml("This is a text with some *emphasis*", pipeline);
```

You can have a look at the [MarkdownExtensions](https://github.com/lunet-io/markdig/blob/master/src/Markdig/MarkdownExtensions.cs) that describes all actionable extensions (by modifying the MarkdownPipeline)

## License

This software is released under the [BSD-Clause 2 license](https://github.com/lunet-io/markdig/blob/master/license.txt).


## Benchmarking

This is an early preview of the benchmarking against various implementations:

**C implementations**:

- [cmark](https://github.com/jgm/cmark): Reference C implementation of CommonMark, no support for extensions
- [Moonshine](https://github.com/brandonc/moonshine): popular C Markdown processor

**.NET implementations**:

- [Markdig](https://github.com/lunet-io/markdig): itself
- [CommonMark.NET(master)](https://github.com/Knagis/CommonMark.NET): CommonMark implementation for .NET, no support for extensions, port of cmark
  - [CommonMark.NET(pipe_tables)](https://github.com/AMDL/CommonMark.NET/tree/pipe-tables): An evolution of CommonMark.NET, supports extensions, not released yet
- [MarkdownDeep](https://github.com/toptensoftware/markdowndeep) another .NET implementation
- [MarkdownSharp](https://github.com/Kiri-rin/markdownsharp): Open source C# implementation of Markdown processor, as featured on Stack Overflow, regexp based.
- [Marked.NET](https://github.com/T-Alex/MarkedNet) port of original [marked.js](https://github.com/chjj/marked) project

**JavaScript/V8 implementations**:

- [Strike.V8](https://github.com/SimonCropp/Strike) [marked.js](https://github.com/chjj/marked) running in Google V8 (not .NET based)

### Analysis of the results:

- Markdig is roughly **x100 times faster than MarkdownSharp**
- **Among the best in CPU**, Extremelly competitive and often faster than other implementations (not feature wise equivalent) 
- **15% to 30% less allocations** and GC pressure

Because Marked.NET and MarkdownSharp are way too slow, we couldn't include them in the following charts:

![BenchMark CPU Time](img/BenchmarkCPU.png)

![BenchMark Memory](img/BenchmarkMemory.png)


### Performance for x86:

```
BenchmarkDotNet-Dev=v0.9.7.0+
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-4770 CPU 3.40GHz, ProcessorCount=8
Frequency=3319351 ticks, Resolution=301.2637 ns, Timer=TSC
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE
JitModules=clrjit-v4.6.1080.0

Type=Program  Mode=SingleRun  LaunchCount=2
WarmupCount=2  TargetCount=10

                     Method |      Median |    StdDev |Scaled | Gen 0  | Gen 1 | Gen 2  |Bytes Allocated/Op |
--------------------------- |------------ |---------- |------ |------  |------ |------  |------------------ |
                    Markdig |   5.5316 ms | 0.0372 ms |  0.71 |  56.00 | 21.00 |  49.00 |      1,285,917.31 |
     CommonMark.NET(master) |   4.7035 ms | 0.0422 ms |  0.60 | 113.00 |  7.00 |  49.00 |      1,502,404.60 |
CommonMark.NET(pipe_tables) |   5.6164 ms | 0.0298 ms |  0.72 | 111.00 | 56.00 |  49.00 |      1,863,128.13 |
               MarkdownDeep |   7.8193 ms | 0.0334 ms |  1.00 | 120.00 | 56.00 |  49.00 |      1,884,854.85 |
                      cmark |   4.2698 ms | 0.1526 ms |  0.55 |      - |     - |      - |                NA |
                  Moonshine |   6.0929 ms | 0.1053 ms |  1.28 |      - |     - |      - |                NA |
                  Strike.V8 |  10.5895 ms | 0.0492 ms |  1.35 |      - |     - |      - |                NA |
                 Marked.NET | 207.3169 ms | 5.2628 ms | 26.51 |  0.00  |  0.00 |  0.00  |    303,125,228.65 |
              MarkdownSharp | 675.0185 ms | 2.8447 ms | 86.32 | 40.00  | 27.00 | 41.00  |      2,413,394.17 |
```
{'fraction_non_alphanumeric': 0.15573302909298345, 'fraction_numerical': 0.04689104392470051, 'mean_word_length': 3.4997433264887063, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 19, 'https://': 68, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '160741', 'n_tokens_mistral': 6293, 'n_tokens_neox': 5560, 'n_words': 1439}