<commit_before># Advanced: Paragraph rule

## Readme

You have already read following document.

- [Creating Rules](./rule.md)

This tutorial describe that creating a rule to handle `Paragraph` nodes.

## Using Code Examples

You can see finished module in the tutorial.

- [textlint-rule-en-max-word-count](https://github.com/azu/textlint-rule-en-max-word-count "textlint-rule-en-max-word-count")

The rule that specify the maximum word count of a **sentence**.

## Using Library

These library are used in the module.

- [object-assign](https://www.npmjs.com/package/object-assign)
- [sentence-splitter](https://www.npmjs.com/package/sentence-splitter)
- [textlint-util-to-string](https://www.npmjs.com/package/textlint-util-to-string)
- [unist-util-map](https://www.npmjs.com/package/unist-util-map)
- [split-string-words](https://www.npmjs.com/package/split-string-words)


## Terms

- **Paragraph** is a type of [`TxtParentNode`](./txtnode.md).
    - Paragraph has `children` nodes.
- **Sentence** is not defined in textlint.
    - It has language dependency.
    - [textlint-rule-en-max-word-count](https://github.com/azu/textlint-rule-en-max-word-count "textlint-rule-en-max-word-count") is for English.
    - It is created from **Paragraph** text by [sentence-splitter](https://www.npmjs.com/package/sentence-splitter)
- **Word** is not defined in textlint.
    - It has language dependency.
    - [textlint-rule-en-max-word-count](https://github.com/azu/textlint-rule-en-max-word-count "textlint-rule-en-max-word-count") is for English.
    - It is created from **Sentence** text by [split-string-words](https://www.npmjs.com/package/split-string-words)

## 1. Start

To create a rule that specify the maximum **word** count of a **sentence**.

```js
// Default options
const defaultOptions = {
    // max count of words >
    max: 50
};
/**
 * @param {TextLintRuleContext} context
 * @param {Object} options
 */
export default function (context, options = {}) {
    const {Syntax, getSource, RuleError, report} = context;
    const maxWordCount = options.max ? options.max : defaultOptions.max;
    return {
        [Syntax.Paragraph](node){
            // node is Paragraph node
            // Paragraph contain `Code`, `Str`, `Strong` node etc...
        }
    };
}
```

### 2. Ignore `Code` node

`Paragraph` node contain `Code`, `Str`, `Strong` node etc...

```
This is text.
```

is described as `Paragraph.children = ["Str"]`

:information_source: Info:

- Please see [txtnode.md](./txtnode.md) for Abstract Syntax Tree details.
- Use [Markdown-to-AST demo](http://azu.github.io/markdown-to-ast/example/)

But, Following `Paragraph` node contain `Code` node. 

```
This text contain `var a = "string";` code.
```

How do you handle `Code` node?

1. ignore `Code`.
2. replace `Code` to dummy object that is a single **word**.

In this case, We select *Case 2*.

Replace `Code` to dummy object that is a single **word** Using [unist-util-map](https://github.com/azu/unist-util-map "unist-util-map").

```js
// Helper for creating new AST using map function
// https://github.com/azu/unist-util-map
// if you want to filter, use https://github.com/eush77/unist-util-filter
import map from "unist-util-map";
import ObjectAssign from "object-assign";
// Default options
const defaultOptions = {
    // max count of words >
    max: 50
};
/**
 * @param {TextLintRuleContext} context
 * @param {Object} options
 */
export default function (context, options = {}) {
    const {Syntax, getSource, RuleError, report} = context;
    const maxWordCount = options.max ? options.max : defaultOptions.max;
    return {
        [Syntax.Paragraph](node){
            // replace code with dummy code
            // if you want to filter(remove) code, use https://github.com/eush77/unist-util-filter
            const filteredNode = map(node, (node) => {
                if (node.type === Syntax.Code) {
                    // only change `value` to dummy
                    return ObjectAssign({}, node, {
                        value: "code"
                    });
                }
                return node;
            });
        }
    };
}
```

### 3. Get text of a Paragraph

We can get text from a node with **Source Map** using [textlint-util-to-string](https://www.npmjs.com/package/textlint-util-to-string).

Why does we use [textlint-util-to-string](https://www.npmjs.com/package/textlint-util-to-string)?

Because, we report via `context.report` original position of the text.

```js
// Helper for creating new AST using map function
// https://github.com/azu/unist-util-map
// if you want to filter, use https://github.com/eush77/unist-util-filter
import map from "unist-util-map";
// Helper for converting plain text from Syntax-ed text(markdown AST
// https://github.com/azu/textlint-util-to-string
import StringSource from "textlint-util-to-string";
import ObjectAssign from "object-assign";
// Default options
const defaultOptions = {
    // max count of words >
    max: 50
};
/**
 * @param {TextLintRuleContext} context
 * @param {Object} options
 */
export default function (context, options = {}) {
    const {Syntax, getSource, RuleError, report} = context;<commit_msg>docs(rule): fix typoe<commit_after># Advanced: Paragraph rule

## Readme

You have already read following document.

- [Creating Rules](./rule.md)

This tutorial describe that creating a rule to handle `Paragraph` nodes.

## Using Code Examples

You can see finished module in the tutorial.

- [textlint-rule-en-max-word-count](https://github.com/azu/textlint-rule-en-max-word-count "textlint-rule-en-max-word-count")

The rule that specify the maximum word count of a **sentence**.

## Using Library

These library are used in the module.

- [object-assign](https://www.npmjs.com/package/object-assign)
- [sentence-splitter](https://www.npmjs.com/package/sentence-splitter)
- [textlint-util-to-string](https://www.npmjs.com/package/textlint-util-to-string)
- [unist-util-map](https://www.npmjs.com/package/unist-util-map)
- [split-string-words](https://www.npmjs.com/package/split-string-words)


## Terms

- **Paragraph** is a type of [`TxtParentNode`](./txtnode.md).
    - Paragraph has `children` nodes.
- **Sentence** is not defined in textlint.
    - It has language dependency.
    - [textlint-rule-en-max-word-count](https://github.com/azu/textlint-rule-en-max-word-count "textlint-rule-en-max-word-count") is for English.
    - It is created from **Paragraph** text by [sentence-splitter](https://www.npmjs.com/package/sentence-splitter)
- **Word** is not defined in textlint.
    - It has language dependency.
    - [textlint-rule-en-max-word-count](https://github.com/azu/textlint-rule-en-max-word-count "textlint-rule-en-max-word-count") is for English.
    - It is created from **Sentence** text by [split-string-words](https://www.npmjs.com/package/split-string-words)

## 1. Start

To create a rule that specify the maximum **word** count of a **sentence**.

```js
// Default options
const defaultOptions = {
    // max count of words >
    max: 50
};
/**
 * @param {TextLintRuleContext} context
 * @param {Object} options
 */
export default function (context, options = {}) {
    const {Syntax, getSource, RuleError, report} = context;
    const maxWordCount = options.max ? options.max : defaultOptions.max;
    return {
        [Syntax.Paragraph](node){
            // node is Paragraph node
            // Paragraph contain `Code`, `Str`, `Strong` node etc...
        }
    };
}
```

### 2. Ignore `Code` node

`Paragraph` node contain `Code`, `Str`, `Strong` node etc...

```
This is text.
```

is described as `Paragraph.children = ["Str"]`

:information_source: Info:

- Please see [txtnode.md](./txtnode.md) for Abstract Syntax Tree details.
- Use [Markdown-to-AST demo](http://azu.github.io/markdown-to-ast/example/)

But, Following `Paragraph` node contain `Code` node. 

```
This text contain `var a = "string";` code.
```

How do you handle `Code` node?

1. ignore `Code`.
2. replace `Code` to dummy object that is a single **word**.

In this case, We select *Case 2*.

Replace `Code` to dummy object that is a single **word** Using [unist-util-map](https://github.com/azu/unist-util-map "unist-util-map").

```js
// Helper for creating new AST using map function
// https://github.com/azu/unist-util-map
// if you want to filter, use https://github.com/eush77/unist-util-filter
import map from "unist-util-map";
import ObjectAssign from "object-assign";
// Default options
const defaultOptions = {
    // max count of words >
    max: 50
};
/**
 * @param {TextLintRuleContext} context
 * @param {Object} options
 */
export default function (context, options = {}) {
    const {Syntax, getSource, RuleError, report} = context;
    const maxWordCount = options.max ? options.max : defaultOptions.max;
    return {
        [Syntax.Paragraph](node){
            // replace code with dummy code
            // if you want to filter(remove) code, use https://github.com/eush77/unist-util-filter
            const filteredNode = map(node, (node) => {
                if (node.type === Syntax.Code) {
                    // only change `value` to dummy
                    return ObjectAssign({}, node, {
                        value: "code"
                    });
                }
                return node;
            });
        }
    };
}
```

### 3. Get text of a Paragraph

We can get text from a node with **Source Map** using [textlint-util-to-string](https://www.npmjs.com/package/textlint-util-to-string).

Why does we use [textlint-util-to-string](https://www.npmjs.com/package/textlint-util-to-string)?

Because, we report error via `context.report` with original position of the text.

```js
// Helper for creating new AST using map function
// https://github.com/azu/unist-util-map
// if you want to filter, use https://github.com/eush77/unist-util-filter
import map from "unist-util-map";
// Helper for converting plain text from Syntax-ed text(markdown AST
// https://github.com/azu/textlint-util-to-string
import StringSource from "textlint-util-to-string";
import ObjectAssign from "object-assign";
// Default options
const defaultOptions = {
    // max count of words >
    max: 50
};
/**
 * @param {TextLintRuleContext} context
 * @param {Object} options
 */
export default function (context, options = {}) {
    const {Syntax, getSource, RuleError, report} = context;
{'fraction_non_alphanumeric': 0.14842924392352772, 'fraction_numerical': 0.0034585454894802575, 'mean_word_length': 3.052160373686259, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 11, 'https://': 38, 'lorem ipsum': 0, 'www.': 18, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1510208', 'n_tokens_mistral': 3309, 'n_tokens_neox': 3185, 'n_words': 917}