<commit_before># Internationalization (i18n)

We're using gettext for translations. All the actual translations are
carried out via [Pontoon](https://pontoon.mozilla.org/).

Some commands wrap standard gettext tools. To run these commands you'll need
to ensure you have done the following steps:

* Run `npm install` to install all the project deps.
* Install [gettext](https://www.gnu.org/software/gettext/) tools for your
platform.
* Install podebug (see [Building the debug locales for more information](#build-the-debug-locales))

*NOTE: All the instructions below show `[MY_APP]` you should replace that with the
name of the app you are updating e.g. `NODE_APP_INSTANCE=disco bin/create-locales`*

## Adding a new language/locale

The supported languages are defined in the configuration. See
[`config/default.js`](https://bit.ly/1XScjwq) and look for the `langs` list.

Add the new language to the list and then run:

```
# create the locale for a newly added language.
NODE_APP_INSTANCE=[MY_APP] bin/create-locales
```

## Updating locales

These docs detail the steps needing to be run to extract newly added strings
and update the locales.

After running each process you will need to commit the changes and push a PR
to the addons-frontend repo.

### Extracting newly added strings

Strings extracted are those wrapped with `i18n.gettext()` or any other
function supported by Jed (the library we use in JavaScript to carry out
replacements for the string keys in the current locale).

The strings are extracted using a babel plugin via webpack. Extracted strings
are added to a pot template file. This file is used to seed the po for each
locale with the strings needing translating when merging locales.

To extract + merge new strings for an app in the addons-frontend project
you should run the following commands:

```
# First extract the new strings.
NODE_APP_INSTANCE=[MY_APP] bin/extract-locales
# Merge the new strings into the locale files.
NODE_APP_INSTANCE=[MY_APP] bin/merge-locales
```

### Building the debug locales

Using this command Generates a unicode debug locale and a flipped rtl debug
locale using [podebug](https://bit.ly/1r3yFy5).

`podebug` is required for this step, you can install it with:

 * OSX: `brew install translate-toolkit`
 * Debian Linux variants: `sudo apt-get install translate-toolkit`
 * For other platforms see the [podebug docs](https://bit.ly/1r3yFy5)

To build the debug locale files run:

```
# build the debug locale files.
NODE_APP_INSTANCE=[MY_APP] bin/debug-locales
```

### Building the JSON locale files

This command create the JSON files which are then built into JS bundles by
webpack when the build step is run.

Since dist files are created when needed you only need to build and commit
the JSON to the repo.

```
# build the JSON.
NODE_APP_INSTANCE=[MY_APP] bin/build-locales
```
<commit_msg>Add docs re: component translation<commit_after># Internationalization (i18n)

We're using gettext for translations. All the actual translations are
carried out via [Pontoon](https://pontoon.mozilla.org/).

Some commands wrap standard gettext tools. To run these commands you'll need
to ensure you have done the following steps:

* Run `npm install` to install all the project deps.
* Install [gettext](https://www.gnu.org/software/gettext/) tools for your
platform.
* Install podebug (see [Building the debug locales for more information](#build-the-debug-locales))

*NOTE: All the instructions below show `[MY_APP]` you should replace that with the
name of the app you are updating e.g. `NODE_APP_INSTANCE=disco bin/create-locales`*

## Adding a new language/locale

The supported languages are defined in the configuration. See
[`config/default.js`](https://bit.ly/1XScjwq) and look for the `langs` list.

Add the new language to the list and then run:

```
# create the locale for a newly added language.
NODE_APP_INSTANCE=[MY_APP] bin/create-locales
```

## Updating locales

These docs detail the steps needing to be run to extract newly added strings
and update the locales.

After running each process you will need to commit the changes and push a PR
to the addons-frontend repo.

### Extracting newly added strings

Strings extracted are those wrapped with `i18n.gettext()` or any other
function supported by Jed (the library we use in JavaScript to carry out
replacements for the string keys in the current locale).

The strings are extracted using a babel plugin via webpack. Extracted strings
are added to a pot template file. This file is used to seed the po for each
locale with the strings needing translating when merging locales.

To extract + merge new strings for an app in the addons-frontend project
you should run the following commands:

```
# First extract the new strings.
NODE_APP_INSTANCE=[MY_APP] bin/extract-locales
# Merge the new strings into the locale files.
NODE_APP_INSTANCE=[MY_APP] bin/merge-locales
```

### Building the debug locales

Using this command Generates a unicode debug locale and a flipped rtl debug
locale using [podebug](https://bit.ly/1r3yFy5).

`podebug` is required for this step, you can install it with:

 * OSX: `brew install translate-toolkit`
 * Debian Linux variants: `sudo apt-get install translate-toolkit`
 * For other platforms see the [podebug docs](https://bit.ly/1r3yFy5)

To build the debug locale files run:

```
# build the debug locale files.
NODE_APP_INSTANCE=[MY_APP] bin/debug-locales
```

### Building the JSON locale files

This command create the JSON files which are then built into JS bundles by
webpack when the build step is run.

Since dist files are created when needed you only need to build and commit
the JSON to the repo.

```
# build the JSON.
NODE_APP_INSTANCE=[MY_APP] bin/build-locales
```

## Setting up translations

To set-up a component to be translated there are two pieces of code to know
about.

### Jed

We use [Jed](https://slexaxton.github.io/Jed/) as the API for providing
`gettext` functions inside React components. An initialized `Jed` instance
has all the `gettext` related functionality exposed as methods. There is a
fancy chained API but we've stuck to a more traditional approach.

Before we get into how to make use of these functions let's take a look at
how the Jed instance is exposed to our components.

### The Translation Provider

The translation provider is used to pass down a Jed instance via context to
components lower down in the component hierarchy. This part is already done
for you in addons-frontend. So you should only need to worry
about wrapping your components as detailed in the next section.

## The translate component wrapper.

The translate Higher Order Component is a helper that wraps any component
and takes the Jed `i18n` instance from context and makes it  available in
the wrapped component's props.

Here's an example of a basic component setup for translation:


```javascript
import React, { PropTypes } from 'react';
import translate from 'core/i18n/translate';


export class MyTranslatedComponent extends React.Component {
  static propTypes = {
    i18n: PropTypes.object.isRequired,
  }

  render() {
    const { i18n } = this.props;
    return (
      <div>
        <p>{i18n.gettext('Something translated')}</p>
      </div>
    );
  }
}

export default translate()(MyTranslatedComponent);
```

That's pretty much all there is to it.

## Using the Jed API

Once you have `i18n` available to your component you can then use
any of the Jed methods exposed on the `i18n` object.

```javascript
gettext = function ( key )
dgettext = function ( domain, key )
dcgettext = function ( domain, key, category )
ngettext = function ( singular_key, plural_key, value )
dngettext = function ( domain, singular_ley, plural_key, value )
dcngettext = function ( domain, singular_key, plural_key, value, category )
pgettext = function ( context, key )
dpgettext = function ( domain, context, key )
npgettext = function ( context, singular_key, plural_key, value )
dnpgettext = function ( domain, context, singular_key, plural_key, value )
dcnpgettext = function ( domain, context, singular_key, plural_key, value, category )
sprintf = function ( string, substitutions)
```

### Using `sprintf`

As you can see a sprintf function is also available. You can use this to
provide substitutions in gettext wrapped strings.

There are two flavours to this, numbered placeholders or named ones.

Here's the numbered approach:

```javascript
i18n.sprintf(i18n.gettext('I like your %1$s %2$s.'), 'red', 'shirt'));
```

and here's the named arg approach:

```javascript
i18n.sprintf(i18n.gettext('I like your %(colour)s %(garment)s.'), { colour: 'red', garment: 'shirt' }));
```

Both of these approaches allow for translators to re-order the substitution
vars.

### Guidance on HTML in translations

Generally we're looking to avoid having HTML in the middle of translation
strings as much as possible.

If you need HTML it's better to use substitutions to add the HTML than
leave HTML in the translation. You can also use DOMPurify to sanitize
strings that may contain HTML following substitutions so that anything
not explicitly allowed is removed. DOMpurify will also help protect
against malformed HTML in case opening and closing tag substitutions
vars get swapped around inadvertently.

{'fraction_non_alphanumeric': 0.0746606334841629, 'fraction_numerical': 0.00495582848524025, 'mean_word_length': 4.545400238948626, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 7, 'https://': 11, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4458138', 'n_tokens_mistral': 2662, 'n_tokens_neox': 2548, 'n_words': 1241}