<commit_before># lib-ui-service: composition

This library contains code for the composition of html pages out of multiple html page containing fragments.

### Key Concept
Every Service deliveres a functional html user inteface in form of complete html pages in the way, that a service can be developed and tested
by it's own. A UI-Service can request multimple pages from different services and compose them to one html page. To support this, the html pages
from the services contain a special html vocabular.

### Composition Process
The composition is done in the following steps:

1. The UI-Service has a `CompositionHandler` in it's handler chain, which answers these which need composition.
2. The `CompositionHandler` has a callback from the UI-Service. This callback gets a `http.Request` object as argument and returns a List of FetchResult.
3. For each request this callback is triggered. So the UI-Service can add a `ContentFetcher` for this request and adds FetchDefinitions for Page using `ContentFetcher.AddFetchJob()`.
4. The ContentFetchter loads the Pages and recursively it's dependencies in parallel. For the actual loading and parsing, it uses the `HtmlContentLoader`.
5. When all `Content` objects are loaded, the `CompositionHandler` merges them together, using `ContentMerge`.

### Merging
The it self is very simple:

- The MetaJSON is calculated by adding all fields of the loaded MetaJSON to one global map.
- All Head fragments are concatenated within the `<head>`.
- The Default Body Fragment of the first Content object is executed and may recursively include other Fragments Content objects.
- All Tail fragments are concatenated at hte end of the `<body>`.

### Execution Order
**Attention**: The execution order of the Content Objects is determined by the order in which they are returned from the `ContentFetcher`.
Currently this is only deterministic within the FetchDefinitions added by `ContentFetcher.AddFetchJob()`. The recursive dependencies are loaded from them are in a random order.
This may case not demerninistic behaviour, if the contain equal named fragments or provide the same MetaJSON attributes.

### Caching
At a later point, the ContentFetcher may provide Caching of FetchDefinitions.

## HTML Composition Vocabulary

### Attribute `uic-remove`
An UI-Service has to remove the element marked with this attribute and all its subelements.
Be careful to have a correct open and closing structure in the html. The standard selfclosing tags are
allowed, e.g. both are working `<br>` and `<br/>`, but if there is a structure error with e.g. a div,
`uic-remove` may lead to strage behaviour.

Example:

```html
<link uic-remove rel="stylesheet" type="text/css" href="testing.css"/>
```

Where: Everywhere (head, body, within framents)

### Script type `text/uic-meta`
A html page may contain a script of type `text/uic-meta`, with a JSON object as content.
The UI-Service has to add the contents of the JSON object to its global meta data object.

Example:

```html
<script type="text/uic-meta">
  {
   "foo": "bar",
   "boo": "bazz",
   "categories": ["animal", "human"]
  }
</script>
```

Where: head

### Fragments
The UI-Service interpretes an html page as a set of fragments. All those fragments are optional.
- One __Head Fragement__, identified by the child elements of the html `<head>` tag.
- One __Body Default Fragement__, identified by the child elements of the `<body>` tag or by a `uic-fragment` without a name attribute.
- Multiple __Named Body Fragements__, identified by `uic-fragment` tag within the body.
- One __Tail Fragment__, identified by the `uic-tail` tag.

#### Head-Fragment
The complete contents of the head is interpreted as the head fragment. The elements marked with `uic-remove`
and the `uic-meta` Script are not cleaned out of the head fragment. If the head framents only contains whitespace,
it is interpreted as not existing.

Example: The Head Fragment contains `<title>The Title</title>`

```html
<head>
  <title>The Title</title>
  <link uic-remove rel="stylesheet" type="text/css" href="special.css"/>
  <script type="text/uic-meta">
    {}
  </script>
</head>
```

#### Body Default Fragment 
All other elements fragments and those elements, marked with `uic-remove`, are removed from the body
and the remaining fragment is taken as Body Default Fragment. The Body Default Fragment is just a fragment with
the empthy name (""). If there is a `uic-fragment` tag without the name in the body, this overwrites the default fragment.

Example: The Default Fragment contains `<h1>Hello World</h1>`

```html
<body>
    Hello World
    <ul uic-remove>
      <!-- A Navigation for testing -->
    </ul>
    <uic-fragment name="headline">
      <h1>This is a headline</h1>
    </uic-fragment>
</body>
```

The complete contents of the body is interpreted as the head fragment. The elements marked with `uic-remove`
and the `uic-meta` Script are not cleaned out of the head fragment. If the head framents only contains whitespace,
it is interpreted as not existing.

Example: The Default Fragment contains `<h1>This is the default</h1>`

```html
<body>
    <h1>Hello World</h1>
    <uic-fragment>
      <h1>This is the default</h1>
    </uic-fragment>
</body>
```

#### Element `uic-fragment`
The body of an html page may contain multiple `uic-fragment` tags. Which contain the fragments for the page.
All content withing the tag is taken as fragment content. Nested Fragment tags are not allowed.

The Fragment Tag my have a `name` attribute, for named the fragment. I no attribute is given, or the name is empty,
the Body Default Fragment is overwritten by this fragment.

Example: Contains two fragments *headline* and *headline*

```html
<body>
  <uic-fragment name="headline">
    <h1>This is a headline</h1>
  </uic-fragment>
  <uic-fragment name="w">
    Bli Bla blub
    <div uic-remove>
       Some element for testing
    </div>
  </uic-fragment>
</body>
```

Where: body

### Templating
All fragments (except the Head Fragment) may contain minimal templating directives which has to be resolved by the UI-Service.
There are two forms of includes and a syntax for variable replacement.

#### Variables (TODO: Not implemented yet)
The UI-Service has to replace Variable directires by the corresponding path out of the global meta data.<commit_msg>formating<commit_after># lib-ui-service: composition

This library contains code for the composition of html pages out of multiple html page containing fragments.

### Key Concept
Every Service deliveres a functional html user inteface in form of complete html pages in the way, that a service can be developed and tested
by it's own. A UI-Service can request multimple pages from different services and compose them to one html page. To support this, the html pages
from the services contain a special html vocabular.

### Composition Process
The composition is done in the following steps:

1. The UI-Service has a `CompositionHandler` in it's handler chain, which answers these which need composition.
2. The `CompositionHandler` has a callback from the UI-Service. This callback gets a `http.Request` object as argument and returns a List of FetchResult.
3. For each request this callback is triggered. So the UI-Service can add a `ContentFetcher` for this request and adds FetchDefinitions for Page using `ContentFetcher.AddFetchJob()`.
4. The ContentFetchter loads the Pages and recursively it's dependencies in parallel. For the actual loading and parsing, it uses the `HtmlContentLoader`.
5. When all `Content` objects are loaded, the `CompositionHandler` merges them together, using `ContentMerge`.

### Merging
The it self is very simple:

- The MetaJSON is calculated by adding all fields of the loaded MetaJSON to one global map.
- All Head fragments are concatenated within the `<head>`.
- The Default Body Fragment of the first Content object is executed and may recursively include other Fragments Content objects.
- All Tail fragments are concatenated at hte end of the `<body>`.

### Execution Order
**Attention**: The execution order of the Content Objects is determined by the order in which they are returned from the `ContentFetcher`.
Currently this is only deterministic within the FetchDefinitions added by `ContentFetcher.AddFetchJob()`. The recursive dependencies are loaded from them are in a random order.
This may case not demerninistic behaviour, if the contain equal named fragments or provide the same MetaJSON attributes.

### Caching
At a later point, the ContentFetcher may provide Caching of FetchDefinitions.



## HTML Composition Vocabulary

### Attribute `uic-remove`
An UI-Service has to remove the element marked with this attribute and all its subelements.
Be careful to have a correct open and closing structure in the html. The standard selfclosing tags are
allowed, e.g. both are working `<br>` and `<br/>`, but if there is a structure error with e.g. a div,
`uic-remove` may lead to strage behaviour.

Example:

```html
<link uic-remove rel="stylesheet" type="text/css" href="testing.css"/>
```

Where: Everywhere (head, body, within framents)

### Script type `text/uic-meta`
A html page may contain a script of type `text/uic-meta`, with a JSON object as content.
The UI-Service has to add the contents of the JSON object to its global meta data object.

Example:

```html
<script type="text/uic-meta">
  {
   "foo": "bar",
   "boo": "bazz",
   "categories": ["animal", "human"]
  }
</script>
```

Where: head


### Fragments
The UI-Service interpretes an html page as a set of fragments. All those fragments are optional.
- One __Head Fragement__, identified by the child elements of the html `<head>` tag.
- One __Body Default Fragement__, identified by the child elements of the `<body>` tag or by a `uic-fragment` without a name attribute.
- Multiple __Named Body Fragements__, identified by `uic-fragment` tag within the body.
- One __Tail Fragment__, identified by the `uic-tail` tag.

#### Head-Fragment
The complete contents of the head is interpreted as the head fragment. The elements marked with `uic-remove`
and the `uic-meta` Script are not cleaned out of the head fragment. If the head framents only contains whitespace,
it is interpreted as not existing.

Example: The Head Fragment contains `<title>The Title</title>`

```html
<head>
  <title>The Title</title>
  <link uic-remove rel="stylesheet" type="text/css" href="special.css"/>
  <script type="text/uic-meta">
    {}
  </script>
</head>
```

#### Body Default Fragment 
All other elements fragments and those elements, marked with `uic-remove`, are removed from the body
and the remaining fragment is taken as Body Default Fragment. The Body Default Fragment is just a fragment with
the empthy name (""). If there is a `uic-fragment` tag without the name in the body, this overwrites the default fragment.

Example: The Default Fragment contains `<h1>Hello World</h1>`

```html
<body>
    Hello World
    <ul uic-remove>
      <!-- A Navigation for testing -->
    </ul>
    <uic-fragment name="headline">
      <h1>This is a headline</h1>
    </uic-fragment>
</body>
```

The complete contents of the body is interpreted as the head fragment. The elements marked with `uic-remove`
and the `uic-meta` Script are not cleaned out of the head fragment. If the head framents only contains whitespace,
it is interpreted as not existing.

Example: The Default Fragment contains `<h1>This is the default</h1>`

```html
<body>
    <h1>Hello World</h1>
    <uic-fragment>
      <h1>This is the default</h1>
    </uic-fragment>
</body>
```

#### Element `uic-fragment`
The body of an html page may contain multiple `uic-fragment` tags. Which contain the fragments for the page.
All content withing the tag is taken as fragment content. Nested Fragment tags are not allowed.

The Fragment Tag my have a `name` attribute, for named the fragment. I no attribute is given, or the name is empty,
the Body Default Fragment is overwritten by this fragment.

Example: Contains two fragments *headline* and *headline*

```html
<body>
  <uic-fragment name="headline">
    <h1>This is a headline</h1>
  </uic-fragment>
  <uic-fragment name="w">
    Bli Bla blub
    <div uic-remove>
       Some element for testing
    </div>
  </uic-fragment>
</body>
```

Where: body


### Templating
All fragments (except the Head Fragment) may contain minimal templating directives which has to be resolved by the UI-Service.
There are two forms of includes and a syntax for variable replacement.

#### Variables (TODO: Not implemented yet)
The UI-Service has to replace Variable directires by the corresponding path out of the global meta data.
{'fraction_non_alphanumeric': 0.08308324119412415, 'fraction_numerical': 0.002685199810456484, 'mean_word_length': 4.247824285122254, 'pattern_counts': {'":': 6, '<': 101, '<?xml version=': 0, '>': 101, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2842518', 'n_tokens_mistral': 3507, 'n_tokens_neox': 3402, 'n_words': 1767}