GraphComponentException if `hidden_layers_size` used with only one (text) overwritten key
### Rasa Open Source version

3.0.0

### Rasa SDK version

3.0.0

### Rasa X version

_No response_

### Python version

3.8

### What operating system are you using?

Windows

### What happened?

I wanted to train a migrated AI with the following config for `DIETClassifier`:

```
- name: DIETClassifier
  random_seed: 42
  loss_type: cross_entropy
  model_confidence: softmax
  constrain_similarities: True
  intent_classification: True
  entity_recognition: False
  use_masked_language_model: True
  epochs: 80
  number_of_transformer_layers: 4
  transformer_size: 256
  drop_rate: 0.2
  batch_size: [64, 256]
  embedding_dimension: 50
  hidden_layers_sizes:
    text: [512, 128]
```
The result was the following error:

```
  File "...rasa\\nlu\\classifiers\\diet_classifier.py", line 1277, in __init__
    self._prepare_layers()
  File "...rasa\\nlu\\classifiers\\diet_classifier.py", line 1437, in _prepare_layers
    self.config[HIDDEN_LAYERS_SIZES][self.label_name],
  File "...wrapt\\wrappers.py", line 394, in __getitem__
    return self.__wrapped__[key]
KeyError: 'label'
```

followed by 

`rasa.engine.exceptions.GraphComponentException: Error running graph component for node train_DIETClassifier5.`

After digging into it, it seems to me, that even if the documentation sais:

```
hidden_layers_sizes: This parameter allows you to define the number of feed forward layers and their output dimensions for user messages and intents (default: text: [], label: []). 
```
it is not enough to specify only one of the keys. After changing the config to:

```
  hidden_layers_sizes:
    text: [512, 128]
    label: []
```

everything worked fine. I am not absolutely sure but this worked in Rasa `2.8.x`.

### Command / Request

_No response_

### Relevant log output

```shell
2021-11-25 14:36:48 INFO     rasa.engine.training.hooks  - Restored component 'LexicalSyntacticFeaturizer' from cache.
2021-11-25 14:36:48 INFO     rasa.engine.training.hooks  - Restored component 'CountVectorsFeaturizer' from cache.
2021-11-25 14:36:48 INFO     rasa.engine.training.hooks  - Restored component 'CountVectorsFeaturizer' from cache.
2021-11-25 14:36:48 INFO     rasa.engine.training.hooks  - Restored component 'MemoizationPolicy' from cache.
2021-11-25 14:36:48 INFO     rasa.engine.training.hooks  - Restored component 'RulePolicy' from cache.
2021-11-25 14:36:49 INFO     rasa.engine.training.hooks  - Starting to train component 'DIETClassifier'.
...
  File "...rasa\\nlu\\classifiers\\diet_classifier.py", line 1277, in __init__
    self._prepare_layers()
  File "...rasa\\nlu\\classifiers\\diet_classifier.py", line 1437, in _prepare_layers
    self.config[HIDDEN_LAYERS_SIZES][self.label_name],
  File "...wrapt\\wrappers.py", line 394, in __getitem__
    return self.__wrapped__[key]
KeyError: 'label'
```

Definition of Done:
- [x] Discovered the problem
- [x] Fixed the problem
- [x] Made sure the same problem does not exist for other parameters of the component and also other ML components like TEDPolicy, ResponseSelector and UnexpecTEDIntentPolicy
- [x] Model regression tests run on the PR
- [x] PR is merged to `3.0.x`
- [ ] New micro version released after the bugfix is merged.

diff --git a/rasa/core/policies/unexpected_intent_policy.py b/rasa/core/policies/unexpected_intent_policy.py
index 7c6067a339e..e578b9291f2 100644
--- a/rasa/core/policies/unexpected_intent_policy.py
+++ b/rasa/core/policies/unexpected_intent_policy.py
@@ -112,6 +112,7 @@ from rasa.utils.tensorflow.model_data import (
 )
 
 import rasa.utils.io as io_utils
+import rasa.utils.common
 from rasa.core.exceptions import RasaCoreException
 from rasa.shared.utils import common
 
@@ -889,7 +890,7 @@ class UnexpecTEDIntentPolicy(TEDPolicy):
 
     @classmethod
     def _update_loaded_params(cls, meta: Dict[Text, Any]) -> Dict[Text, Any]:
-        meta = train_utils.override_defaults(cls.get_default_config(), meta)
+        meta = rasa.utils.common.override_defaults(cls.get_default_config(), meta)
         return meta
 
     @classmethod
diff --git a/rasa/engine/graph.py b/rasa/engine/graph.py
index 8c84ac9a422..9c34f4deab2 100644
--- a/rasa/engine/graph.py
+++ b/rasa/engine/graph.py
@@ -11,6 +11,8 @@ from rasa.engine.exceptions import (
     GraphSchemaException,
 )
 import rasa.shared.utils.common
+import rasa.utils.common
+
 from rasa.engine.storage.resource import Resource
 
 from rasa.engine.storage.storage import ModelStorage
@@ -359,10 +361,9 @@ class GraphNode:
         self._constructor_fn: Callable = getattr(
             self._component_class, self._constructor_name
         )
-        self._component_config: Dict[Text, Any] = {
-            **self._component_class.get_default_config(),
-            **component_config,
-        }
+        self._component_config: Dict[Text, Any] = rasa.utils.common.override_defaults(
+            self._component_class.get_default_config(), component_config,
+        )
         self._fn_name: Text = fn_name
         self._fn: Callable = getattr(self._component_class, self._fn_name)
         self._inputs: Dict[Text, Text] = inputs
diff --git a/rasa/utils/common.py b/rasa/utils/common.py
index babf3f86996..3d2c5ea106b 100644
--- a/rasa/utils/common.py
+++ b/rasa/utils/common.py
@@ -1,4 +1,5 @@
 import asyncio
+import copy
 import logging
 import os
 import shutil
@@ -310,6 +311,35 @@ def update_existing_keys(
     return updated
 
 
+def override_defaults(
+    defaults: Optional[Dict[Text, Any]], custom: Optional[Dict[Text, Any]]
+) -> Dict[Text, Any]:
+    """Override default config with the given config.
+
+    We cannot use `dict.update` method because configs contain nested dicts.
+
+    Args:
+        defaults: default config
+        custom: user config containing new parameters
+
+    Returns:
+        updated config
+    """
+    if defaults:
+        config = copy.deepcopy(defaults)
+    else:
+        config = {}
+
+    if custom:
+        for key in custom.keys():
+            if isinstance(config.get(key), dict):
+                config[key].update(custom[key])
+            else:
+                config[key] = custom[key]
+
+    return config
+
+
 class RepeatedLogFilter(logging.Filter):
     """Filter repeated log records."""
 
diff --git a/tests/engine/test_graph_node.py b/tests/engine/test_graph_node.py
index 38a29cef495..38559b9afdb 100644
--- a/tests/engine/test_graph_node.py
+++ b/tests/engine/test_graph_node.py
@@ -329,3 +329,46 @@ def test_loading_from_resource_eager(default_model_storage: ModelStorage):
 
     assert actual_node_name == node_name
     assert value == test_value
+
+
+def test_config_with_nested_dict_override(default_model_storage: ModelStorage):
+    class ComponentWithNestedDictConfig(GraphComponent):
+        @staticmethod
+        def get_default_config() -> Dict[Text, Any]:
+            return {"nested-dict": {"key1": "value1", "key2": "value2"}}
+
+        @classmethod
+        def create(
+            cls,
+            config: Dict,
+            model_storage: ModelStorage,
+            resource: Resource,
+            execution_context: ExecutionContext,
+            **kwargs: Any,
+        ) -> ComponentWithNestedDictConfig:
+            return cls()
+
+        def run(self) -> None:
+            return None
+
+    node = GraphNode(
+        node_name="nested_dict_config",
+        component_class=ComponentWithNestedDictConfig,
+        constructor_name="create",
+        component_config={"nested-dict": {"key2": "override-value2"}},
+        fn_name="run",
+        inputs={},
+        eager=True,
+        model_storage=default_model_storage,
+        resource=None,
+        execution_context=ExecutionContext(GraphSchema({}), "123"),
+    )
+
+    expected_config = {"nested-dict": {"key1": "value1", "key2": "override-value2"}}
+
+    for key, value in expected_config.items():
+        assert key in node._component_config
+        if isinstance(value, dict):
+            for nested_key, nested_value in expected_config[key].items():
+                assert nested_key in node._component_config[key]
+                assert node._component_config[key][nested_key] == nested_value
diff --git a/tests/nlu/classifiers/test_diet_classifier.py b/tests/nlu/classifiers/test_diet_classifier.py
index 6481095cc91..9eeaa59fea3 100644
--- a/tests/nlu/classifiers/test_diet_classifier.py
+++ b/tests/nlu/classifiers/test_diet_classifier.py
@@ -5,6 +5,7 @@ import numpy as np
 import pytest
 from typing import Callable, List, Optional, Text, Dict, Any, Tuple
 
+import rasa.utils.common
 from rasa.engine.graph import ExecutionContext, GraphComponent
 from rasa.engine.storage.resource import Resource
 from rasa.engine.storage.storage import ModelStorage
@@ -35,6 +36,7 @@ from rasa.utils.tensorflow.constants import (
     ENTITY_RECOGNITION,
     INTENT_CLASSIFICATION,
     MODEL_CONFIDENCE,
+    HIDDEN_LAYERS_SIZES,
 )
 from rasa.nlu.tokenizers.whitespace_tokenizer import WhitespaceTokenizer
 from rasa.nlu.classifiers.diet_classifier import DIETClassifier
@@ -73,7 +75,9 @@ def create_diet(
 
         default_execution_context.is_finetuning = finetune
         return constructor(
-            config={**DIETClassifier.get_default_config(), **config},
+            config=rasa.utils.common.override_defaults(
+                DIETClassifier.get_default_config(), config
+            ),
             model_storage=default_model_storage,
             execution_context=default_execution_context,
             resource=default_diet_resource,
@@ -307,6 +311,16 @@ async def test_train_persist_load_with_different_settings(
     create_diet(config, load=True, finetune=True)
 
 
+@pytest.mark.timeout(240, func_only=True)
+async def test_train_persist_load_with_nested_dict_config(
+    create_train_load_and_process_diet: Callable[..., Message],
+    create_diet: Callable[..., DIETClassifier],
+):
+    config = {HIDDEN_LAYERS_SIZES: {"text": [256, 512]}}
+    create_train_load_and_process_diet(config)
+    create_diet(config, load=True, finetune=True)
+
+
 @pytest.mark.timeout(210, func_only=True)
 async def test_train_persist_load_with_only_entity_recognition(
     create_train_load_and_process_diet: Callable[..., Message],
diff --git a/tests/utils/test_common.py b/tests/utils/test_common.py
index 1ddadcb6501..e7f5d17d169 100644
--- a/tests/utils/test_common.py
+++ b/tests/utils/test_common.py
@@ -136,3 +136,13 @@ def test_find_unavailable_packages():
 )
 def test_module_path_from_class(clazz: Type, module_path: Text):
     assert rasa.utils.common.module_path_from_class(clazz) == module_path
+
+
+def test_override_defaults():
+    defaults = {"nested-dict": {"key1": "value1", "key2": "value2"}}
+    custom = {"nested-dict": {"key2": "override-value2"}}
+
+    updated_config = rasa.utils.common.override_defaults(defaults, custom)
+
+    expected_config = {"nested-dict": {"key1": "value1", "key2": "override-value2"}}
+    assert updated_config == expected_config
['rasa/engine/graph.py', 'rasa/core/policies/unexpected_intent_policy.py', 'tests/utils/test_common.py', 'tests/engine/test_graph_node.py', 'tests/nlu/classifiers/test_diet_classifier.py', 'rasa/utils/common.py']