hierarchical output in mongoDB bot
Hi,
When option hierarchical_output is turn to false, the bot crashed.
Pymongo seems to not accept dot in key => like "feed.name"
Even it is possible to do that in mongo, it seems to be a really bad practice because the dot notation is used to access element in the document.
Maybe it's possible to add a parameter for define the separator which replace the dot.
To discuss :)

Tested on ubuntu 18.04, with pymongo  and a mongoDB 3.6.3
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 80ef13d3f..0a2d85b6e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,9 @@ CHANGELOG
 
 #### Outputs
 - `intelmq.bots.outputs.redis`: Fix sending password to redis server.
+- `intelmq.bots.outputs.mongodb`:
+  - New parameter `replacement_char` (default: `'_'`) for non-hierarchical output as dots in key names are not allowed (#1324, #1322).
+  - Save value of fields `time.observation` and `time.source` as native datetime object, not as string (#1322).
 
 ### Documentation
 
diff --git a/NEWS.md b/NEWS.md
index 91e019b8e..2fd8d240b 100644
--- a/NEWS.md
+++ b/NEWS.md
@@ -16,6 +16,9 @@ See the changelog for a full list of changes.
 
 ### Postgres databases
 
+### MongoDB databases
+In previous version the MongoDB Output Bot saved the fields `time.observation` and `time.source` as strings in ISO format. But MongoDB does support saving datetime objects directly which are converted to its native date format, enabling certain optimizations and features. The MongoDB Output Bot now saves these values as datetime objects.
+
 1.1.0 Feature release (2018-09-05)
 ----------------------------------
 ### Requirements
diff --git a/docs/Bots.md b/docs/Bots.md
index 25d03bc9c..97250686e 100644
--- a/docs/Bots.md
+++ b/docs/Bots.md
@@ -1110,6 +1110,8 @@ If the field used in the format string is not defined, `None` will be used as fa
 
 ### MongoDB
 
+Saves events in a MongoDB either as hierarchical structure or flat with full key names. `time.observation` and `time.source` are saved as datetime objects, not as ISO formatted string.
+
 #### Information:
 * `name:` mongodb
 * `lookup:` no
@@ -1126,6 +1128,7 @@ If the field used in the format string is not defined, `None` will be used as fa
 * `host`: MongoDB host (FQDN or IP)
 * `port`: MongoDB port
 * `hierarchical_output`: Boolean (default true) as mongodb does not allow saving keys with dots, we split the dictionary in sub-dictionaries.
+* `replacement_char`: String (default `'_'`) used as replacement character for the dots in key names if hierarchical output is not used.
 
 #### Installation Requirements
 
diff --git a/intelmq/bots/BOTS b/intelmq/bots/BOTS
index fdcfd2288..79e317ca9 100644
--- a/intelmq/bots/BOTS
+++ b/intelmq/bots/BOTS
@@ -820,7 +820,8 @@
                 "db_pass": null,
                 "hierarchical_output": true,
                 "host": "localhost",
-                "port": "27017"
+                "port": "27017",
+                "replacement_char": "_"
             }
         },
         "PostgreSQL": {
diff --git a/intelmq/bots/outputs/mongodb/output.py b/intelmq/bots/outputs/mongodb/output.py
index 086f94944..7d8350a8e 100644
--- a/intelmq/bots/outputs/mongodb/output.py
+++ b/intelmq/bots/outputs/mongodb/output.py
@@ -4,6 +4,7 @@
 """
 
 from intelmq.lib.bot import Bot
+import dateutil.parser
 
 try:
     import pymongo
@@ -16,7 +17,9 @@ class MongoDBOutputBot(Bot):
     def init(self):
         if pymongo is None:
             raise ValueError('Could not import pymongo. Please install it.')
-
+        self.replacement_char = getattr(self.parameters, 'replacement_char', '_')
+        if self.replacement_char == '.':
+            raise ValueError('replacement_char should be different than .')
         self.connect()
 
     def connect(self):
@@ -42,8 +45,26 @@ def connect(self):
     def process(self):
         event = self.receive_message()
 
+        if self.parameters.hierarchical_output:
+            tmp_dict = event.to_dict(hierarchical=True)
+            if "time"in tmp_dict:
+                if "observation" in tmp_dict["time"]:
+                    tmp_dict["time"]["observation"] = dateutil.parser.parse(tmp_dict["time"]["observation"])
+                if "source" in tmp_dict["time"]:
+                    tmp_dict["time"]["source"] = dateutil.parser.parse(tmp_dict["time"]["source"])
+        else:
+            # flat version
+            # replace . in key by replacement_char
+            tmp_dict = {key.replace('.', self.replacement_char): value for key, value in event.to_dict().items()}
+            time_obs = "time%sobservation" % self.replacement_char
+            if time_obs in tmp_dict:
+                tmp_dict[time_obs] = dateutil.parser.parse(tmp_dict[time_obs])
+            time_src = "time%source" % self.replacement_char
+            if time_src in tmp_dict:
+                tmp_dict[time_src] = dateutil.parser.parse(tmp_dict[time_obs])
+
         try:
-            self.collection.insert(event.to_dict(hierarchical=self.parameters.hierarchical_output))
+            self.collection.insert(tmp_dict)
         except pymongo.errors.AutoReconnect:
             self.logger.error('Connection Lost. Connecting again.')
             self.connect()
['NEWS.md', 'CHANGELOG.md', 'intelmq/bots/outputs/mongodb/output.py', 'intelmq/bots/BOTS', 'docs/Bots.md']