EgoReID: Person re-identification in Egocentric Videos Acquired by Mobile Devices with First-Person Point-of-View

Widespread use of wearable cameras and recording devices such as cellphones have opened the door to a lot of interesting research in first-person Point-of-view (POV) videos (egocentric videos). In recent years, we have seen the performance of video-based person Re-Identification (ReID) methods improve considerably. However, with the influx of varying video domains, such as egocentric videos, it has become apparent that there are still many open challenges to be faced. These challenges are a result of factors such as poor video quality due to ego-motion, blurriness, severe changes in lighting conditions and perspective distortions. To facilitate the research towards conquering these challenges, this paper contributes a new, first-of-its-kind dataset called EgoReID. The dataset is captured using 3 mobile cellphones with non-overlapping field-of-view. It contains 900 IDs and around 10,200 tracks with a total of 176,000 detections. Moreover, for each video we also provide 12-sensor meta data. Directly applying current approaches to our dataset results in poor performance. Considering the unique nature of our dataset, we propose a new framework which takes advantage of both visual and sensor meta data to successfully perform Person ReID. In this paper, we propose to adopt human body region parsing to extract local features from different body regions and then employ 3D convolution to better encode temporal information of each sequence of body parts. In addition, we also employ sensor meta data to determine target's next camera and their estimated time of arrival, such that the search is only performed among tracks present in the predicted next camera around the estimated time. This considerably improves our ReID performance as it significantly reduces our search space.

Introduction
Person Re-Identification (ReID) aims at associating the same pedestrian across multiple cameras [11,54]. This task has drawn increasing attention in recent years due to its importance in applications, such as surveillance [43], activity analysis [26] and tracking [48]. Person ReID remains a challenging problem because of complex variations in camera viewpoints, human poses, lighting, occlusions, and background clutter.
Currently, in almost all attempts to solve person ReID problem, the source of data is from fixed cameras with wider field of view (FOV). Mostly, in such datasets, full bodies of pedestrians are captured and intra camera variations of targets are very small, as pose of targets often do not change within the same camera. On the contrary, videos from a moving first-person point-of-view (POV) devices pose unique challenges, such as frequent occlusion of targets due to close proximity of camera and targets, blurriness caused by camera motion, sever lighting and background changes, both within and across  Figure 1: (a) shows three tracklets of the same identity from the same camera, its evident to see that due to both camera and target motions a target is captured in different poses, backgrounds and illuminations. (b) each row shows tracklets of the same identity captured from camera 1, 2 and 3, respectively. Due to close proximity of target and the cameras, different body regions of the target are missing in Camera 1 and 2. cameras, appearance difference due to different perspectives. Moreover, due to constant motion of both target and cameras, there is a frequent inter camera appearance changes between sequences of the same target. This is mainly due to the fact that a target could be recorded in several poses in a single camera as shown in Fig. 1 (a). The scene is more dynamic as compared to a traditional fixed camera scenario.
Besides the unique challenges posed by the nature of the video, these devices (such as cellphones) provide additional source of information. Most new generation devices are equipped with additional sensors such as gyros, accelerometers, magnetometers, GPS and are Internet-enabled, it is now possible to obtain large amounts of first-person point-of-view (POV) data un-intrusively.
Aiming to facilitate the research towards such unique video domain, we propose a new dataset called EgoReID. Different from existing datasets EgoReID presents several new features. 1) 3 synchronized mobile cellphones with non-overlapping FOV are used to record the videos. 2) Throughout the recording, all 3 cameras are moving around campus covering larger area, resulting in complex scene transformation and background. 3) 12 sensor meta data along with the videos are also collected. 4) YOLO9000 [30] and FCDSC [39] are used for pedestrian detection and tracklet generation, respectively, which will be provided with the dataset.
In a typical person ReID pipeline, first, each pedestrian video sequence is represented by a single feature vector and then is matched in a task-specific metric space, where the feature vectors of same target are expected to have smaller distance than that of different target [46,23,24]. These approaches show impressive results, when target sequences do not undergo drastic view changes or occlusion. However, in egocentric videos pedestrians undergo drastic change in both appearance and pose due to frequent motion of both camera and targets. Furthermore, frequent occlusion of different body parts of targets is common due to close proximity of a target to the camera as shown in Fig. 1 To address aforementioned challenges posed by the first-person POV videos, we propose a framework which uses human semantic parsing to extract local features from human body parts. It is natural to make use of semantic body regions in this particular setup, since mostly pedestrians are in close proximity from cameras, thus, different body regions are more visible. However, most of Person Re-ID datasets deal with image-based re-ID problem and do not deal with videos, therefore very few video-based Person re-ID datasets are available for training video deep networks for re-ID. In this paper, we propose a novel approach to leverage network trained on 10 image-based re-ID datasets for video-based Person Re-ID problem. Here, we employ a model, trained on these image-based datasets, to extract frame level local features for each semantic region, then we employ 3D convolutions to encode the temporal information in each sequence of regions. Instead of directly encoding the whole sequence, we use sequence of each regions to learn discriminative local features and compare respective regions separately to find a match. In addition to traditional video features, we successfully demonstrate the use of additional sensor meta data to complement appearance cues. In particular, we use meta data information to estimate target's next camera and its time of arrival, that way we significantly reduce the search space by pruning several unreliable pairs which violate the estimated time of arrival constraint.
The contributions of this paper can be summarized as follows: 1) A unique and challenging EgoR-eID dataset is collected and will be released with detections, tracks and sensor meta data. Compared to existing datasets, EgoReID has a unique feature which can help advance research in solving realistic and challenging task of person re-identification in a first-person POV videos. 2) We propose a new model to solve person ReID problem in egocentric videos, where human semantic parsing is employed to extract local visual cues and compare corresponding semantic regions to find a match. 3) We propose a method to employ sensor meta data information to significantly reduce our search space by first predicting the next camera of a target then estimating its time of arrival, thus, we only search for our match among tracks which satisfy the time constraint.

Related Works
In this section, we review related works in the areas of video-based person re-identification and egocentric vision.
Video-based person re-identification: The success of deep learning in wide range of computer vision areas has been inspiring a lot of studies in person re-identification. In recent years, researchers are considering more realistic scenarios such as larger dataset [53,52], complex scenarios [55,31] and combining different modalities such as text descriptions in their approaches [56,19].
The success of deep learning in image classification [16], has inspired several studies in person re-identification. The effectiveness of Convolutional Neural Network (CNN) in learning discriminative image representations from large scale datasets have been exploited effectively in [40,41,25,17,6,51,58,49,38].
Utilization of video data is further facilitated by the powerful feature learning ability of Convolutional Neural Networks.
In video-based ReID [29,58,46,42,59,28], the learning algorithm is given pair of video sequences instead of images. Authors in [29], introduce an RNN model to encode temporal information. The features from all timesteps are then combined using temporal pooling to give an overall appearance feature for the complete sequence and then compute the feature similarity of two videos. In [42] the most discriminative video fragments are selected from noisy/incomplete image sequences of people from which reliable space-time and appearance features can be computed, whilst simultaneously learning a video ranking function for person ReID. Authors in [28], introduce a new space-time person representation by encoding multiple granularities of spatio-temporal dynamics in form of time series. In [58] attention weights are combined per-frame with visual features and the forward propagated RNN hidden variables, in order to weaken the influence of the noisy samples. Authors in [45] employ spatial pooling layer to select regions from each frame, while temporal attention pooling is performed to select informative frames over the sequence, both pooling guided by the information from their distance matching. In [36], the dual attention mechanism is used, in which both intra-sequence and intersequence attention strategies are employed for feature refinement and feature-pair alignment, respectively.
Egocentric vision: Recently, visual analysis of egocentric videos has been a hot topic in computer vision, and the work ranging from object detection [9] to recognizing daily activities [8,7], predicting gaze behavior [23,3], and video summarization [27] have been proposed.

EgoReID Dataset
In this section, we briefly introduce our new EgoReID dataset.

Dataset Description
During collection, we use 3 synchronized Samsung mobile cellphones with non-overlapping field of views. Throughout the period of recording, all 3 cameras were constantly moving around and covered larger area. To generate tracklets, we first use YOLO9000 [30] to detect pedestrians and then employ FCDSC tracker [39] to generate trackles. Sample tracklets are shown in Fig. 1 and more are given in the supplementary material. EgoReID consists of around 900 different identities with 10,200 tracklets with a total of 176,000 human detection bounding boxes. Camera 1 and 2 have 190 IDs in common while camera 2 and 3 share 256 IDs in common. Around 103 IDs are present in all three cameras. 3164, 2748 and 4353 tracklets are present in camera 1,2 and 3, respectively.
Evaluation protocol: Similar to most of previous datasets, we utilize the Cumulative Matching Characteristics (CMC) curve to evaluate the ReID performance. For each query, multiple true positives could be returned. Therefore, we also consider person ReID as a retrieval task, and also employ mean Average Precision (mAP) as the evaluation metric.

Proposed Method
This section briefly presents the proposed model which consists of 3 main components: frame-level feature extraction, semantic segmentation and videolevel feature extraction. The overview of the proposed network is shown in Figure 3.
Our model is inspired by the model [14], where human parsing is successfully applied in imagebased ReID. Important feature of their model is that it is trained on ten different dataset providing robust system. In the unique nature of our dataset, since targets are close to the cameras, different body regions of a target are clearly visible, therefore above method is pretty suitable for our problem. However, their method is image-based, in this paper, we extend their method to videos and achieve significantly better results as compared to the existing state-of-the-art method [35] on our dataset.
Among several ways to extend the image-based model to videos, one is to inflate [4] the bottom 2D layers to 3D, such that it accepts a video clip and generates its single compact representation. Since we have two modules: feature extraction and segmentation to process a video, we need to inflate both of them. Even though we can employ the abovementioned modification over the feature extraction module, since we have enough labeled video dataset for training our extended model, however, our semantic segmentation module suffers from the lack of semantic segmentation annotations at video level. Therefore, we apply feature extraction and semantic segmentation at frame level and then pool each feature map using different segmentation regions. This is followed by 3D convolution over feature activa-tion map, with attention applied to each region, to better encode the temporal information in the video sequence.
Below, first we describe Feature Extraction and Semantic Segmentation Modules followed by videobased re-ID .
Feature sequence extraction module: In our model, we adopt Inception-V3 [37] as a backbone of our feature extraction module. Given a video sequence X ∈ R H×W ×3×S of length S, where each frame (RGB) is of size H ×W , each frame in video at a time-step is passed to Inception-V3 to produce the frame-level feature maps. At the top of the Inception-V3, using bilinear interpolation, the feature maps are scaled up to the size of the segmentation maps.
Semantic segmentation module: For the segmentation module, as in the feature extraction module, we use the Inception-V3 architecture. However, we make two important modification to this architecture compared to the one used for feature extraction. The first modification is that the output stride of the model is reduced from 32 to 16 to get the feature maps with adequate resolution for semantic segmentation task. The extra computation cost resulting from this modification is eliminated by the replacing the convolutions in the last Inception block with the dilated convolutions [47]. The second modification is the use of atrous spatial pyramid pooling [5], instead of global average pooling to exploit multi-scale information. Refer to [14] for more details about the feature extraction and semantic segmentation modules.
Video-based person ReID model: Given a video sequence, each frame in a video at a time-step is passed to an Inception-V3 to produce the frame-level feature maps. Simultaneously, our semantic segmentation module is fed the same sequence, which generates segmentation (probability) maps of 3 semantic regions for each frame in the sequence. We pool the output of our feature extractor model using the 3 segmentation maps. This results in a set of 3 maps, each focusing on a different body region of the sequence. Next, to encode the temporal information, we pass the activation sequence of each region to 3 consecutive 3D convolutions that do not share the weights. As a result of this encoding process, a tensor is obtained for each body region and feature vectors are constructed using the Global Average Pool-  1  12  23  34  45  56  67  78  89  100  111  122  133  144  155  166  177  188  199  210  221  232  243  254  265  276  287  298 Rotation -X   Figure 3: Proposed Network: First, the input video clip consisting of S frames are input to the Inception-V3 models, the feature maps and 3 segmentation maps for each frame (f-1 to f-S) are generated in the upper and lower branches of the model, respectively. We pool the feature maps using the segmentation maps and obtain sequences of 3 feature maps, each focusing on a different body regions: lower, upper and full. Next, we aggregate the temporal information within these sequences utilizing 3D convolutions, which operate with the stride of 2 and with the kernel size of 3 for the temporal 1 for the spatial dimensions. With this encoding process, we obtain a tensor of size (30x30x2048) for each body region and apply global average pooling (GAP) to construct the feature vectors. In addition to these vectors, we construct another feature vector taking the average of the output feature maps of Inception-V3 (top branch) and applying GAP on the resulting tensor.
ing (GAP) on these tensors. We also use one more feature vector, constructed by taking the average of the feature maps produced in the feature extractor module and applying GAP on it. During the training of the model, we pass the 3 feature vectors of the body regions to different soft-max layers that do not share the weights and compute the classification losses separately. We do not use the global features while training the model. At the test time, we concatenate all region and global features to get the final descriptors for the input videos.

Improving ReID Employing Sensor
Meta-data In this section, we discuss the proposed method for employing sensor meta data information to further help refine our re-identification results. In particular, we use the heading, speed and GPS (longitude and latitude) of the camera, which are captured by our recording devices.
Let T a i represents an observation of a person i in camera a. Its trajectory is given by a set of points where t s and t τ represent the time it entered and exited camera a, respectively. ρ(T a i ) denote appearance feature representation of tracklet T a i generated by the proposed model. GPS and heading of camera a at time t τ are respectively represented by C a g t τ and C a h t τ . The heading of a target i in camera a, T a h i , is inferred from the camera a s heading at time t τ , that is, if target is moving in the same direction as camera (target is walking away from the camera), then T a h i = C a h t τ . Where as, if a target is moving towards the camera, like the target in Fig. 4, then T a h i = C a h t τ + 180 0 , that is, we assign the opposite heading w.r.t the camera's heading. The direction of target's movement is determined from the tracklet direction.
Estimating the next camera: Naïvely, one can select the closest camera to the target at time, t τ , as the next camera without considering the target's direction of motion. As can be seen from the example in Figure.4, such approach will end up selecting the wrong camera, which is located on the other side of the target. So, first, we need to select candidate cameras which are spatially located in the direction of target's heading. Then, we select the closest camera among those based on their GPS distance from the The black doted lines shows the distance between camera a and the other two cameras. The green boxes and arrows respectively represent a target's tracklet and its heading. In this particular scenario, its evident that camera j should be selected as the next camera, since the target is heading towards it, even though camera m is closest camera to the target. And the estimated time is computed using the first case of eq. 1, as both camera j (next camera) and target have the same heading. target at time t τ . In this set up, we not only select the closest camera but also we ensure the next camera is located along the way of target's motion. Lets define a function H(GP S 1 , GP S 2 ), which returns heading angle between two GP S points. In particular, it determines which direction GP S 2 is located w.r.t GP S 1 and is computed as follows: where, X = sin(λ 2 − λ 1 ) * cos(ϕ 2 ) and Y = cos(ϕ 1 ) * sin(ϕ 2 )−sin(ϕ 1 ) * cos(ϕ 2 ) * cos(λ 2 −λ 1 ). (λ 1 , ϕ 1 ) and (λ 2 , ϕ 2 ) represent tuple of longitude and latitude of GP S 1 and GP S 2 , respectively.
For a camera, C m , to be in a candidate set, K, of target T a i , the following should hold: . That is, for a camera C m to be selected as a candidate, its heading w.r.t the location of current camera of the target, C a g t τ , is the same as target's heading T a h i . Finally, the next camera for target i in camera a is estimated by selecting the closest camera at time t τ among the candidate cameras, and is given by is GPS of target i of camera a at time t τ and due to very close proximity of the cameras to the targets, we can approximate target's GPS by the camera a s GPS at time t τ , C a g t τ . Estimating time of arrival: Next, we estimate the time required to travel between camera a (current camera) and j (the next camera). Since our cameras topology is very dynamic (due to camera motion), we implicitly compute the time as a division of distance and speed. The speed of a target, T a v i , is estimated to be 1.3 m/s, which is the average walking speed of individual [32]. While the speed of camera j, C j v , can be inferred from the sensor metadata data. The distance between the next camera, C j , and the target is computed between their GPS. Formally, the time required by target, T a i , to reach camera j from camera a can be computed as follows: where T a h i is the heading of target i at the time of exit, t τ , from camera a and is inferred from the heading of camera a at time t τ .
Note that in eq. 1, since both target and camera are moving, time computation depends on the direction of their motion. If both are moving in the same direction (i.e. same heading), first case in eq. 1, then we divide the distance by the difference of their speeds, otherwise, we estimate the time by dividing the distance with the sum of their speed.
Finally, we impose our time constraint on our appearance similarity. The final affinity between query target, T a i , and the rest of tracks in camera j is updated as follows: where ψ(ρ(T a i ), ρ(T j l )) is appearance similarity between two tracks features, T j l,t s is an entrance time of track T j l and |T j | is total number of tracks in camera j. As can be inferred from eq. 2, during ReID, we only compare track T a i with tracks in camera j which appear after the estimated time of arrival of track T a i at camera j, E T a i ,C j . Thus, we can significantly reduce our search space by pruning several wrong matching which violate our time constraint.

Datasets
In addition to the EgoReID, we present evaluation on widely used, fixed cameras dataset, MARS [52]. It consists of 6 cameras and 1261 different pedestrians. There are 625 identities for training and 636 identities for testing.
Evaluation settings: Training/Testing split of EgoReID dataset contains 567 identities for training and 309 identities for testing. For MARS dataset, we follow the same setting as the authors of the dataset. To evaluate performance for each algorithm, we report the Cumulative Matching Characteristic (CMC) metric and mean average precision (mAP).
Training the Network: We first train the semantic segmentation module of our proposed model using Look Into Person (LIP) [10] dataset. We then train our feature extractor module using the probability maps produced with segmentation module. Since only 3 semantic regions are used in the model, we group the probability maps of different regions to create the probability maps for foreground, upperbody and lower-body. During the training of the feature extractor module, following the same settings mentioned in [14], we use a training set consists of 10 image-based re-identification datasets. We use a training set, which is constructed by the aggregation of 10 image-based re-identification datasets. Next, we fine-tune the feature extractor by image-based training using the images from the video datasets (MARS or EgoReID). In the last step of the training, we freeze feature extractor and train 3D convolution layers using video clips of 15 frames. As mentioned in Section 4, there are three consecutive 3D convolution layers for each region. Each of these layers produce a tensor of size (30 × 30 × 2048), by encoding the temporal information with the kernel size of (1 × 1 × 3) and with the stride of 1 and 2 in spatial and temporal dimension, respectively. We construct the feature vectors from these tensors applying GAP and compute the loss for each region by performing multi-class classification separately. Finally, the loss of the model is obtained by summing the three losses. More implementation details are provided in the supplementary material.
Ablation study of our approach: We investigate the effect of each component of our model by conducting several experiments. In Table. 1, we show the results of each component in the proposed network. We evaluate the effects of each body regions and sensor meta data. As can be noted from Table. 1, lower body features perform worst on EgoReID, while they achieve reasonable result on MARS. This is mainly due to the fact that in EgoReID dataset lower body is frequently missing as the camera is very close to the target. We can also observe from Table. 1 that, jointly using features from different body regions, leads to improvements in the performance than using only one body region.
We can also see that the proposed approach for using sensor information to prune several unreliable matching pairs, significantly improves the ReID performance of our approach on EgoReID dataset. As shown in Table. 1, we are able to improve rank-1 and mAP by around 15% and 12%, respectively.  To further show the effectiveness of the proposed model, we compare our results with [14], where we apply their method on frame by frame bases and then employ average pooling over temporal dimension to generate features from each region. As can be seen from Table. 2, our approach gives significantly better results in both rank-1 and mAP (i.e. 20% and 34% respectively). This shows the effectiveness of the proposed 3Dconv module in encoding the temporal information.  Table 2: Comparison of SPReID [14] and the proposed approach on MARS dataset.

MARS
Comparison to state-of-the-art Methods: In Table. 3 and Table. 4, we compare our approach against the state-of-the-art methods on EgoReID and MARS, respectively. As can be observed from Table. 3, our approach significantly outperforms state-of-theart approache on EgoReID dataset. This shows the effectiveness of our human semantic region based local feature extraction approach on our dataset. This is mainly due to different body parts of pedestrians being clearly visible in EgoReID. In Table. 4, we observe that our method has competitive r-1 results, while it has better results in r-5 and mAP. This further demonstrate the robustness of our approach in handling different domains of video.

Conclusion
In this paper, we presented a new EgoReID dataset which is captured using 3 mobile cellphones with non-overlapping FOV. EgoReID presents substantial variations in lighting, scene, background, human pose, etc. Compared with existing video-based ReID datasets, EgoReID poses unique and realistic challenges to person ReID task. Moreover, we have recorded 12 sensor meta data for each video. We have also proposed a novel approach which employs human semantic parsing to extract local discriminative features and compare corresponding semantic regions to find a match. Experiments conducted on MARS and EgoReID showed the effectiveness of our approach in different video domains. In addition, We have also successfully employed sensor meta data information to determine target's next camera and its estimated time of arrival, thus, we only search for a target in the predicted camera around the estimated time of arrival. This significantly improved our ReID performance by reducing our search space. Fig. 5 shows sample tracklets from each camera. Each row contain tracklets of the same person, while each column represent different camera. As can be noted from fig. 5 , our dataset contains several illumination, pose and background changes both within and across cameras. Fig. 6 and 7 provide more detailed statistics on our dataset. For instance, camera 3 captures the most IDs and tracklets. Camera 2 captures the second most IDs but ranked third in total number of tracklets it contain.

C. Qualitative Results
Person re-identification: In Fig. 16 -17, given different example queries from EgoReID dataset, top 10 retrieved tracklets from the gallery are shown for different methods. In all examples, the first row is a result from Inception-v3 where we extract frame level global features and perform average pooling for each tracklet. The second row represent results where we apply [14] on every frame in a tracklet and apply average pooling along the temporal dimension. While the third and fourth rows show results of our method before and after employing sensor meta data information. Correctly matched tracklets are shown with green border while the incorrect ones are show in red. To avoid clutter, we only show three frames per tracklet.

D. Implementation Details
We train the feature extractor module for 200K iterations using the training set consists of 10 imagebased ReID datasets (Market-1501 [53], CUHK01 [21], CUHK02 [20], CUHK03 [22], DukeMTMC-reID [56], 3DPeS [1], PRID [13], PSDB [44], Shinpuhkan [15] and VIPeR [12]), then fine-tune it on the video datasets for 20K iterations. The initial learning rates for these two processes are 0.01 and 0.001, respectively, and we decay them 10 times with the rate of 0.9 by utilizing the exponential shift. Similarly, we use 0.001 as the learning rate while training the 3D convolution layers and decay it 10 times. While training the feature extractor, the batch size, momentum and weight decay are 15, 0.9 and 0.0005 for the respective values. To train the 3D layers, we set the batch size to 6 and the length of the clips to 15. The input images with the size of 512 × 170 are used to train both the feature extractor module and the 3D convolution layers.
We follow the same settings mentioned in [14] to train the semantic segmentation module. We set the initial learning rates to 0.01, 0.1 and 0.1 for the Inception-V3 backbone, atrous spatial pyramid pooling and the 1 × 1 convolution layer (classification layer), respectively, and decay them 10 times. The minibatch size is set to 8 and the input images with the size of 512 × 512 are used. The other settings are similar to the mentioned above. We perform the training of the re-identification and the segmentation models using Nesterov Accelarated Gradient [2] and used the pre-trained InceptionV3 models on ImageNet [33]. Figure 5: Sample tracklets from EgoReID. Each row corresponds to different identities, while each column represent tracklets from cameras 1, 2 and 3. (from left to right). As can be noted, in addition to changes in background, pose and illumination of the same tracklet across camera (each row), due to camera motion, different tracklets from the same camera (each column) are also captured with different background, illumination and poses. This makes our dataset challenging but more close the reality.


The Digital Transformation of Education

For society to beneﬁt fully from its investment in Earth Observation, the data must be accessible and familiar to a global community of users who have the skills, knowledge and understanding to use the observations appropriately in their work. Future ‘Environmental Data Scientists’ will need to draw on multiple data and information sources, using data analysis, statistics and models to create knowledge that is communicated effectively to decision-makers in government, industry, and civil society. Networks, cloud computing and visualization will become increasingly important as citizen scientists, data journalists and politicians increasingly use Earth observation products to give their arguments and decisions scientiﬁc credibility. The overarching aim of Earth Observation education must therefore be to support life-long learning, allowing users at all levels to remain up-to-date with EO technologies and communication mechanisms that are relevant to their individual needs. Current and emerging methodologies for interactive education (such as “MOOCs” and mobile learning), and hands-on engagement with real data (such as through citizen science projects) will be central to outreach, training and formal education in this ﬁeld. To achieve this, it will be important to engage a wider community of experts from a range of disciplines, and to establish a comprehensive network of educators, technical experts, and content producers. It will also be important to encourage “crowd-sourcing” of new contributions, to help maintain scientiﬁc and educational quality. A case study from the World Bank’s Open Learning Campus illustrates the opportunities to inﬂuence thinking much beyond the environmental data scientist community.

Environmental stewardship has "gone mainstream". In just a few short years, public understanding of the vulnerabilities of our environment has moved from marginal to a central part of policy, public discourse and education. The achievement of this tipping point has been so emphatic that there is now a growing and tangible demand, even a thirst, amongst the general population for greater knowledge about how the Earth system works and how to protect it.
EO has been at the heart of the creation of this societal imperative, from the cultural impact of the first Apollo images of the Earth, to the increasing familiarity of astronaut photography and high resolution video from LEO, distributed through social media, news outlets and mainstream media. But there remains a significant disconnect between this increased awareness of EO data, and a detailed understanding of its applications and what it can truly reveal.
To sustain this interest and increase the depth of public and professional understanding of the data, there is a need for new forms of education and training which can cut through a fast-moving and information-rich world. A new generation of decision-makers, social entrepreneurs, educators, media professionals and active citizens are looking to equip themselves quickly and efficiently with deployable, practical knowledge and skills, and the confidence to bring EO data into their work.
Simultaneously, the scientific research community, along with environmental and humanitarian organizations, is in need of a greater through-flow of skilled and knowledgeable data scientists and remote sensing experts. They require the tools to quickly share new ideas, techniques and practice, and provide a window onto emerging developments in EO across multiple disciplines.
Fortuitously, these needs have converged with key developments in a number of arenas which now make all of this possible. Advances and growth in online education, the "open educational resources" (OER) movement (Atkins et al. 2007), and interactive web technologies have coincided with the emergence of open EO data, easier access to high resolution imagery, and a thriving EO app development scene. This has made it possible to provide accessible but authoritative mass education and training and opportunities to work practically with EO data, to diverse audiences around the world.
There are challenges associated with configuring online learning in a way that accommodates a wide array of prior knowledge. Ensuring that the training provided is accessible enough for newcomers but also credible to those with some expertise requires a careful and detailed "learning design" process and clearly signposted pathways to further in-depth learning. But when this is done well, the outcome can be a powerful combination of professional development and academic learning for some learners, and an inspirational and awareness raising tool for new audiences.
A key initial step is to ensure that the learning design is linked to the desired learning outcomes. The process becomes complicated when the course modules target learners who have very different stocks of prior knowledge and different learning objectives, such as raising awareness, professional development or academic learning. A case study from the World Bank's Open Learning Campus (OLC) illustrating how these challenges could be met is included in this paper.
The synergies between the nature of EO education and training, and the growth of interactive online learning create great opportunities for both arenas. The results can often go beyond intended learning, outcomes, with an often emotional and empowered response from those understanding for the first time what EO data can reveal, which in turn may have knock-on societal benefits.
This has been greatly exemplified in a series of EO "MOOCs" (massive open online courses) which have been instigated by ESA. The first two ESA MOOCs (focused on climate monitoring and sensing in the optical realm respectively), along with a course from EUMETSAT (on ocean monitoring) have had over 30,000 registered participants in just 2 years, and further courses are planned. These courses have focused principally on data applications, but have also provided rudimentary training on the basis and history of satellite EO technology and instrumentation, and have acted as a hub for various data access and visualization tools to enable even novice learners to experiment directly with real data.
ESA's first full-scale MOOC, on "Monitoring Climate from Space", was a 5-week course deployed initially on the FutureLearn platform, consisting of a total of 30 original videos, interactive tests and exercises, and 3-5 h of independent study each week. The course attracted 9000 registrants on its first "run", with a participation rate of over 54% and a course completion rate of 22%, above average for large-scale MOOCs on FutureLearn. Over 80% of participants were classified as "active learners" (i.e. following through multiple steps of the course), and around 30% were classified as "social learners", actively involved in peer-learning and online comments. Additionally, the course achieved exceptional qualitative outcomes, with very high approval ratings for the format, quality and "layered" nature of the content, and very high levels of "emotional engagement" and stated intention to continue with further learning in the subject. Significant numbers of newcomers to the subject expressed a profound new appreciation of Earth science, the evidence for climate change and the detail of the data available through satellite observations. Others with existing knowledge of the policy and environmental contexts expressed a renewed intention to incorporate EO into their professional work and decision-making, and an interest in deepening their knowledge of the data and applications (  diplomats, policy-makers, senior decision-makers, school teachers, students (high school, undergraduate, postgraduate and home-educated), campaigners, media professionals, science communicators, retired individuals and casual learners. This demonstrated the potential of MOOCs and OER to address all levels of need in EO education and training, which can be broadly grouped as follows: 1. Students in higher education (including those on courses or in research in "adjacent" fields). 2. Professional development and knowledge transfer (for existing data scientists, and other professionals working with EO). 3. School-based education (especially high school, where links to the curriculum are essential-Early EO education can be seen as "capacity building" for EO skills, two generations hence). 4. Awareness-raising and orientation (for newcomers to the field, including decision and policy makers, professional communicators and educators, and the wider public).
All of these areas have a common set of challenges to which new forms of education and training are applicable, but they also have distinct problems which require specific considerations, as outlined below.
In general, EO data is inherently powerful and compelling once understood, but for many potential new audiences it feels inaccessible and highly technical. There are perceived barriers to entry which stem from the technical terminology used, the diversity of data types and the myriad of data portals. It is important to "personalise" the concepts of EO for new audiences and to make the content feel less "remote". "Emotional engagement" and the personal impact of EO imagery and data play an important part in establishing an "embedded" understanding of EO, and in creating a sense of responsibility for the issues the data reveal. In EO education, this level of engagement can be achieved through clear learning design and explanation, but also through high "production values" in the quality of apps, animation, video, infographics, data visualization and "story-telling".
For new and general audiences, OER-based approaches to EO education (especially those which are video-led such as MOOCs) show greatest added value when they serve to "demystify" the subject and increase the visibility of the data and its applications, (i.e. actually "showing" learners how EO technology and data applications work, what the data looks like, and how it can have impact, in science and "on the ground" through real-world case studies).
In the case of higher education, traditional modes of delivery are not well geared to the demands and pace of EO education and training, and radical changes online have already begun to occur which should be embraced by the EO arena. The basic model of higher education existing today was created in the eleventh century, operates on a nineteenth-century calendar and model, and yet is still expected to prepare students for life in the twenty-first century. Technology is now widely changing the game in higher education by leveraging the potential of "social learning". Rapid developments in information and communication technologies (ICT), and in particular the Internet (web 2.0), have enabled consumers of content to become producers of content, and share this content through social networking sites, blogs, wikis and virtual communities. These technology developments have enabled the emergence of a more participatory approach for open online distance education, especially in the form of MOOCs, and the use of more innovative learning materials (such as videos, audio recordings, interactive tests and collaborative exercises). A popular and early example of the MOOC was MIT's Open Course Ware initiative, which today provides open access to undergraduate and graduate-level materials and modules from more than 1700 courses. Other initiatives, such as the Open University, iTunesU and the Khan Academy, are also key precursors of Open Educational Resources (OER) (Mehaffy 2012; Diaz et al. 2013).
The term "MOOC" was first coined by Dave Cormier in 2008 for a course on "Connectivism and Connective Knowledge". The idea however took off in 2011 when Sebastian Thrun and Peter Norvig, at Stanford University, decided to offer the course "Introduction to Artificial Intelligence" free to anybody in the world. They sent out a single e-mail, and 160,000 people from 190 countries signed up. More than 23,000 students completed the course, and by the end of the course, only 30 of the original 200 Stanford students were still going to face-to-face classes. Out of the 23,000-plus course completers, 248 students had a perfect score in the course, and none of them were Stanford students ( MOOCs are widely regarded as being part of a new paradigm (and possibly revolution) in "democratization of a new type of education serving the needs of twenty-first century students", enabling higher education transition towards digital networked learning. The success of the MIT project led to the creation of Udacity in 2012, and inspired many universities to start their own OER and MOOC initiatives, such as EdX, Coursera, and FutureLearn, to deliver free educational material to millions of people across the world. While some await further evidence for the long term impact of MOOCs, it is apparent that the OER movement is well aligned with the learning expectations of the new generation of higher education students, and  (Smith & Casserly 2006) is particularly well attuned to fast-moving, visual and complex subjects such as EO (Mehaffy 2012).
The ESA MOOCs have in some respects built on the work of previous online EO education projects supported by ESA such as the Learn EO! initiative and the Bilko software. What is different about the MOOCs however is that rather than providing a set of resources tailored for particular target audiences, they are instead designed to offer a spectrum of depth-of-learning for wide ranging audiences. The conundrum for the learning designers has been in finding a way to create a broad offering, whilst responding to the increasing expectation of online learners for a personalized experience. In the ESA courses, a sense of personalization comes in two forms: firstly the "social learning" aspects (including online discussions with other learners and feedback from expert educators); and secondly the range of options available for self-directed learning beyond the core materials of the course.
Others have approached the challenge of personalization of MOOCs in alternative ways. For example, the World Bank Group, which has invested heavily in MOOCs and online learning to help support sustainable development goals, uses online courses as part of a "blended" education and capacity building approach. Their principal target audience is "mid-career" practitioners involved in development, and their aim has been to create a flexible, collaborative learning approach.
This has taken the form of the "Open Learning Campus" (OLC), which includes a combination of full-scale MOOCs, "bitesize" learning resources, short "e-courses" and some face-to-face training (both online and in centres). This has led to a shift in the capacity building training and education delivered by the World Bank online from 5% to 95% over 4 years.
The OLC has so far made available six MOOCs, 500 short courses and 5000 bitesize resources, and their own analysis has concluded that this has enabled 350,000 target learners to access structured education and training, equivalent to around 10,000 face to face training courses delivered in a conventional way.
The flexible array of learning tools provided by the OLC now also includes resources tailored for on-the-move and "just-in-time" mobile learning. They are also pioneering the use of AI to further personalize the learning experience based on prior courses and other preferences.
Over the last 10 years, substantial evidence has started to emerge about the impact of online and video-led professional development training and knowledge transfer in a range of sectors. It has been shown to be particularly effective in sectors where there is a need to break down professional "silos" and enable a crossdisciplinary approach to innovation. This could be said to be very applicable to the training and professional development needs in EO. MOOCs and other forms of structured OER provide an ideal way to quickly unlock critical information and impart know-how for both professional and casual learners.
Professor Richard Elmore from the Harvard Graduate School has described learning as "the conversion of information into knowledge". In fast-innovating fields such as EO, professionals are often operating at the very limit of their knowledge, using the latest innovation they are aware of. To help them incorporate new knowledge into their work, they often need to be literally "shown" latest information and how innovations are being applied elsewhere (Elmore 2015a, b). In the EO arena, the emerging big data revolution will present a challenge for organizing the vast amounts of new data into accessible information, but it will also be a corresponding challenge for EO education to convert that information into tangible, understandable and applicable knowledge.
In the case of knowledge transfer and general awareness raising of EO in other arenas, much can also be learned from the use of OER and video-led training in other areas of science. For example, the UK research funding body EPSRC (Engineering and Physical Sciences Research Council) have, like other science bodies, used video extensively to showcase the capabilities and facilities of their grant recipients to industry, in order to promote and foster new research collaborations. For instance, in 2010/11 EPSRC funded a major UK university, the University of Sheffield, to showcase its large-scale and interdisciplinary research activities, expertise and infrastructure through a series of over 40 short documentary films, accompanied by other forms of text-based information. Watched in their entirety, the videos (presented in six thematic areas relating to major global challenges), represent a form of ad hoc, on-demand training course. In practice, the videos were viewed by their target audience over 150,000 times, were shown at numerous professional and public events and had additional public engagement impacts. The project overall is believed to have led to many significant investments and funded research partnerships, and has been emulated with follow-up video projects of this kind (Research at Sheffield 2010). MOOCs and similar video resources could greatly help to raise the visibility of EO data and applications in this way, amongst policymakers, senior business executives and funders, effectively training them as "casual" learners.
MOOCs and OER also have a role to play in bringing EO education into high school-based learning, but the approach to how these tools are used has to be modified to take into account the "time poverty" experienced by many teachers, and the need to align resources to the curriculum. School-based education is generally still fairly formalized, and ease of access, in-depth curriculum links and teacher guidance are essential requirements for ensuring take-up in schools. On the other hand, there is often a willingness to embrace technology and create skillsbased and personalized learning. EO provides inspirational and compelling learning opportunities in the science, geography and "citizenship" curricula. Educational research has also established that giving students in schools the opportunity to engage directly with real scientific data, (and even contribute to real scientific research), has an extremely positive effect on student engagement, familiarity with key subject matter, follow-on educational interests, and attainment levels. Engaging with real scientific data also helps students to orientate themselves geographically, and provides them with a better understanding of physical aspects of the world around them, such as a sense of scale and rates of change of natural processes. Much of the subject matter in traditional science curricula relates to phenomena which are either too large or too small to be observed directly, so creating a sense of scale and enabling visual understanding of scientific concepts is key to in-depth learning. EO data has the potential to play a significant role in this (Kapur et al. 2007;Facer 2007). At higher levels, the concept can evolve and be scaled up to the Fablab experience, as originally born at MIT's Center for Bits & Atoms Fab Lab Program. In this case, as in the "Fabspace 2.0" project (2016) the main objective is to set up and operate at a university level a free access place and service where students, but also researchers and external people, can make use of a data platform and, under the open innovation umbrella, design and test their own geoinformation applications using EO images.
EO-related citizen science also offers substantial and wide-ranging opportunities in EO education, enabling teachers to explore EO data with their students in a structured manner, and embed it into curriculum delivery and formal teaching. Teachers and students generally relish the opportunity to engage directly with realworld data. It creates depth of learning and insights that are not possible otherwise, and maximises the prospect of students feeling they can make a contribution to science. And there are opportunities for EO education to emulate the successful open research collaborations taking place in other areas of science education. For example, in astronomy a "crowd-sourcing" approach has been established to enable student access to real science infrastructure, such as through the National Schools Observatory project in the UK, and the Faulkes Telescope initiative (Faulkes Telescope Project 2008; National Schools Observatory Project n.d.). Elsewhere, other projects use supercomputer simulation models, or enable school students to engage personally in research side by side with researchers, such as the LUCID project and other initiatives at the Langton Star Centre in the UK (LUCID Project/Langton Star Centre 2014). In lower age groups, EO education is well-placed to capitalize on emerging research into cloud-based education methodologies, such as the "Self-Organised Learning Environments" (or "SOLEs") being pioneered by Professor Sugata Mitra (Dolan et al. 2013).
New variations on the MOOC concept are already emerging which could further widen the potential uses of these approaches in EO education and training. For example, the MOOC tools and format can be delivered to more limited numbers rather than being "massive" (so-called "OOCs"), or distributed to large but private audiences rather than being fully "open" ("MOCs"). Several universities have experimented with the concept of "xMOOCs", where tailored courses are created from a centralized pool of OER and video content, and disseminated through a range of pathways rather than a single platform. Such an approach could be highly applicable to EO education, where content needs to be drawn from multiple agencies and research groups and courses need to be regularly updated. Others are creating courses where content is only accessed "on-demand", with learners encouraged to "re-mix" or "mash-up" the content and share it as they need. What remains essential however, is the need for consistency and clarity of learning design, ensuring ease of access to the content, and supporting opportunities for collaborative, social learning (Diaz et al. 2013;Brown & Adler 2012).
While MOOCs can offer a rich, open and flexible learning experience, they are in effect a new "instructional genre" (Mitchell et al. 2015), and we must use new metrics for assessing their impact and modify our expectations in evaluating success. To many learners using these new tools, "success metrics" and "completion" may not have the same meaning as in traditional education. In the case of EO especially, many learners will be happy to simply access the information they are most interested in, be exposed to new ideas and discover new applications, rather than seek formal accreditation or qualifications (Liyanagunawardena et al. 2014).
Techniques such as MOOCs also enable a two-way flow of insights, helping to inform trainers of what a particular group of learners knows and does not know about EO and the environment, and benefitting the scientific community through the fresh questions and challenges posed by learners. MOOCs include stimulus questions and discussions, and the participants are invited to discuss topics raised in the videos in whatever way they wish. For example, in the EUMETSAT Monitoring the Oceans from Space MOOC, many questions were asked about how the quality of the satellite data is assessed. The participants were using their own language to articulate questions of calibration and validation. These questions provide an indication of how the "story" of creating and managing data sets in science could be told in new ways. The participants seemed to have a real appreciation for the contribution of data validation to the scientific disciplines that rely on the data. As an important side benefit to the main aims of the course, the MOOC showed learners how this key part of the science works by teaching them about specific areas of the science itself.
MOOCs can also be effective in placing the learner at the heart of the science, enabling them to better appreciate the implications of EO data and applications for themselves and their communities. The ESA and EUMETSAT MOOCs have all been designed to highlight connections between the satellite observations, the resulting science and applications, and the impact and opportunities for the individual citizen. Providing tools for participants to create and share examples of their findings is an important element in the teaching process for these MOOCs. This mirrors the process for professionals involved in EO data provision and the efforts to improve the reach of EO data. By encouraging learners to find, obtain and share data with other learners and to comment on what they have learned and what has surprised them, they are more likely to understand why it is not enough to simply provide high quality data, in a stable format and on an easily accessible platform. This practical, interactive, peer learning aspect of the MOOC experience deepens their familiarity with the data and the need to provide tools to visualize, manipulate and communicate about the data for use by wider audiences. It also introduces them to the different ways in which different scientific communities engage with the data (e.g. in meteorology vs. oceanography), and the need for interdisciplinary approaches. In turn, observing how the learners engage with these challenges can provide further insights to inform the development of new interdisciplinary teaching and training.

Case Study: How the World Bank Group's Open Learning Campus is Partnering with Earth Observation Satellite Data to Enrich the Learning Experience
The World Bank Group is a key actor in addressing complex development challenges such as sustainable development, climate change and environmental problems. In this role the organization facilitates training, education and outreach. So too do Earth observation capacity building programs, wherein learners work to understand how sharing and visualizing geospatial data can lead to problem-solving and policy planning for development solutions. The overlap of the need for more real-world learning experiences with context-specific data has led to a partnership between the WBG and Earth observation community that is helping to transform development learning.

Learning as an Accelerator to Achieve Development Goals
The WBG has invested significantly in digital and blended learning, not only relevant to the Earth observation community, but also to a much wider global development community, including policy makers, practitioners, academics, NGOs and the general public. The WBG offers overall online education through its virtual platform, the Open Learning Campus (OLC), through which staff can refresh knowledge on a continuous basis, bridge knowledge gaps jointly with clients and co-create solutions to complex development issues, such as climate change disasters and managing epidemics.
Several drivers led the WBG to invest in the OLC as a go-to destination for development learning. Principally, it is to share this global institution's vast repository of knowledge with the wider development community and thereby transform it into a "Solutions Bank". Internally, such a platform allows for continuous learning on technical, operational and leadership topics to staff, enabling them to remain cutting-edge in their respective fields while also acquiring new competencies that improve their overall effectiveness in the workplace. It also enables uniform career development opportunities for staff, many of whom are in geographically dispersed time zones. Externally, Bank client countries and partners are interested in learning and co-creating solutions, leveraging the repository of development knowledgeespecially tacit knowledge-available in the institution.
At the same time several hugely and positively disruptive changes are taking place in the way education and learning is delivered. With close to six billion mobile technology connections projected by 2020 and with about 50% of the world using the Internet 1 , the world is increasingly connected and adept at making use of that connectivity for not only entertainment and communication but also for bringing learning tools available any time and at any place. A lot of content is found online through publicly available open educational resources (OER) like Khan Academy's flipped classroom, information downloads possible from Wikipedia, or YouTube. Vast networks of peers are available to respond to queries through Facebook, Snapchat and WhatsApp. Open learning through MOOCs has created learning communities, engaging millions of active learners worldwide. Much of this digital education in developing countries is mobile first, a by-product of the fact that computer ownership is rare for much of the developing world. Digital education is growing 14 faster than traditional higher education 2 , suggesting the financial motivation for further change. The increased ability to leverage big data has greatly enhanced learning analytics to understand the contexts in which learning takes place, so that online learning platforms customize learning to meet the specific development needs of learners.
The OLC is converting more and more of WBG's knowledge products and flagship reports (such as the World Development Report and country and regional technical reports) into easily accessible learning products. It is also unlocking tacit knowledge available with many of its staff, clients and partners and packaging this information into absorbable content. Rapid advances in pedagogy and technology have made these innovations possible.

Progress of the OLC so Far
The WBG has historically had a culture of face-to-face learning that is expensive and limited to small numbers of participants. It wanted to move learning away from being an "event" to becoming deeply embedded in its work. As the WBG was shifting to become more knowledge-enabled, it explored how best to deliver knowledge in the form of learning at scale and at significantly lower unit cost. The first foray into e-learning was the establishment of the e-Institute in 2011, which delivered a proof of concept for what later became the OLC. The e-Institute allowed WBG to test client and staff responsiveness and to demonstrate e-learning as a viable tool to enhance the knowledge and skills of development practitioners who were geographically dispersed.
Encouraged by the results of the e-Institute, WBG launched the OLC in January 2016 with the objective of becoming the preferred destination for development learning. Since the launch, more than two million learners from 190 countries have visited the OLC, and approximately 6000 digital learning activities have been designed and curated, gradually starting to realize the vision of the OLC. In the years to come, the OLC will continue to convene, connect, contextualize and cocreate learning for and with our clients and development partners, making the OLC not just a destination for just-in-time content but perhaps even more importantly, a destination for problem-solving conversations and communities.

Flexible Pathways to Learning
The OLC virtual platform leverages advances in pedagogy and technology to provide distinctive formats to match the evolving needs of learners through structured courses, bite-sized lessons and vibrant communities of practice. The OLC harnesses educational technology, mobile capabilities, cloud-computing and innovations in elearning to produce a world-class learning ecosystem through three "schools" of learning, with robust content, exciting delivery methods and access from anywhere.
The first school, WB Talks, enables the learner to explore nuggets of knowledge through talks, podcasts, videos and games. These bite-sized learning opportunities are equally useful for the busy professional and the inquisitive researcher or student. The second school, WB Academy, helps the learner to unpack deep learning related to development challenges and solutions through virtually facilitated or selfdirected e-courses, MOOCs and materials from face-to-face courses. The third school, WB Connect, enables learners to engage with others through peer and expert learning to find crowd-sourced solutions to development challenges. Using a range of tools, from social media to mobile texts, knowledge exchange can be promoted between WBG staff, clients, partners and global citizens. Examples of such communities include urban floods, Smart Cities and integrated water management and land and house thematic groups.
The OLC provides a rich library of learning courses and content across all sectors and regions that is curated from operational groups within the WBG and includes external global knowledge. The learning focuses on best practices and lessons from failures and is facilitated by global and regional experts on development topics from the WBG and the broader development community. The OLC also facilitates opportunities to learn from other client experiences (described as "South-South").

Increasing Importance of Geospatial Data
Various layers of geospatial data, ranging from continuous feeds from Earth observation satellites to geo-coded information on specific locations made available through sensors, are now retrievable and used for development policy and planning applications. Adding a geographic or spatial dimension gives the development learner greater authenticity in terms of the learning experience, while data visualization enables learners to experience realistic simulations by contextualizing information.
For example, learners are able to track air pollution in congested road arteries by the time of the day, send feedback through their mobile phones, and join discussions with traffic planning experts on the best ways to mitigate congestion and air pollution in the streets they travel in. Public participation in planning new facilities in a city, such as a park or shopping mall, could benefit from citizens who are able to visualize spatial outcomes of different proposals under consideration.

Examples of How the OLC is Incorporating Spatial Data
MOOCs are powerful vehicles to distil and share knowledge among various stakeholders, ranging from policy makers to the general public. The OLC offers the MOOC, From Climate Science to Action, thrice in English, twice in Spanish and now under finalization in French. The course guides learners through the latest research that forms the basis for predictions around the effects of climate change on their lives. It explores the projected effects of climate change across sectors, with in-depth analysis of each world region through a series of region-specific modules. Understanding how levels of carbon emissions today will affect the world tomorrow galvanizes public support for political action.
Themes from two European Space Agency (ESA) MOOCs have been incorporated into the WBG climate change MOOC. Videos from the ESA MOOC, "Monitoring Climate Change from Space," have been used to explain Earth observation concepts, how climate data from satellites are valuable inputs in policy formulation and the importance of Earth observation megatrends in wind patterns, ocean currents, etc. for policy applications and promoting global dialogues. Videos from the ESA MOOC, "Earth Observation: The Optical View," have been used to help learn about causality processes in climate change and appreciate the impacts in terms of changes in land use, forest cover, etc.
A second vehicle the OLC offers is facilitated e-learning courses. A series of facilitated e-learning courses in disaster risk management have considerable inputs from Earth observation satellites. An introductory course familiarizes learners with contemporary concepts and practices in disaster management and explains how planning approaches have moved from reactive to pro-active modes. Another course on safe and resilient cities provides urban planners with tools on vulnerability assessments, policies and programs, stock-taking and gaps analysis and financial access. The discussion also focuses on decision-making in uncertainty to identify options and establish priorities, making the city more resilient to climate change impacts and natural disasters. The seven modules of the "Sustainable Urban Land Use Planning" course give the learner a thorough grounding in best practices in urban land use planning and how to sequence regulations and investments to mitigate disaster risks.
A third vehicle is bite-sized learning modules that provide just-in-time learning that complements the above two categories in bringing about meaningful change across all levels-from grassroots action to national policy making to international cooperation. A good example is the Spatial Agent Tutorial, which teaches how to curate data from hundreds of data sets to create on-demand visualization on wideranging topics of interest to the Earth observation satellite community, ranging from transport and trade to watersheds and climate change. Bite-sized learning on "Bits and Bricks" focuses on cities undergoing profound transformations, and on how the convergence of digital information (bits) and physical environment (bricks) are facilitated through the Internet to develop new solutions. In support of this initiative, the OLC has prepared a series of bite-sized learning videos that benefited from the Bits and Bricks conference organized by the WBG, with MIT and World Economic Forum.

Issues Relevant to This Experience for the Earth Observation Community
Key issues and messages gathered from deploying training and education in the OLC so far include: • Transformation in learning: Learning is an important accelerator to eradicate poverty and boost shared prosperity whenever global knowledge is converted into practical actions through evidence-based and iterative learning. The OLC transforms how its target audiences learn. It integrates innovations in technology and instructional design to provide accessible, quality learning in a resource-efficient way. Emerging immersive learning modalities facilitate this transformation, by accelerating learning adoption and application. Examples include serious games, geospatial data, artificial intelligence and mobile-and cloud-based learning. • Role of Earth observation data: The OLC provides a unique opportunity for the Earth observation community to disseminate their professional insights to the wider development community for cross-sectoral, "anytime and anywhere" learning. Spatial and Earth observation data are becoming increasingly important in several areas of WBG engagement, notably related to climate change, infrastructure service planning and delivery and citizen engagement because the costs of installing sensors are falling dramatically. • Digital changes and connectivity: Global connectivity is increasing, leading to a "Facebook nation" of more globalized, coordinated, economic, social and political actions across borders. As WBG and its partners tackle complex challenges, many staff members and officials need to learn new skills to leverage the massive expansion in data available-not only from Earth observation satellites but also from mobile call records and sensors on the ground and under water. This data will be valuable only when it can be applied to policy formulation and project design. • Wider dissemination: Partnerships are emerging with regional and national institutions and donors (such as the Korea Development Institute, Monterrey Tec, Chinese Academy of Governance, Indian National Institute of Urban Affairs) to localize courses and scale up in-country. The know-how can be disseminated globally, with the OLC playing a facilitator role. • Citizen engagement: Influential NGO networks are also participating and contributing to the OLC, so that citizens have the opportunity to acquire the necessary knowledge and skills, and burgeoning citizen movements are persuaded to support the right kind of reforms.

Case Study Conclusion
The OLC represents the WBG commitment to harnessing the global revolution in learning to lift people out of poverty and to boost shared prosperity. Adding the geographic or spatial dimension to the OLC's learning offerings is invaluable because it provides greater authenticity to the learning experience, through big data analytics with data visualization. These types of data enable learners to undertake realistic simulations of complex policy choices and will contribute to more effective decision-making.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made. The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.


New Model of Information Technology Governance in the Government of Gorontalo City using Framework COBIT 4.1

This study aims to determine the level of maturity of information technology governance in Gorontalo city government by applying the COBIT framework 4.1. The research method is the case study method, by conducting surveys and data collection at 25 institution in Gorontalo City. The results of this study is the analysis of information technology needs based on the measurement of maturity level. The results of the measurement of the maturity level of information technology governance shows that there are still many business processes running at lower level, from 9 existing business processes there are 4 processes at level 2 (repetitive but intuitive) and 3 processes at level 1 (Initial/Ad hoc). With these results, is expected that the government of Gorontalo city immediately make improvements to the governance of information technology so that it can run more effectively and efficiently.

Introduction
Information technology (IT) is one of the important aspects of governance organizations as a whole. The success of the organization will be greatly influenced by its ability in utilizing IT optimally [1]. IT governance is a procedure directive implementation of organizational arrangements to support the management of IT integrally and follow up the goals and strategies of organizations that have responsibility [2]. Another definition of IT governance is the decision making that ensures the allocation of IT usage in the organization's strategies [3].
IT governance at government institutions is critical to ensure the successful use of IT in supporting the performance of government institutions, as mandated by Presidential Instruction No. 33 of 2003 on "National policies and strategies for e-government development" [4].
To create a progressive and competitive government, Gorontalo City has a great vision "The implementation of the excellent service of Gorontalo city government to realize the prosperous, progressive, active, religious and educated of Gorontalo". In line with that vision, one of the mission related to IT field is to increase the availability of IT infrastructure in urban area by opening cyber city and techno park area, as well as organizing IT-based government service program [5].
At the implementation level, the use of IT in governance needs to take into account the limitations of resources such as data, technology, facilities, and human resources including the fact that IT is relatively costly in its utilization. Limited resources is then an important factor the need for IT governance governing the use of IT in government organizations. IT governance in the process of data management is less good will cause some problems that are weaknesses that will cause threats such as the occurrence of loss, destruction, theft and tapping of important data company or organization. Based on that, it is necessary to do an analysis of IT needs for the government of Gorontalo city through the measurement of maturity level of IT governance so that it can be known the level of IT governance maturity that runs in Gorontalo city government institutions. In measuring the maturity level of IT governance it will be used COBIT (Control Objective for Information and Related Technology) which is a framework that provides a standard within the framework of a domain consisting of a set of IT processes that represent controllable and structured activities [6] [7] [8].
COBIT is an integrated framework for alignment between IT resource management and organizational objectives with the direction of integration and risk associated with IT and is an IT governance management concept addressed to management, IT service staff, departmental control, audit functions and business process owners for confidentiality, integrity and availability of sensitive and critical data and information [9] [10].

Method
This research method using case study method by conducting survey, collecting data through questionnaire questionnaire and interview related support of information technology service at government institutions of Gorontalo City. There are four stages in this research that is 1). Surveying and data collection, 2). Preparation of Information Technology Governance Framework, 3). Measurement of Information Technology Governance, 4). Information Technology Governance Analysis.

The Result of Compilation of IT Governance Framework
Referring to the COBIT standard 4.1 the scope of information technology governance is divided into several stages of the process: Identification of Business Goals, IT Goals Identification, IT Process Identification and Identification of Control Objectives. Referring to figure 1 on the linkage between COBIT business goals and strategic goals the government of Gorontalo City can be seen from four perspectives that exist, that the priority of business target relationship is in the perspective of service and human resource. This indicates that the government of Gorontalo City wants to improve the service to the society and want to increase the capacity and competence of the government employs to create good and quality governance. Learning and Growth 17 Acquire and maintain skilled and motivated people 9

IT Goals Identification
Based on the results of the mapping in Table 1, the COBIT IT goals identified in the Gorontalo City Government as described in table 2. the following : Ensure satisfaction of end users with service offerings and service levels 4 Optimize the use of information 5 Create IT agility 7 Acquire and maintain integrated and standardized application systems 8 Acquire and maintain an integrated and standardized IT infrastructure 9 Acquire and maintain IT skills that respond to the IT strategy 10 Ensure mutual satisfaction of third-party relationships 12 Ensure transparency and understanding of IT cost, benefits, strategy, policies and service levels 13 Ensure proper use and performance of the applications and technology solutions 18 Establish clarity of business impact of risk to IT objectives and resources 19 Ensure that critical and confidential information is withheld from those who should not have access to it 20 Ensure that automated business transactions and information exchanges can be trusted 21 Ensure that IT services and infrastructure can properly resist and recover from failures due to error, deliberate attack or disaster 22 Ensure minimum business impact in the event of an IT service disruption or change. 23 Make sure that IT services are available as required 24 Improve IT's cost efficiency and its contribution to business profitability 26 Maintain the integrity of information and processing infrastructure 27 Ensure IT compliance with laws, regulations and contracts

IT Process Identification.
At this stage, the results of the identification of IT processes selected in accordance with the IT Goals applicable in the Government of Gorontalo City.

The Result of Measurement of Maturity Level from Selected IT Process
The calculation of the maturity level of the selected IT process is done gradually for each IT process by using COBIT 4.1-Maturity Tools. Table 4. Below shows the weight of each level of maturity Furthermore, for each selected IT process will be presented in Table 5 which is the result of maturity level calculation on IT governance that runs on the government of Gorontalo City. Based on table 5 it can be seen that the value of maturity level of each selected IT process is mostly at level 2 i.e. there are 4 processes (PO4, PO7, AI3 and DS7). Which get the highest level (level 3) that is on process of PO1 and AI4. While there are 3 processes at the lowest level (level 1) i.e. the process DS1, DS4 and ME1.

Discussion
In this research produce a measurement value of maturity level of IT governance at government of Gorontalo City. Acquisition of the value of this maturity level refers to the application of the COBIT 4.1 framework. Referring to Gap maturity level produced, the government of Gorontalo City should be willing to cover the gap of the existing gap so that all IT resources owned can be managed optimally, and all business processes in the government of Gorontalo City more effective and efficient, accordingly with strategic objectives that have been established through the vision and mission the government of Gorontalo City. The following recommendations are given from this study to be improved by the government of Gorontalo city so as to increase the value of maturity.


extfieldsofstudy:
Engineering
Computer Science

text:
Robust universal neural vocoding

This paper introduces a robust universal neural vocoder trained with 74 speakers (comprised of both genders) coming from 17 languages. This vocoder is shown to be capable of generating speech of consistently good quality (98% relative mean MUSHRA when compared to natural speech) regardless of whether the input spectrogram comes from a speaker, style or recording condition seen during training or from an out-of-domain scenario. Together with the system, we present a full text-to-speech analysis of robustness of a number of implemented systems. The complexity of systems tested range from a convolutional neural networks-based system conditioned on linguistics to a recurrent neural networks-based system conditioned on mel-spectrograms. The analysis shows that convolutional neural networks-based systems are prone to occasional instabilities, while the recurrent approaches are significantly more stable and capable of providing universalizing robustness.

INTRODUCTION
Statistical parametric speech synthesis (SPSS) has seen a paradigm change recently, mainly thanks to the introduction of a number of autoregressive models [1,2,3,4,5], turning into what could be called statistical speech waveform synthesis (SSWS). This change has closed the gap in naturalness between statistical text to speech (TTS) and natural recordings, maintaining the flexibility of statistical models.
In the case of traditional vocoding [6,7,8,9], approaches commonly relied on simplified models (e.g. source-filter model [10]) that were defined by acoustic features such as the voicing decisions, the fundamental frequency (F0), melgeneralized cepstrum (MGC) or band aperiodicities. The quality of those traditional vocoders was limited by the assumptions made by the underlying model and the difficulty to accurately estimate the features from the speech signal.
Traditional waveform generation algorithms, while capable of generating speech from their spectral representation such as Griffin-Lim [11], are not capable of generating speech Corresponding author email: truebaj@amazon.com. Paper submitted to IEEE ICASSP 2019 with acceptable naturalness. This is due to the lack of phase information in the short-time Fourier transform (STFT).
Neural vocoders are a data-driven method where neural networks learn to reconstruct an audio waveform from acoustic features [1,2,12]. They allow us to overcome the shortcomings of traditional methods [13] at a very significant cost in computation power and data requirements. However, due to sparsity (it is unlikely that we will ever be able to cover all possible human-generated sounds) the neural vocoder models are prone to over-fit to the training speaker characteristics and have poor generalization capabilities [14]. Several recent studies attempted to improve the adaptation capabilities of such models [15,16], commonly using explicit speaker information (either as a onehot encoding or some other form of speaker embedding) [17]. There are however reports in literature of initial successes training neural vocoders without providing explicit speaker information [18], however the investigation did not cover the details of the robustness of this model to changes in domain and to unseen speakers. This paper makes the following contributions: 1) it demonstrates that a speaker encoding is not required to train a top quality speaker-independent (SI) neural vocoder; 2) quality with a SI neural vocoder is as good, if not better than, that of a speaker dependent (SD); 3) with the SI neural vocoder we can synthesise with speakers that were unseen in the training data, which is not possible with vocoders trained with explicit speaker information. The SI vocoder in this paper appears to be very effective at reading mel spectrograms and producing the corresponding speech waveform, without requiring any supplementary information, behaving as a universal vocoder.

SYSTEMS DESCRIPTION
Even though CNN-based systems have been thoroughly researched and real-time implementations have been proposed [4,19], it is known that they are prone to instabilities [20] which occasionally affect perceptual quality. RNN-based systems, on the other hand, can be expected to provide a more stable output due to the persistence of their hidden state, at least for the vocoding task in which context is not critical beyond the closest spectrogram frames (a known characteristic of RNNs). Table 1 overviews the systems. The linguistic RNN system was not up to par, so it was discarded.  [2], only with minor changes in the conditioning network) is described in Figure 1. The autoregressive side consists of a single forward GRU (hidden size of 896) and a pair of affine layers followed by a softmax layer with 1024 outputs, predicting the 10-bit mu-law samples for a 24 kHz sampling rate. The conditioning network consists of a pair of bi-directional gated recurrent units (GRUs) with a hidden size 128. The mel-spectrograms used for conditioning the network were extracted using Librosa library [21], with 80 coefficients and frequencies ranging from 50 Hz to 12 kHz.
We trained system RNN MS in 4 different configurations, whose details are shown in Table 2.1. First three SD systems were trained on American English speakers, two female (F1 and F2) and 1 male (M1) from our internal corpora.
We also trained 3 multi-speaker vocoders, one with all the training data from the 3 SD voices (3Spk), another one with 7 American English speakers (7Spk) comprising 4 females, 2 males and 1 child but with restricted amounts of training data per speaker (5000 utterances). This 7Spk neural vocoder aims to check whether variability or data (i.e. 3Spk) are more important for robustness in general. Finally we trained what is introduced as our universal neural vocoder with 74 different voices, in a 22 males and 52 females, extracted from 17 languages, with approx. 2000 utterances per speaker. This neural vocoder was designed with the expectation of being generalizable to any incoming speaker regardless of whether it was seen during training or not. Multiple (17) Early implementations of the universal neural vocoder explicitly included speaker information via a variational encoding of the input spectrograms to handle the inherent variability in the training data better, it proved to be unnecessary. Such encodings did not provide any naturalness improvements, which can be explained by the mel-spectrogram already containing sufficient information about the speaker, making it redundant in our scenario.

CNN models
The trained CNN-based models follow the topology introduced in [5] and were trained only on a single speaker, F1. The only variations from those in [5] are the input to the model (linguistic features as used in [5] or mel-spectrograms as an addition in the present paper) and the structures of the CNN layers. The mel-spectrograms and audio specifications were analogous to those of the RNN models.
The CNN-based approaches implemented were: 1) an end-to-end linguistics-conditioned system (CNN LI, the system evaluated in [5]), 2) a vocoder (CNN MS) and 3) a smaller vocoder (SCNN MS). The purpose of training a smaller CNN vocoder was to increase inference time by reducing model complexity.

Naturalness analysis
To properly characterize the generalization capabilities of the different vocoders in terms of naturalness we considered a number of scenarios. First of all a topline scenario in which we generated speech from speakers present in the training data of all the vocoders, but with utterances not seen during training (section 4.2). Then, we also generated speech for speakers not present in the training data of any of the vocoders, with a mixture of male and female speakers neutral extracted from VCTK [22], with a speaking style similar to the training data but out of domain in terms of speaker and recording conditions. Finally we also evaluated highly expressive children audiobook speech extracted from Bliz-zard2013 development set [23], which was out of domain in speaker, speaking style and recording conditions. Experiments were carried out with oracle spectrograms unless stated otherwise. The naturalness perceptual evaluation was designed as a MUltiple Stimuli with Hidden Reference and Anchor (MUSHRA) test [24], where the participants were presented with the systems being evaluated side-by-side, asked to rate them in terms of naturalness from 0 (very poor) to 100 (completely natural). The test consisted of 200 randomly-selected utterances for the different speakers, these were not included in the training data. The evaluation was balanced so that every utterance was rated by 5 listeners, and each listener rates 20 screens. The evaluations were conducted with self-reported native American English speakers using Amazon Mechanical Turk. In total 50 listeners participated in each evaluation.
Paired Student T-tests with Holm-Bonferroni correction were used to validate the statistical significance of the differences between systems, considering it validated when p − value < 0.01. We use the ratio between the mean MUSHRA score of a system and natural speech, from now on 'relative MUSHRA', to illustrate the quality gap with the reference.

Glitch analysis
This evaluation is aimed at gauging the nature of speech errors present in the evaluated SSWS systems. 4 internal listeners were asked to rate 50 different samples for each of the 4 considered systems, for a total of 200 rated utterances per system. For each sample the evaluators decided if a speech error was present, the perceived nature of the error and its severity (i.e. minor, medium, critical).

CNN vs. RNN evaluation
To narrow down the number of systems to be analyzed in the robustness investigations we compared first of all the complete set of introduced systems, aiming only the best amongst them. For this evaluation we considered predicted spectrograms in the case of vocoders, to provide a fair comparison ground for CNN LI. The spectrograms were predicted with our internal sequence-to-sequence model, based on Tacotron2  [3] and explained in detail in [25]. The naturalness evaluation of the systems (figure 2 showed no significant difference between the vocoders, with CNN LI falling behind, which we hypothesize is likely due to its less dynamic prosody. From the glitch analysis (table 3) the following conclusions can be drawn: First of all, system CNN LI showed to be overall more prone to glitches (51 out of 200 utterances) due to the much more complex nature of the task, as there is no support module capable of generating an intermediate acoustic representation such as the mel-spectrograms. Secondly, we can see how reducing the size of the CNN-based vocoder (system SCNN MS) significantly increased the medium + critical glitches compared to the full-size implementation (system CNN MS) (25 vs. 14). Finally, this analysis confirms the hypothesis that the RNN-based system (RNN MS) is the most stable approach in both absolute terms (20) and when only looking at medium + critical (8).
From this point onwards we will focus only on the RNNbased system (RNN MS).

In-domain speakers
This evaluation considered 2 female speakers and 1 male speaker (the same speakers used to train the 3Spk vocoder). The results in Figure 3 show a clear story. There is no significant difference in terms of evaluated naturalness when using any of the trained vocoders as long as the speakers were part of the training data. This is a strong result for the proposed universal vocoder, as it showed no degradation when compared to the highly specific SD neural vocoder. Moreover, while there was a statistically significant difference, between vocoded and natural naturalness, it was minimal (98.5% relative MUSHRA scores). It must be noted that while there were inter-speaker differences, those did not affect the rankorder of the systems' ratings, so results are presented only as averages.

Out-of-domain speakers
In this evaluation, we considered out of domain speakers for which an SD vocoder was not available. As such, results are expectedly poor in comparison to some of the more general neural vocoders. It was included as a reference and selected by looking for the system with the smallest distance as per the protocol defined in the following sub-section.

Selecting speaker dependent vocoders
In order to limit the perceptual evaluation complexity we tried to predict the best SD vocoder to run inference with. For that we built multi-variate Gaussian Mixture Models (GMMs) of the training data of the different vocoders and of the target speaker, then obtained the Kullback-Leibler divergence (KLD) between the GMMs. This could be improved by applying conventional speaker similarity measures, but the measure proves to be reliable with a correlation of 0.81 between the KLD and the naturalness degradation obtained through the perceptual evaluations, with degradation being defined as the difference between natural speech and system mean MUSHRA. The analysis concluded that F1 was closest to the female out-of-domain speakers, and M1 to the males.

Neutral style speakers
Results ( Figure 4) show an interesting picture. The more variety in number of training speakers the better the achieved quality, to the point where the proposed universal system is capable of providing practically the same relative MUSHRA score as for in-domain speakers (98% vs. 98.5%). This speaks very strongly about the generalization capabilities of such a system. Moreover, we can see how the vocoder trained with more speakers but with less training data (7Spk) is capable of providing better quality than the other two systems (SD and 3Spk), suggesting that variability is more important than quantity for generalization. Similarly to what happened in the in-domain evaluation, the inter-speaker variability did not affect the rank-ordering, so we only show averaged results.

Audiobook style speakers
In the case of highly expressive data, including disfluencies and onomatopoeias, (see Figure 5) the universal vocoder is still capable of proving steady quality, once again maintaining the relative MUSHRA scores at 98%. Both SD and 7Spk show comparatively poor performance, while 3Spk breaks the trend. This is confirmed by the KLD between the audiobook speaker and those of the vocoders (2.64 against Univ, 5.42 against 3Spk, 14.45 against 7Spk and 14.62 against SD).

Other evaluations
We carried out evaluations analogous to those presented but with Japanese, which was an in-domain language. Interestingly, results were extremely similar to those in US English (98% relative MUSHRA), hence we do not report them in detail. This comforts us in the observation that the universal neural vocoder can be expected to be robust not only to speaker, style and recording conditions but also language.

CONCLUSIONS
We have introduced a robust universal neural vocoder conditioned on mel-spectrograms, without any form of speaker encoding. The performance of this vocoder has been shown to be as good, if not better, than a SD neural vocoder. The universal vocoder has proven to be capable of generating speech for speakers not present in the training corpora, regardless of speaking style or language, with a relative MUSHRA score of 98%. A universal neural vocoder allows for future work to focus on spectrogram estimation from text to any new speaker, language or style without being constrained by training a specific neural vocoder.