VIRTUAL TERRESTRIAL LASER SCANNER SIMULATOR IN DIGITAL TWIN ENVIRONMENT

We are presenting a Terrestrial Laser Scanner simulator a software device which could be a valuable educational tool for geomatics and engineering students. The main goal of the VirScan3D project is to cover engineering digitisation and will be solved through the development of a virtual system that allows users to create realistic data in the absence of a real measuring device in a modelled real-life environment (digital twin). The prototype implementation of the virtual laser scanner is realised within a game engine, which allows for fast and easy 3D visualisation and navigation. Real-life objects can be digitised, modelled and integrated into the simulator, thus creating a digital copy of a real-world environment. Within this environment, the user can freely navigate and define suitable scanning positions/stations. At each scanning station, a simulated scan is performed which is adapted to the technical specifications of a real scanner. The mathematical solution is based on 3D line intersection with the virtual 3D surface including noise and colour as well as an intensity simulation. As a result, 3D point clouds for each station are generated, which will be further processed for registration and modelling using standard software packages.


Digital Twin in heritage conservation
In heritage education and heritage conservation, the production of as-built surveys in plan and section in representational scales from an overview of 1:100 up to a detail of 1:10 is still a subject that is classically taught in courses across Europe. The strategically set and annotated 2D plans and sections are a stand-in for the real building and its information about materiality and construction. The 2D representations allow a holistic understanding of a historic building, foster the opportunity to display a building research project, apply condition and damage mapping as a further layer and serve as a basis for efficient planning of rehabilitation measures and building monitoring. The use of hand-drawing and total stations in heritage buildings and cultural heritage environments, such as archaeology, is now extended to the production of digital 3D surveys by terrestrial laser scanning and photogrammetry with the same intentions.
3D imaging can be used as a teaching tool, to work with models that represent the reality and must, of course, respect the same requirements and appropriateness for accuracy, resolution and information as the existing formats of the 2D as-built surveying described above (Kravchenko, Luhmann, & Shults, 2016). Here the idea is much likened to the use of digital surrogates or the function of a 'Digital Twin' in 3D.
In industry 4.0, the goal of a Digital Twin is the simulation of functionality processes of machines and sensors and reproduction of these functionalities in a virtual world in order to ensure a flawless accelerated production environment with equivalent reactions and within thresholds for quality control (Gómez-Cabrera, Paez, & Alvarado, 2016;Peruzzini, Pellicciari, & Bil, 2018).
These goals for a Digital Twin are seen quite differently from the perspective of preservation and management of immobile heritage (Jouan & Hallot, 2020). The creation of digitised 3D survey products with added information can now be seen as the Digital Twin of the tangible heritage, which can be connected with rich semantic information and real-time on-site sensors for monitoring to support the life-cycle of heritage buildings and management in line with the sustainable preservation of our built cultural heritage. This is supported by the new way of representing 3D survey models as the database in H-BIM (Historic Building Information Modelling after Dore & Murphy, 2012). Further steps in the connection of survey, H-BIM and heritage management have already taken place (Jouan & Hallot, 2019) and are more and more established in heritage authorities (Bryan, 2017).
In a time, and especially at the time of writing during the Covid-19 pandemic, where e-learning tools have gained an essential significance, and where students are ever more familiar with virtual gaming environments, the teaching of practical capabilities in relation to heritage conservation for as-built survey can take advantage of the opportunity of digitalisation of the activity of teaching (Vlachopoulos & Makri, 2017). The proficiency of the next generation of practitioners in heritage and geomatics in 3D imaging is an educational goal that needs to be included in curricula, such as in the new M.Sc. Digital Technologies in Heritage Conservation in Bamberg university (Hess et al., 2019). Here, a step into a simulator for teaching and learning with the appropriate delivery in reality-like application, resolution and quality outcome is desirable for heritage practitioners.

VirScan 3D scanning simulator as an educational tool
In the areas of environmental planning, building construction, infrastructure maintenance or cultural heritage conservation, the use of digital geospatial information has significantly grown during the last decade, for instance in the field of building information modelling (BIM). Location-based data and geoinformation, in general, are nowadays also a key factor in the economy such as logistics, energy, communication or internet business. Specifically, new instruments for digital recording and processing tools are increasingly established in those application areas.
Since digitalisation is one of the major worldwide trends, the teaching curricula, related to digitisation in engineering practice (e.g. geo-information sciences, cadastral mapping, geodesy) should educate their students according to current needs and demands from the professional market and industry. State-of-the-art technologies in these fields (e.g. 3D laser scanning, photogrammetry, UAVs) lead to large amounts of digital data and are difficult to process, analyse and transform into useful information. For this purpose, appropriate tools and workflows have to be provided, otherwise, neither teachers nor students have realistic chances to adapt to new methods in due time, and they will not be able to contribute to further developments.
However, this objective is hard to achieve, since there is a lack of knowledge among many teachers, out-of-date equipment (hardware and software), insufficient ICT infrastructure and insufficient finances for necessary investments, especially in developing countries. Furthermore, the development of recording instruments has a rapid change cycle of ca. five years. For instance, expensive terrestrial laser scanners are very rare in higher education institutions of those countries. In addition, a strong interdisciplinary approach should be followed since these technologies are relevant in many other subjects such as civil engineering, machine construction, cultural heritage or informatics (Hess, Garside, Nelson, Robson, & Weyrich, 2017).
We are presenting a concept and initial results of a terrestrial laser scanner simulator -a tool, which could be a valuable educational tool for geomatics and engineering students with the aim of experiential learning strategies (Falloon, 2019;Dib, Adamo-Villani, & Garver, 2014).

Objectives for VirScan 3D scanning simulator
The project group of VirScan3D project consists of two German universities and one Ukrainian university based on long-term cooperation that has started in 2002.
The virtual scanner simulates commercially available tools with realistic user interfaces and creates 3D point clouds according to individual specifications and settings in a digital real-world environment. Any digital 3D model (such as a cultural or technical building) that is virtually scanned in a digital environment can be used as a training object. With this system, teachers and students will be able to learn how to manage such systems, generate realistic big data and complete the entire data processing chain. In order to make learning processes more immersive, it is possible to simulate scanning in a virtual twin environment.
A virtual learning environment will solve the problem of providing high-precision (and highly-priced) devices in accordance with professional standards. Especially in geospatial sciences, the simulation in virtual environments can be used for the virtual reconstruction of natural scenes like cultural heritage objects or 3D cities (Kersten & Edler, 2020). It is relevant for the specialists in the fields of geodesy, photogrammetry, GIS and land management as well as neighbouring disciplines (architecture and cultural heritage digitalization, computer vision, mathematics etc.).

Simulator description
As mentioned before, VirScan3D is developed for students of geodetic and other higher education departments with limited access to laser scanning equipment. It allows simulating of all processes of the fieldwork phase of terrestrial laser scanning and generates results that are very close to data generated with real physical scanners on-site. Moreover, the digital environment can simulate real-world objects, i.e. buildings or sites. These results can be used for further processing in scanning software in the office work phase.
The project is developed using readily available resources and tools and it is designed to run on Windows platforms.
The product functionality includes the following features:


Selecting the scanner type which will be simulated;  Laser scanning process simulation with visualisation to obtain point clouds with intensity and noise values according to scanner specifications (noise and intensity not fully implemented yet);  Scan station management which allows to specify settings for each scan station, create and delete stations;  Target management which allows placing and removing targets of different types;  Project management which allows to save the project as a user and to continue work later;  Batch scanning;  Real-time visualisation of generated point clouds;  Exporting the point cloud.
The simulator offers two scenarios of work: 1) The user is navigating within the virtual environment and selects target positions as in real life. Then the scanner position and scan settings are specified and the scanning process of the virtual device is started. This is visualised in real-time. When the scanning is finished the user avatar 'goes' to the next station.
2) The user navigates within the virtual environment and selects positions for all targets and all scan stations. Subsequently, scan settings are specified for all stations or for each station separately. Finally, the scanning simulation is started for all stations one by one.
The result of the simulation is a structured 3D point cloud in ASCII format for import in further software. For each scan station, a single point cloud is generated, hence similar to reality no registration is provided. It is also possible to export target coordinates to use it in registration software for cloud referencing.
VirScan3D is developed within Unreal Engine (a wellknown gaming engine); therefore, user's computers should satisfy its minimum requirements (Unreal, 2020). However, the speed of scan simulation directly depends on computer performance. Therefore, to obtain the speed of simulation no slower than scanning in the real world, the following specifications are recommended: Main functions are programmed with Blueprint Visual Scripting system of Unreal Engine. It allows making the system very flexible and modifying software modules without recompiling the whole code. Some specific modules were programmed in C++ due to Blueprint limitations.
Implementation of advanced features in future versions of software might need the use of additional external software, e.g. Blender (2020).

Digital twin environment integration
In order to create a real-world replica/digital twin within the simulator environment, the existing building should be recreated as a 3D model and integrated. It was decided to use the "Scan to BIM" approach, i.e. perform the 3D scanning of the desired object and model it using Autodesk Revit software. This BIM model can be integrated into Unreal Engine environment using the Datasmith importer plugin, which allows to convert 3D data from a variety of industry-standard design applications into Unreal assets.

Test objects
Initially, the simulator testing was intended to be carried out as part of a summer school for project participants. Thus, the campus buildings of the University of Bamberg have been selected as test objects. The buildings form an ensemble and are the individual monuments in the centre of the UNESCO world heritage in Bamberg.
The building complex with the address 'An der Universität 2' begins its history in the 17 th century as a Carmelite monastery, which is reflected in its three-storey baroque architecture with a courtyard surrounded by three-storey buildings and a church (Fig. 1). After a construction period of almost 50 years, the building was completed in 1742 with the establishment of the theological library (Wied, 2007). In the following, this building will be referred to as 'Baroque building'. Another building -'An der Universität 9' -integrates a former tower of the ancient city wall with its characteristic late medieval sandstone stone blocks and octagonal roof. It has received building extensions and numerous alterations in the 17 th and 19 th century, which can also be seen in the appearance of the building (Fig. 2). In the following, this building will be referred to as 'university office building'.

Data acquisition
The objects have been scanned with high resolution in their real-world environment using Leica BLK360 and Faro Focus 3D scanners. Both trial cases include the elements of different materials, such as sandstone, building stone, tiled roof, glass and steel structures with glass (e.g. the modern library corridor). The details also differ in their colours (Fig. 3a). This has been reflected in the resulting point cloud (Fig. 3b). In the resulting point cloud, the geometries were recorded with a resolution of 1 cm which allows detailing fine elements and decoration in later modelling. Due to the material properties and object colours, different intensity values were obtained. Sometimes the individual elements could not be captured due to over radiation or penetration, which consequently led to the gaps in the point cloud.

Modelling
In the simulator, the object material was represented as an object colour. In the future, the specific material properties will be analysed and integrated into the software (reflection intensity and noise according to specific element material).
In order to generate 3D test objects that are compatible with the simulator software, the recorded buildings have been modelled in Autodesk Revit BIM software. This involves object geometry modelling and assigning materials to object elements. The pre-modelled components available in the Revit library also consist of material properties. This mostly concerns standard Revit elements, such as glass windows and facades, walls and roof materials. For new and unique geometries modelled by the user, the materials were configured manually.
According to the simulator software requirements, the optimal number of model polygons should not exceed 30000 elements.
In addition to the main geometries of objects, the individual elements such as window and door decorations are presented in detail in this case study (Fig. 4). In combination with geometry and object property representation, the level of detail of generated models is equal to LoD 300 according to AIA E202 (Document E202, 2008). The detail precision varies within +/-5 cm.
For the realistic representation, the surrounding objects were also modelled, but their level of detail is much lower (LoD200, Fig. 5). All models have been stored in a Revit format (.rvt) and additionally in IFC format for further import by the Datasmith plugin.

Data integration
As rendering in Unreal Engine is made in real-time several aspects need to be considered during the integration of 3D data: In addition, BIM models include information about the material of elements which is important for intensity generation functions. Figure 5 shows the model of the building with its surroundings integrated into the simulator.
Unique elements of the object might be modelled with the "Model-in-place" feature. However, the same structured approach has to be implemented to consider the correct calculation of collisions.

Experiment -working example of the VirScan3d software
As a prototype for the terrestrial 3D scanners (TLS) that can be chosen in the VirScan3D simulator, we used the Leica BLK 360 laser scanner and an integrated BIM model of one of the historical buildings, the office building described in Section 3.1., in the city of Bamberg. The current simulator version offers four integrated scanner models (Faro X330, Leica Scanstation P40, Leica RTC360 and Leica BLK). In the future, other common laser scanner brands and models on the market will be simulated to extend the training variety.
The process starts with the placing of numbered targets and positioning of scanner stations (Fig. 6). The user can choose either checkboard targets and/or spheres that can be attached to walls and surfaces like in real life. By picking a point in the 3D model the user specifies a scan position. Then the system checks if the station is correct, and saves it.
There are some functional requirements to be noted:  A station is incorrect if it is too close (20 cm) to other objects or inside of other objects;  A station is incorrect if it is located on vertical (±30º) surfaces.
The access to scanner settings is implemented through realistic interfaces of the selected scanner model (example in Fig. 7). The user can select scanning parameters and set the instrument height (height is set to 1.60m by default). It is possible to change the scanning position or remove a station. After the simulated scanning, a 3D point cloud is generated in Cartesian coordinates (X, Y, Z). According to the station settings, an angular grid of rays is projected from the station point. The intersection of the ray with the first surface gives a discrete point, and its XYZ coordinates are stored in a text file. The origin of the coordinate system has been set to the station point. The maximum range of surface search is limited to the scanner specifications. Figure 8 represents an example of a single point cloud, generated by the simulator. It can be seen that point intensities are not fully simulated with the current version of the software; i.e. the material reflectance dependent part of intensity is not implemented yet. As mentioned above, the current prototype version consists of four terrestrial laser scanner models and one 3D environment for simulation. It is expected that release versions of the software will have more models of scanners implemented and a possibility to configure custom scanners. A highly detailed and precise 3D model of the real environment of the University of Bamberg, based on real laser scanning data, was used as a reference environment (see Section 3.1). In future releases, more 3D environments will be accessible. The functionality to place targets is already implemented, which allows users to register resulting point clouds by a target-to-target approach. Also, it is expected that future versions of the simulator will be able to fully simulate intensity and noise.
Since the reference environment shows a realistic and quite complicated example, it is necessary to occupy several scanning locations in order to cover the building surface geometry completely. As in a real-life situation, the roof is the most difficult part to capture in simulation, because of the occlusions and narrow streets surrounding the object of interest. For this reason, some scanning locations were selected at longer distances from the university office building, allowing to scan the roof details as much as possible. During the simulation, 18 stations were selected in order to record the building. The distribution of the simulated scan stations is shown in Figure 9 below.
After the simulation, the 18 scans were exported/saved in a typical industry-standard text file format containing coordinates, intensities and RGB values. At this stage, the simulation part of the fieldwork phase of terrestrial laser scanning is completed and further processing (cleaning, registration, modelling) could be done using any available laser scanning software. In the presented case we used Leica Cyclone software. The 18 simulated datasets were imported and registered by a cloud-to-cloud approach, using the "visual registration" tool for initial alignment. The mean absolute error for enabled constraints results to 0.001 m; while registration errors for individual clouds not exceeding 0.004m. The error histogram for cloud constraint between stations 10 and 11 is shown in Figure  10.  The registration statistics show slightly better results than one can expect from real scanning and cloud-to-cloud registration. One possible explanation could be that simulated data is free of noise, since noise simulation is not implemented yet in the current prototype version of the VirScan3D simulator.
The final registered point cloud was cleaned and exported as (.pts) file. The unusual intensity values are the result of not fully implemented intensity simulation (Fig. 11).
This chapter has shown that the whole sequence of scanning could be simulated in a digital twin environment, thus generating realistic scanning data which later can be processed, registered and used in the entire data processing chain.
The comparison with real data will be conducted during the future 'summer school' application weeks planned for 2021.

Discussion
The recent Covid-19 pandemic has demonstrated that digitalization of teaching is necessary, even in those academic programs where classroom teaching is unavoidable, for example, in practical courses with surveying equipment. VirScan3D, as a new virtual instrument for digitised teaching of terrestrial laser scanning, can be used to create 3D point clouds from different stations including strategic target placement; in a further step, these 3D point clouds have to be registered (and if needed geo-referenced) and processed to higherlevel information such as 3D models, architectural drawings or maps. The virtual 3D scanner emulates commercially available instruments with realistic user interfaces and creates 3D point clouds in a virtual simulation according to individual specifications and settings. Highly detailed and accurate BIM models, which represent real-world objects, can be used as training objects within this environment. With such a virtual system, tutors, as well as students, can learn and prepare to operate those devices, generate close-to-reality large 3D data sets and perform the complete chain of data processing.

Conclusion
We would like to summarise the work presented here by an evaluation and an outlook for further work on the Virscan3D system.

Evaluation
The developed software tool for the simulation of 3D terrestrial laser scanning allows users to reproduce the sequence of actions at the stage of fieldwork and create realistic 3D scanning data in the absence of a real measuring device. Users can plan their fieldwork in the virtual environment, i.e. specify 3D scanner type and target positions or change scanner settings. In addition, users can observe the process of virtual scanning and analyse the results, e.g. check the density of the point cloud and surface coverage of the recorded object.
Summarizing the simulator features, we state that VirScan3D:  Gives a good 3D laser scanning visualisation for a hands-on process explanation to use a terrestrial laser scanning device;  Helps to learn basic terrestrial laser scanning parameters and interface usage, useful for planning of an individual workflow;  Allows the generation of scanning results comparable with real data;  Replaces real equipment for practical exercises in case of lacking financing of expensive equipment;  And is a useful tool for distance learning courses, e-learning and online teaching.

Simulator optimization and further work
Future work will consist of a more realistic simulation of laser intensity based on:  Material-dependent reflection functions resulting as intensity/RGB values;  Estimated beam diameter for the area where the laser beam hits a surface element (laser spot).
Point cloud noise simulation functions are also to be implemented:  Deviations due to distance and angular measurement accuracy;  Range error due to surface reflectivity;  Advanced noise functions for surfaces with specific reflectance (water, glass, mirror etc).
Integration of more scanner models with reality-like user interfaces and specifications, and a variety of laser scanning settings specifying such parameters as that can be defined for each scan station:  Station name;  Field of view;  Resolution;  Quality, etc.
It is also desired that users can load their own 3D model into the virtual environment, which is, however, currently not supported by Unreal Engine.