In this short note we introduce ResearchDoom, an implementation of the Doom first-person shooter that can extract detailed metadata from the game. We also introduce the CocoDoom dataset, a collection of pre-recorded data extracted from Doom gaming sessions along with annotations in the MS Coco format. ResearchDoom and CocoDoom can be used to train and evaluate a variety of computer vision methods such as object recognition, detection and segmentation at the level of instances and categories, tracking, ego-motion estimation, monocular depth estimation and scene segmentation. The code and data are available at this http URL
Another recent effort similar to ResearchDoom is Unreal CV @cite_5 . This engine can also extract metadata similar to ResearchDoom and, by building on the Unreal engine, can potentially be applicable to a huge variety of modern video games. ResearchDoom is much simpler than Unreal CV as it applies to a single and relatively old game. While Doom incorporates several limitations, such as the fact that the camera can only rotate around the vertical axis, the data is nevertheless fairly complex. Furthermore, Doom provides a relatively restricted and consistent world form which data can be extracted for experiments. We leverage the latter fact to define and provide abundant pre-computed data with detailed metadata as well as with annotations using the Microsoft Coco format.
@cite_5
2884161042
In training deep neural networks for semantic segmentation, the main limiting factor is the low amount of ground truth annotation data that is available in currently existing datasets. The limited availability of such data is due to the time cost and human effort required to accurately and consistently label real images on a pixel level. Modern sandbox video game engines provide open world environments where traffic and pedestrians behave in a pseudo-realistic manner. This caters well to the collection of a believable road-scene dataset. Utilizing open-source tools and resources found in single-player modding communities, we provide a method for persistent, ground truth, asset annotation of a game world. By collecting a synthetic dataset containing upwards of @math images, we demonstrate real-time, on-demand, ground truth data annotation capability of our method. Supplementing this synthetic data to Cityscapes dataset, we show that our data generation method provides qualitative as well as quantitative improvements---for training networks---over previous methods that use video games as surrogate.