Game engines were initially created to create fictional worlds for entertainment purposes. However, these same engines can be used to create digital twins, which are exact replicas of genuine worlds. Researchers from Osaka University have discovered a way to train deep learning models that can quickly scan photos of real cities and precisely differentiate the buildings that appear in them using the images that were automatically created by digital twin city.
A deep learning neural network called a convolutional neural network is made for processing structured data sets like photographs. The way tasks like architectural segmentation are carried out has fundamentally changed as a result of recent deep learning developments. However, a deep convolutional neural network (DCNN) model requires a substantial amount of labeled training data and classifying this data manually can be time-consuming and incredibly expensive.
The researchers used a 3D city model from the PLATEAU platform, which has incredibly detailed 3D models of the majority of Japanese cities, to build the synthetic digital twin city data. They imported this model into the Unity game engine, where they put up a camera on a virtual automobile that drove about the city and captured virtual data photographs in a variety of lighting and weather scenarios. For the trials, actual street-level photographs of the same research region were then obtained using the Google Maps API.
The researchers discovered that using data from the digital twin city produces better outcomes than using only virtual data without any real-world counterparts. Furthermore, segmentation accuracy is increased when synthetic data is added to a real dataset. Most notably, however, the researchers discovered that the segmentation accuracy of the DCNN is greatly improved when a specific percentage of real data is included in the digital twin city synthetic dataset. In fact, its performance approaches that of a DCNN trained entirely on real data. Tomohiro Fukuda, the paper’s corresponding author, says that our suggested synthetic dataset could be used instead of the real photos in the training set.
For construction management and architecture design, large-scale measurements for retrofits and energy analyses, and even seeing building facades that have been demolished, automatically breaking out the different building facades that appear in a picture is helpful. Multiple cities tested the system, proving the transferability of the suggested framework. For the majority of contemporary architectural styles, the hybrid dataset of real and artificial data produces encouraging prediction results. Because manual data annotation is not required, it offers a potential method for training DCNNs for architectural segmentation tasks in the future.
