Many-to-Many Metrics: A New Approach to Evaluate the Performance of Structural Damage Detection Networks

In the last years, Computer Vision and Deep Learning techniques have proved to be useful in supporting structural inspections of buildings and civil infrastructures. Particularly, in the case of post-disaster structural safety assessment, automated damage detection algorithms can accelerate the analysis of survey images and thus contribute to a fast screening of impacted areas. The identification of the various types of damage can be seen as a special case of object detection in which the goal is to identify sub-parts of a large object (e.g., cracks on a building). However, in this scenario, traditional evaluation metrics for object detection tend to underestimate the actual performance of the detector, since they considered a one-to-one match between a ground-truth box and a predicted box. Such approach could be sub-optimal for damage detection: for example, a crack can be labeled as a single entity in the ground truth but detected as two small cracks in inference or vice versa. To compensate this issue and better asses the performance of the detector, we introduce a new set of metrics called Many-to-Many. We tested these metrics using a YOLO network on two datasets containing images of damaged bridges and civil structures, and we collected evidence of an improved evaluation capability.