Researchers have uncovered significant fairness issues related to age and skin colour in the detection systems of autonomous vehicles, revealing that children and individuals with darker skin are at more risk on the street, a new study has shown.

According to the study conducted by researchers at King’s College in London, a fairness analysis of eight different AI-powered pedestrian detectors trained on “widely-used, real-world” datasets revealed that the programmes were significantly worse at detecting darker-skinned pedestrians than lighter-skinned pedestrians.

They found through testing over 8,000 images through these pieces of software that detection accuracy for adults was 19.67 per cent higher compared to children, and there was a 7.52 per cent accuracy disparity between light-skinned and dark-skinned individuals.

However, gender showed only a difference of 1.1 per cent in detection accuracy.

“Autonomous driving systems are on track to become the predominant mode of transportation in the future. However, these systems are susceptible to software bugs, which can potentially result in severe injuries or even fatalities for both pedestrians and passengers,” said Jie Zhang, a Kings College lecturer in computer science and a co-author of the study.
Moreover, researchers found that detection performance for the dark-skin group decreases under low-brightness and low-contrast conditions compared to the light-skin group, in other words, night.

For instance, the difference in undetected proportions increases from 7.14 per cent to 9.86 per cent between day time and night time scenarios.

“Fairness issues in autonomous driving systems, such as a higher accuracy in detecting pedestrians of white ethnicity compared to black ethnicity, can perpetuate discriminatory outcomes and unequal treatment based on race,” the researchers said.

“This can result in harm to individuals belonging to marginalised groups, further exacerbating existing social inequalities. Therefore, it is crucial to prioritise fairness testing in autonomous driving systems,” they added.