Chris Liebold
Researchers from CCS group Anke Stoll (now University of Amsterdam), Jingyuan Yu, Aliya Andrich, and Emese Domahidi has published new open access article “Classification bias of LLMs in detecting incivility towards female and male politicians in German social media discourse” in the journal Communication Methods and Measures.
The CCS team examined how models like BERT, GPT-4, and Meta’s Llama 3 identify different types of uncivil language directed at politicians on YouTube and X (formerly Twitter). Analyzing over 24,000 user comments, researchers found that while the models were effective at flagging overt insults and vulgarities, they often missed subtler forms of hostility such as stereotyping and discrimination, particularly against female politicians. This shortfall raises concerns that current AI moderation systems may unevenly protect targeted groups and overlook harmful but implicit incivility. The study calls for more transparent, fair, and democratic AI tools that can better recognize the full spectrum of online abuse.
This paper belongs to the special issue “Understanding and Addressing Bias in Computational Social Science”, guest edited by Dr. Valerie Hase, Prof. Dr. Marko Bachl, and Prof. Dr. Nathan TeBlunthuis. The CCS team extends its sincere thanks to the editors for their valuable support and guidance throughout the research and publication process.
More details about the paper: https://doi.org/10.1080/19312458.2025.2551693