A critical reflection on the use of toxicity detection algorithms in proactive content moderation systems
Published in International Journal of Human-Computer Studies, 2025
Recommended citation: Warner, M., Strohmayer, A., Higgs, M., & Coventry, L. (2025). A critical reflection on the use of toxicity detection algorithms in proactive content moderation systems. International Journal of Human-Computer Studies, 198, 103468. https://www.sciencedirect.com/science/article/pii/S1071581925000254
We describe contextual factors impacting on perceptions of toxicity and system efficacy, and system efficacy across different end-user groups. We identify and describe ways in which these systems are open to abuse. We discuss how these systems could have unintended consequence. We present considerations for designing proactive moderation systems.
Recommended citation: Warner, M., Strohmayer, A., Higgs, M., & Coventry, L. (2025). A critical reflection on the use of toxicity detection algorithms in proactive content moderation systems. International Journal of Human-Computer Studies, 198, 103468.