Applying Ethical AI Frameworks in practice: Evaluating conversational AI chatbot solutions

Abstract

Ethical AI frameworks are designed to encourage the accountability, responsibility and transparency of AI applications. They provide principles for ethical design. To be truly transparent, it should be clear to the user of the AI application that the designers followed responsible AI principles. In order to test how easy it is for a user to assess the responsibility of an AI system and to understand the differences between ethical AI frameworks, we evaluated four commercial chatbots against four responsible AI frameworks. We found that the ethical frameworks produced quite different assessment scores. Many ethical AI frameworks contain requirements/principles that are difficult to evaluate for anyone except the chatbot developer. Our results also show that domain-specific ethical AI guidelines are easier to use and yield more practical insights than domain-independent frameworks. We conclude that ethical AI researchers should focus on studying specific domains and not AI as a whole, and that ethical AI guidelines should focus more on creating measurable standards and less on stating high level principles.

Ähnliche Arbeiten