Image-to-UI Accessibility: Assessing ChatGPT’s Effectiveness in Generating Accessible Android Screens from Figma Templates

Abstract

As accessibility remains a persistent challenge in mobile applications, recent advances in Large Language Models (LLMs) offer promising support for addressing these issues in the development process. This study investigates the potential of ChatGPT-4 to generate accessible Android user interfaces directly from screen mockups. Using a dataset of 60 screens in 8 Figma-based UI templates, we prompted ChatGPT to produce Jetpack Compose code solely from screen images. We then evaluated the generated code using Google’s Accessibility Scanner, identifying a total of 302 accessibility violations. Through iterative prompting guided by the scanner’s feedback, we achieved a reduction of more than 50% in the reported issues. However, the process also introduced new violations in some cases, highlighting important limitations in the consistency and reliability of the model. Our findings suggest that while LLMs like ChatGPT can support accessibility improvements in mobile User Interface (UI) development, human oversight remains essential. This study contributes to ongoing discussions on the role of AI in inclusive software design and the future of image-to-code generation pipelines.

Ähnliche Arbeiten