Deployment-Oriented Benchmarking of Open-Source Large Language Models for Social Robots

Abstract

Large language models (LLMs) offer new opportunities to enhance human–robot interaction by enabling humanoid robots to engage in natural, context-aware dialogue. However, deploying LLMs on social robots operating in real-time environments remains challenging due to latency constraints, limited onboard hardware, and privacy considerations. This paper introduces a deployment-oriented benchmarking framework for evaluating open-source LLMs that are feasible for on-device execution on humanoid robots. We implement and analyze ten lightweight LLMs (≤2 billion parameters), using the Pepper robot as a representative use case in CS1/CS2 laboratory courses where the robot functions as a teaching assistant. The models were evaluated using four normalized metrics: instruction-following accuracy, conversational clarity, response latency, and on-device feasibility. Results identify clear trade-offs within the lightweight tier, emphasizing models that best balance responsiveness with instructional quality. This work provides a reproducible methodology and practical deployment guidelines for integrating LLM-driven instructional capabilities into humanoid robots to support more autonomous, student-centered learning in introductory computer science education.

Ähnliche Arbeiten