Artificial Intelligence (AI) systems can provide automatic, personalized and real-time feedback to learners in distance learning settings when a human expert is unavailable. AI feedback has the potential to be always available and can allow learners to practice deliberately and repeatedly at their own pace. Embedding AI feedback into immersive and multi-modal technologies, like Augmented and Virtual Reality (AR/VR) or sensor-based systems, enables learners to train physical learning tasks alongside the more traditional cognitive tasks. We refer to these AI systems as Multi-modal Tutors. One scenario that needs to be considered when replacing human feedback is that the AI system may deviate from the learner’s expectations and the teacher’s learning goals, risking becoming unethical. This challenge is known in AI ethics research as the “alignment problem” and refers to the importance of designing AI systems that align with human values.
This project tackles the “alignment problem” in distance teaching in the particular context of presentation skills training. To tackle the problem, we design a Multi-modal Tutor to train presentation skills such as body position, pauses, and voice intonation. The Multi-modal Tutor features various game-oriented immersive scenarios on a mixed-reality headset to allow learners to practice autonomously, supported by AI-generated feedback. The human experts can monitor the learning process asynchronously via an Alignment Dashboard. They can assess the learner performance and the quality of the AI feedback, ensuring it is correctly aligned with the learning goals.
The project will finance two PhD candidates, one at DIPF and one at Cologne Game Lab.