MILKI-PSY – Multimodal Immersive Learning with Artificial Intelligence for Psychomotor Skills

Project Description

The COVID-19 pandemic has shown that many teaching/learning activities can be performed without physical presence. This is hardly true for psychomotor skills: their development, as required in many disciplines (e.g., medicine, engineering, chemistry, artistic activities, sports), requires hands-on practice, direct feedback and reflection. In order to achieve the desired learning successes, personnel support and material input are therefore indispensable. Both of these factors increase costs and limit the scalability of the courses concerned: experts are rare and expensive, and the use of materials causes further costs.

Current technological developments are changing this situation:

• Mixed, augmented and virtual reality make it possible to create immersive learning and practice spaces.
• Modern sensor technologies can track and record fine-granular movements.
• Big Data methods and their application in learning analytics can analyse and evaluate large amounts of data, which is especially indispensable for data-intensive learning, such as real-time analysis of psychomotor skills.
• Machine learning (e.g., reinforcement/deep learning) and generative artificial intelligence (e.g., generative adversarial networks) techniques can interpret and infer large data sets and generate individualized feedback.

Project Objectives

To date, these technologies have largely been considered separately. MILKI-PSY aims to create AI-supported, data-intensive, multimodal, immersive learning environments for independent learning of psychomotor skills. In doing so, a cross-domain approach is emerging that enables multimodal recording of expert activities and the use of these recordings as blueprints for learners. With the help of artificial intelligence and automated error detection, the learning progress is analysed and individual feedback is generated. This creates holistic, innovative learning environments for learning psychomotor skills, in which personalised, AI-supported learning support enables individual learning processes based on complex data analyses.

Funding

Cooperations

TH Köln – University of Applied Sciences (Projektkoordination), Deutsches Forschungszentrum für künstliche Intelligenz, Rheinisch Westfälische Technische Hochschule, Deutsche Sporthochschule Köln, Institut für Produktentwicklung und Konstruktionstechnik, TH Köln

Publications

Project Management

Project Team

• Fernando Cardenas-Hernandez
• Gianluca Romano

Project Details