What We Learned from ROLEPL-AI’s Experimentation Pilots
- jzaraik
- Aug 6
- 2 min read
We're pleased to announce the publication of Deliverable 5.3 – Results Analyses, a key milestone in the ROLEPL-AI project. This report brings together the results of all experimentation pilots conducted over the course of the project, offering valuable insights into how users interacted with the ROLEPL-AI training environment—and how it can continue to evolve.
The aim of these pilots was to evaluate how learners, trainees, and educators experienced ROLEPL-AI, a simulation-based platform designed to support the development of soft skills through immersive virtual environments and AI-driven interactions.
Following the evaluation framework defined in earlier stages of the project, we gathered feedback from participants using standardized questionnaires focused on four main areas:
Soft skills self-efficacy
Sense of presence
Flow experience
User experience (UX)
For the second experimentation phase, we also included feedback on participants' attitudes toward AI and detailed evaluations from educators involved in the pilots and the project's training day.
Key Findings
Moderate levels of presence and flow were reported, showing promise in terms of user engagement with the simulation.
Learners generally found the environment immersive, but challenges in navigation, technical stability, and clarity of tasks limited the learning impact in some cases.
Feedback varied significantly between pilot groups, with factors such as language proficiency, technical familiarity, and preparation playing a major role in perceived experience.
Educators highlighted the importance of better onboarding and clearer task structures to support meaningful learning outcomes.
Understanding how users experience ROLEPL-AI is essential to developing an effective educational tool. This analysis not only validates design choices but also uncovers real-world challenges in implementing AI-based simulations in education and vocational training.
The insights from this report will feed directly into the next steps of the project, including preparation for broader transferability (D5.4) and continued dissemination and engagement with educators, learners, and stakeholders.
Comments