Research: Creatium | Creatium
AI Coaching
The proof is in the results
28% greater test score improvement with our interactive coach
- +0.48 effect size, above the +0.37 average for one-to-one human tutoring
- The coach adapts to what each learner needs most
- The same video, the same learners, one difference: a few minutes of conversation
Experimental Evidence
- Experimental evidence that conversational AI improves learning outcomes
- Quantifying the learning benefits of AI-enhanced video instruction
- Dialogic assessment reduces cheating
- AI voices excel despite being identifiable
- The science of smarter learning with AI avatars
- Transforming video production with AI
Learning Outcomes
Test Score Improvement
Pretest and Posttest Score
Test score:
Pretest: 45%
Posttest: 79%Learning videos:
Pretest: 42%
Posttest: 85%
Creatium’s AI coach adapts to what the learner needs most.
Two hundred learners participated, with the same video and one difference: a few minutes of conversation.
Interactive Video Assessment
- Effect Hedge’s g: +0.36 overall, +0.67 for learners with lower prior knowledge
Mean scores:- 76.3% for video only
- 84.2% for interactive video by Creatium
220 teachers were randomized.
AI-powered simulations require learners to immediately apply their knowledge, incorporating answering, sorting, and roleplaying instead of passive watching.
Cheating Rates by Assessment Format
- 52.9% for multiple choice
- 14.7% for dialogue-based assessment from Creatium
Natural conversations are a better reflection of understanding; written answers can be pasted from anywhere, whereas spoken responses require comprehension first.
Voice Quality Assessment
Mean Rating on a 7-Point Scale
- 5.48 for Creatium’s AI voices
- 5.08 for human voice
The consistent quality of delivery from Creatium's AI provides a clear and engaging experience.
Avatar vs. Audio Narration
- Mean post-test percentage:
- 75% for audio
- 79.88% for Creatium’s avatars
Avatar-based videos foster more engagement than audio-only instructional videos, leading to better results in less time.
Video Production Time
- Mean time to produce one finished video:
- 10.63 minutes for Creatium’s AI
- 49.27 minutes for traditional methods
Creatium reduces weeks of production to just an afternoon.
From our avatar to the custom personas, rich narrations, and engaging interactives . . . I don’t feel like I’m watching an avatar—I feel connected. It’s rich and it’s even powerful.
— Juanita Montes de Oca, Director, Instructional Design, Generation Citizen
Creatium’s tool stands out for its thoughtful design and clear alignment with the learning design process. It’s grounded in evidence and research, and the way it leverages multi-modes of AI offers a compelling example of how technology can meaningfully support educators and designers.
— Dr. Amin Marei, Researcher, Harvard Graduate School of Education
Creatium is the greatest improvement to professional learning that I have ever witnessed in my 30+ year career.
— Wendy Amato, PhD, Teaching Channel
Research Insights
Topics Covered
- AI in education
- Assessment innovation
- Instructional design