How does OttoLearn measure proficiency?
For a learner to increase their mastery level for a Concept, they must first complete a related Activity proficiently (not just accurately).
OttoLearn has a proprietary algorithm that pulls in a number of data points during a learner’s completion of an Activity. This data is used to determine whether the learner is proficient (knew the answer) or not proficient (didn’t know, or was likely guessing).
Some of the factors OttoLearn uses to measure proficiency are:
- Past performance
- Comparison to other learners
- Other signals
A measure of whether the learner answered an Activity correctly or incorrectly.
Each learner must self-assess their confidence level. This forces the learner to reflect on their choice and sometimes change to another one.
We track the learner's self-identified confidence rating, and have a gamified algorithm that rewards learners for properly identifying their confidence level about an answer.
The signature for how long it takes each learner to complete Activities in general — across multiple content areas. This allows us to build a model for how the learner generally responds to Activities, which we use as a baseline measure.
Comparative Duration Signature
When a learner completes an Activity, we compare their duration signature for that Activity (adjusted for their baseline) against their past performance and the duration signatures of other learners who completed that same Activity at the same Mastery Level.
ProTip: Ottolearn develops a personal duration signature for each learner. This ensures that the system won't penalize learners who always answer a little bit slower, for example.
OttoLearn uses other signals, and modifies the algorithm from time-to-time, to provide the most accurate representation of whether a learner has completed an Activity proficiently or not.