Reference Home

Problem Summary

Last updated November 23, 2015

CTAT 3.2


Problem Summary

Below is a sample problem summary compiled at the end of a problem. This case shows a problem with 8 required steps (see notes below):

    <ProblemSummary ProblemName="problemName" CompletionStatus="incomplete"
            Correct="8" UniqueCorrect="4" UniqueCorrectUnassisted="2"
            Hints="5" UniqueHints="4" HintsOnly="2"
            Errors="5" UniqueErrors="4" ErrorsOnly="2"
            UniqueSteps="8" RequiredSteps="9" TimeElapsed="31343">
        <Skills> see more detail below... </Skills>
    </ProblemSummary>

note name sample description
ProblemName aProblem brd and start state name
CompletionStatus incomplete complete means a successful Done step; else incomplete
1Correct 8 raw count of correct responses
2UniqueCorrect 4 unique steps with only hint or correct responses
3UniqueCorrectUnassisted 2 unique steps with only correct responses, no hints
4Hints 5 raw count of hint requests, not counting 2nd- or other-level hints
5UniqueHints 4 unique steps with hint requests
6HintsOnly 2 unique steps with hint requests but no errors
7Errors 5 raw count of incorrect responses
8UniqueErrors 4 unique steps with error responses
9ErrorsOnly 2 unique steps with error responses and no hint requests
UniqueSteps 8 number of different steps attempted
RequiredSteps 9 number of different steps required for completion
TimeElapsed 31343 milliseconds from problem initialization until successful Done step

Notes:

  1. The student had to eventually answer all 8 steps correctly to proceed.
  2. On 4 different steps, the student may have requested a hint, but committed no error.
  3. On 2 different steps, the student answered correctly on the first try without requesting a hint.
  4. The student asked for a top-level hint 5 times, not counting requests for 2nd-level hints.
  5. The 5 hint requests were for 4 different steps (student requested a hint on 1 step twice).
  6. On 2 different steps, the student did request a hint, but committed no error.
  7. The student entered incorrect answers 5 times.
  8. The student entered incorrect answers on 4 different steps (got 1 step wrong twice).
  9. On 2 different steps, the student entered an incorrect answer but didn't request a hint.

Skills

In addition to the summary information, the tutor transmits updated values for the skills of interest. The attributes for each skill are as follows:

Name Description
nameUnique name for the skill within the category (namespace)
labelName visible to the student in the skill meter display
categoryNamespace for the skill name
descriptionDescription shown on skill reports
pKnownThe probability that the student has mastered the skill
pGuessThe probability that the student would guess correctly without having mastered the skill
pSlipThe probability that the student would err after having mastered the skill
pLearnThe probability that the student would come to master the skill

Here are example skills from an exercise in adding fractions:

<Skills>

<Skill name="reduce-numerator" label="reduce-numerator" category="fraction-addition" description="Calculate numerator when reducing a fraction to lowest terms." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="reduce-denominator" label="reduce-denominator" category="fraction-addition" description="Calculate denominator when reducing a fraction to lowest terms." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="copy-answer-denominator" label="copy-answer-denominator" category="fraction-addition" description="Copy the common denominator from the addends to the sum." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="add-numerators" label="add-numerators" category="fraction-addition" description="Add numerators after addends have a common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="convert-numerator" label="convert-numerator" category="fraction-addition" description="Calculate numerator when converting addends to have a common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="multiply-denominators" label="multiply-denominators" category="fraction-addition" description="Multiply the denominators to calculate a common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="determine-lcd" label="determine-lcd" category="fraction-addition" description="Calculate the least common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

</Skills>

CTAT and SCORM

How some CTAT data elements could be used in SCORM:

SCORM Data Model CTAT Data Element Notes
cmi.scorecorrect countThe preferred score in SCORM is cmi.score.scaled, which is a number in the interval [-1.0,1.0]. In CTAT this could be calculated various ways, including uniqueCorrectUnassisted/uniqueStepCount. SCORM also permits a raw score, which can be any numeric value.
cmi.completion_statusCompletionStatusBoth SCORM and CTAT have statuses of not attempted, incomplete and complete.
cmi.interactionsstepsEach interaction could be a step attempt (a single {selection,action,input}) or a step (identified by {selection,action}).
cmi.objectivesknowledge components (skills)Both SCORM objectives and CTAT knowledge components refer to the acquisition of competence in some domain-specific subfield. Both use a numeric metric between 0.0 and 1.0, both have a concept of mastery, both can be used for problem sequencing.