# Problem Summary

Last updated November 23, 2015

CTAT 3.2

## Problem Summary

Below is a sample problem summary compiled at the end of a problem. This case shows a problem with 8 required steps (see notes below):

<ProblemSummary ProblemName="problemName" CompletionStatus="incomplete" Correct="8" UniqueCorrect="4" UniqueCorrectUnassisted="2" Hints="5" UniqueHints="4" HintsOnly="2" Errors="5" UniqueErrors="4" ErrorsOnly="2" UniqueSteps="8" RequiredSteps="9" TimeElapsed="31343"> <Skills> see more detail below... </Skills> </ProblemSummary>

note | name | sample | description |
---|---|---|---|

ProblemName | aProblem | brd and start state name | |

CompletionStatus | incomplete | complete means a successful Done step; else incomplete | |

1 | Correct | 8 | raw count of correct responses |

2 | UniqueCorrect | 4 | unique steps with only hint or correct responses |

3 | UniqueCorrectUnassisted | 2 | unique steps with only correct responses, no hints |

4 | Hints | 5 | raw count of hint requests, not counting 2nd- or other-level hints |

5 | UniqueHints | 4 | unique steps with hint requests |

6 | HintsOnly | 2 | unique steps with hint requests but no errors |

7 | Errors | 5 | raw count of incorrect responses |

8 | UniqueErrors | 4 | unique steps with error responses |

9 | ErrorsOnly | 2 | unique steps with error responses and no hint requests |

UniqueSteps | 8 | number of different steps attempted | |

RequiredSteps | 9 | number of different steps required for completion | |

TimeElapsed | 31343 | milliseconds from problem initialization until successful Done step |

Notes:

- The student had to eventually answer all 8 steps correctly to proceed.
- On 4 different steps, the student may have requested a hint, but committed no error.
- On 2 different steps, the student answered correctly on the first try without requesting a hint.
- The student asked for a top-level hint 5 times, not counting requests for 2nd-level hints.
- The 5 hint requests were for 4 different steps (student requested a hint on 1 step twice).
- On 2 different steps, the student did request a hint, but committed no error.
- The student entered incorrect answers 5 times.
- The student entered incorrect answers on 4 different steps (got 1 step wrong twice).
- On 2 different steps, the student entered an incorrect answer but didn't request a hint.

## Skills

In addition to the summary information, the tutor transmits updated values for the skills of interest. The attributes for each skill are as follows:

Name | Description |
---|---|

name | Unique name for the skill within the category (namespace) |

label | Name visible to the student in the skill meter display |

category | Namespace for the skill name |

description | Description shown on skill reports |

pKnown | The probability that the student has mastered the skill |

pGuess | The probability that the student would guess correctly without having mastered the skill |

pSlip | The probability that the student would err after having mastered the skill |

pLearn | The probability that the student would come to master the skill |

Here are example skills from an exercise in adding fractions:

<Skills>

<Skill name="reduce-numerator" label="reduce-numerator" category="fraction-addition" description="Calculate numerator when reducing a fraction to lowest terms." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="reduce-denominator" label="reduce-denominator" category="fraction-addition" description="Calculate denominator when reducing a fraction to lowest terms." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="copy-answer-denominator" label="copy-answer-denominator" category="fraction-addition" description="Copy the common denominator from the addends to the sum." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="add-numerators" label="add-numerators" category="fraction-addition" description="Add numerators after addends have a common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="convert-numerator" label="convert-numerator" category="fraction-addition" description="Calculate numerator when converting addends to have a common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="multiply-denominators" label="multiply-denominators" category="fraction-addition" description="Multiply the denominators to calculate a common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

<Skill name="determine-lcd" label="determine-lcd" category="fraction-addition" description="Calculate the least common denominator." pKnown="0.25" pGuess="0.1" pSlip="0.2" pLearn="0.15" />

</Skills>

## CTAT and SCORM

How some CTAT data elements could be used in SCORM:

SCORM Data Model | CTAT Data Element | Notes |
---|---|---|

cmi.score | correct count | The preferred score in SCORM is cmi.score.scaled, which is a number in the interval [-1.0,1.0]. In CTAT this could be calculated various ways, including uniqueCorrectUnassisted/uniqueStepCount. SCORM also permits a raw score, which can be any numeric value. |

cmi.completion_status | CompletionStatus | Both SCORM and CTAT have statuses of not attempted, incomplete and complete. |

cmi.interactions | steps | Each interaction could be a step attempt (a single {selection,action,input}) or a step (identified by {selection,action}). |

cmi.objectives | knowledge components (skills) | Both SCORM objectives and CTAT knowledge components refer to the acquisition of competence in some domain-specific subfield. Both use a numeric metric between 0.0 and 1.0, both have a concept of mastery, both can be used for problem sequencing. |