Search CTAT

Building a Cognitive Tutor for fraction addition

Last updated February 17, 2017

Table of Contents

In this tutorial you will learn to create and modify production rules for a fraction addition tutor. You will use the CTAT Jess tools along with the fraction addition materials supplied with CTAT to analyze and modify your cognitive model.

Note: Although this tutorial shows screenshots from the Flash version of fraction addition, you can follow this tutorial with either the Flash or Java version of fraction addition included with CTAT.

Prerequisites:

  • CTAT 2.0+
  • For Flash version of fraction addition:
    • Flash Player 7+
    • Student interface:
      {CTAT}\Projects\FlashFractionAddition\fractionAddition-mx.swf
    • BRDs, productionRules.pr, wmeTypes.clp, WME files:
      {CTAT}\Projects\FlashFractionAddition\CognitiveTutorProblems
  • For Java version of fraction addition:
    • Student interface:
      {CTAT}\Projects\FractionAdditionTutor\FractionAddition.class
    • BRD:
      {CTAT}\Projects\ProblemsOrganizer\Examples\Arithmetic\FractionAddition\
    • ProductionRules.pr, wmeTypes.clp, WME files:
      {CTAT}\Projects\FractionAdditionTutor\
  • (Optional) Eclipse with Jess plugin for editing production rules. Eclipse should be configured according to the configuration instructions.

1. The CTAT Jess Tools

CTAT provides a suite of Jess tools for working with a cognitive model and tutor. In this portion of the tutorial, we'll examine a complete model for fraction addition using the following tools:

  • Working Memory Editor
  • Conflict Tree
  • Why Not?
  • Eclipse

Let's start by launching these tools, as well as the Eclipse environment:

  1. Launch CTAT for Java if using a Java student interface, or CTAT for Flash if using a Flash interface.
  2. Set the Tutor Type to Cognitive Tutor (Jess). All Jess windows will open. (If you don't see Jess tool windows, select Windows > Restore Default View.)
  3. Open the student interface: for Java, click File > Open Student Interface and select the Java class file specified above. For Flash, open the interface SWF file specified above in a web browser or the Flash player by double-clicking it.
  4. Open a behavior graph (location specified above) by clicking File > Open Graph. For Java, you can browse for the file directly, or use the Problems Organizer (Unit Arithmetic, Section FractionAddition) to locate the behavior graph file.
  5. Check the Cognitive Model Folder preference in CTAT so that it can locate the Jess files for the fraction addition tutor.

    Note: It's important to be aware of the "Cognitive Model Folder" setting when creating a cognitive tutor. This is the folder containing the the production rules file (productionRules.pr), working memory templates (wmeTypes.clp), and the working memory file for the behavior graph you are using ({graph-name}.wme).

    You can tell what the current Cognitive Model Folder is by looking at the top of the WME Editor window. If this folder isn't the one you expect to be there, you can set the Cognitive Model Folder in the CTAT preferences by going to Tools > Preferences and scrolling down under the General category. The Cognitive Model Folder is set for you whenever you open a behavior graph or, in the case of CTAT for Java, when you open a student interface.

  6. Launch Eclipse.

2. Examining working memory

The Working Memory Editor window is one of the Jess tools available in CTAT. It can be used to inspect and modify the contents of the cognitive model's working memory, or a collection of facts.

To populate working memory:

  1. If you don't see the Working Memory (WME) Editor in CTAT, select Windows > Show Window > WME Editor.
  2. Select File > Open Graph and open a graph from the BRD directory specified above (any file ending in .BRD will work).
  3. You should see a few messages in the Jess Console window reporting that the Jess files were loading successfully:
    *****Engine Cleared*****
    
    Reading wmeTypes.clp (D:\apps\CTAT_2_9\Projects\FlashFractionAddition\
    	CognitiveTutorProblems\wmeTypes.clp)
    Reading productionRules.pr (D:\apps\CTAT_2_9\Projects\FlashFractionAddition\
    	CognitiveTutorProblems\productionRules.pr)
    Reading 1-3plus2-3.wme (D:\apps\CTAT_2_9\Projects\FlashFractionAddition\
    	CognitiveTutorProblems\1-3plus2-3.wme)
    
    If you don't see similar messages, ensure the Cognitive Model Folder is specified properly (see above note) and that you have a behavior graph opened.

The Working Memory Editor window shows the "facts", or contents of working memory, for the current problem. You'll notice that a majority of these facts mirror the set of widgets in the student interface. Why would working memory mirror the interface? The structure of a problem often (although not always) follows the interface. This concept should become clearer as you work through the tutorial.

In working memory, there's always a key fact called problem; and all the other facts are linked to that fact. The fact problem always has a slot called interface-elements which contain facts corresponding to each of the widgets in the interface—in this example, Help and Done buttons, and a number of TextArea widgets. (Most of the time, you will ignore the working memory facts for the Help and Done buttons, since these must be on every interface you design and should not be deleted.)

working memory editor

Figure 1. Working Memory Editor with fraction template and facts shown

But textArea widgets alone are not useful in a cognitive model. Much more useful is the concept of a fraction, a collection of textArea facts in a unit that represents a fraction. To create fractions, we've extended working memory to include a template type of fraction. You can view the current problem's fractions by expanding the fraction template, and clicking on the various fraction facts (see Figure 1). These facts contain references to the widgets in the interface. To find a particular value, we need to follow these references to the textArea widget.

For example, since the start state for the problem 1/3 + 2/3 has 1, 3, 2, and 3 entered in the textArea widgets of the interface, you can view these values in the Working Memory Editor by clicking on the textArea template, and the facts within it (e.g., firstNumGiven, firstDenGiven). These facts are referenced from the various fraction facts: within the fraction template, click on givenFraction1; its numerator slot has a value of <Fact-7>, which maps to the firstNumGiven interface widget (which in turn holds the value of "1").

You can do quite a lot with the Working Memory Editor besides just viewing the facts. From here, you could edit the start state and save it (useful when the start state the tools generate for you is not sufficient).

You may also notice three templates that begin with "special-tutor-fact." These are necessary to maintain backwards compatibility with older models, and should be ignored.

3. Examining Templates, Facts, and Production Rules in Eclipse

The Working Memory Editor visualizes working memory and makes the facts it contains easily accessible. But where is this initial set of data stored? And what if we want to do a larger, more powerful overhaul to working memory or the fact types?

3.1 Working Memory types (templates)

The types, or templates, for facts are stored in the file wmeTypes.clp, which is generated by CTAT during problem authoring. We can open the file in Eclipse to see its contents. For fraction addition, this file was modified to include new templates such as add-fractions-goal, convert-fraction-goal, and fraction. The type problem was also amended to consist of given-fractions, converted-fractions, and answer-fractions. We refer to this process as augmenting working memory.

In Eclipse, all deftemplate statements are collapsed by default. You expand each statement by clicking the "+" icon next to the start of each deftemplate statement.

3.2 Fact assertions, slot assignments

The initial set of variables and facts used in working memory is defined in a file called {problem-name}.wme. This file must have the same problem name as the BRD file that accompanies it.

Open the file in the Eclipse IDE to see its contents. While covering the syntax of the Jess language is beyond the scope of this tutorial, you should note the fact assertions for the various fraction types, and the slot assignments for each.

Like wmeTypes.clp, the .wme file is automatically generated by CTAT to aid in cognitive modeling. As we will see later, modifying the list of facts that will appear in working memory is also useful.

3.3 Production rules

Now that we've explored the two main Jess files for supporting working memory, we can delve into the meat of the cognitive model: the production rules file.

Production rules files in CTAT tutors have the name productionRules.pr, and must reside in the same directory as the rest of the cognitive model files. You can open a production rules file by selecting it from within the Eclipse IDE.

Tip: If you don't see a file in Eclipse, try refreshing Eclipse's view: select the project in Eclipse that contains the CTAT files and press F5 on the keyboard.

The production rules for fraction addition are each prefaced by the name of the rule, and an English version of the Jess production rule. The English version is pseudo-code, or an algorithmic description of what the rule does. This English version has two parts: IF and THEN. IF corresponds to the left-hand side (LHS) of the rule; THEN corresponds to the right-hand side (RHS). Good English versions of rules help not only during authoring and debugging, but when someone who's not familiar with the rule tries to understand it.

We'll revisit the production rules file after experimenting with the fraction addition tutor.

4. The Conflict Tree

When the student performs a step in the interface, the model-tracing algorithm searches for a sequence of rule activations that produce the same action as the student. This search is represented visually in the Conflict Tree.

The Conflict Tree can best be understood by demonstration. So let's enter a value into the interface, and examine the Conflict Tree:

  1. With the problem 1/3 + 2/3 open, set CTAT to Test Tutor mode. Also ensure that the Tutor Type is set to Cognitive Tutor (Jess).
  2. In the interface, enter the value 3 as the numerator of the answer fraction at the bottom.
CTAT Conflict Tree

Figure 2. The Conflict Tree after entering the numerator of the answer fraction.

The number 3 should turn green, since 3 is a valid numerator for the answer fraction. In the Conflict Tree, a number of rows should appear as shown in Figure 2.

We can see that the rule same-denominators fired first since it's the first item in the main Chain. But why did this rule match, and what were its effects? We can find out by clicking the rule same-denominators.

Clicking the rule same-denominators brings up a new colorful window. Some green text tells us that the LHS matched. In the top half of the window, the rule definition is displayed, with variables highlighted. Hover your mouse cursor over the variable ?d. A tool-tip tells you that the value is 3. Lines 5 and 6 of the rule represent a pattern that tests if both denominators are the same value.

What was the effect of this rule's activation? We look at the right-hand side of the rule--the Jess code after the => delimiter--and see that a new sub-goal was created: add-fractions-goal. This goal is now visible in the Working Memory Editor under the template add-fractions-goal.

The chain of rule activations didn't stop there. As you can see in the Conflict Tree, the rule add-numerators also activated. Furthermore, three green checks appeared to the left of the rule name in the columns S, A, and I. Clicking on any of the checks shows more information about the model during the past step: the rule's prediction and the student's entered value. According to the rule add-numerators, the model predicted an input of 3 on the widget ansNum1, and the student (you, in this case) entered 3 into the same widget.

Let's enter an incorrect value to see what the model does. Enter 4 as the denominator of the answer fraction. The value turns red, and the Conflict Tree updates with new information. If you click the first rule copy-answer-denominator, you see that the model predicted the input 3, but we demonstrated 4, a clear mismatch.

Replace the incorrect denominator in the interface with the correct value, 3. Three green checks appear next to the rule copy-answer-denominator; the model matches the student input.

We can also see how hints are defined for a problem. Click the Hint button to see a hint. At this point in the tutor, if we click the name of the rule in the Conflict Tree, we see that the hint is defined in the RHS of the rule. (It even references a few variables.)

4. Create a new problem

We're now ready to create a new fraction addition problem. We'll create a new BRD for the problem 1/4 + 3/8, as well as a new WME file. To do so, we can use a .WME file from an existing problem as a template.

For Java:

  1. In Windows Explorer or Finder, duplicate the file 1-4_plus_1-6.wme and save it in the same directory ({CTAT}\Projects\ProblemsOrganizer\Examples\Arithmetic\FractionAddition\).
  2. Give the duplicate file the name 1-4plus3-8.wme.
  3. Open the new file in Eclipse.
  4. Modify ?var11, with name of givenDenom2, to have a value of 8:
    ; MAIN::textField
    (modify ?var11
        (name givenDenom2)
        (value 6)
    )
  5. Save and close the file.

For Flash:

  1. In Windows Explorer or Finder, duplicate the file 1-4plus3-16.wme and save it in the same directory ({CTAT}\Projects\FlashFractionAddition\CognitiveTutorProblems).
  2. Give the duplicate file the name 1-4plus3-8.wme.
  3. Open the new file in Eclipse.
  4. Modify ?var11, with name of secDenGiven, to have a value of 8:
    ; MAIN::textArea
    (modify ?var11
        (name secDenGiven)
        (value 8)
    )
  5. Save and close the file.

We're now able to create a new problem in the Behavior Recorder:

  1. Click File > New Graph
  2. In the student interface, enter the fractions 1/4 and 1/8 as given fractions.

    Be sure to tab off of the last field, or it will not be recorded as part of the start state!

  3. In the Behavior Recorder, click Graph > Create Start State.
  4. Enter the name 1-4plus3-8.
  5. Click File > Save Graph As and name the graph 1-4plus3-8.

You should now have a working problem for 1/4 + 3/8. You can test the model in Test Tutor mode to verify that it works.

6. Writing a production rule for a new strategy

We're now going to discover a weakness of the current model, and create a new production rule to address it.

  1. Convert the first fraction, 1/4, by entering 2 for the numerator and 8 for the denominator
  2. Skip to the answer fraction, and enter 5, the correct numerator, as shown in Figure 3.
    The model considers this input to be incorrect.

Figure 3: The current model requires all fractions to be converted, even if one already has the LCD as its denominator.

What's going on here? The current model has no understanding of fractions where one denominator is a multiple of the other. Rather, it understands 1/4 and 1/8 as two fractions that both need to be converted!

We will now create a new rule called one-denominator-multiple-of-other

6.1 Write the rule

We first step to creating the rule one-denominator-multiple-of-other is to start with an existing similar rule as a template. The most similar rule in productionRules.pr is determine-lcd. In Eclipse, copy this entire rule to the space below it. Then modify the name of the rule to be our new rule one-denominator-multiple-of-other. Do this in both the comment at the top of the rule, and the line that starts (defrule .

6.2 Pseudo-code/English version

Next, we modify the English version of the rule to match the desired functionality of the new rule. We use the pattern-matching capabilities of Jess to define a succinct pseudo-code rule:

;; ONE-DENOMINATOR-MULTIPLE-OF-OTHER
;; IF there are no subgoals
;; and F1 is one of the given fractions
;; and F2 is the other given fraction
;; and D1 is the denominator of F1
;; and D2 is the denominator of F2
;; and D1 is a multiple of D2
;; THEN
;; Set a subgoal to convert F2 to denominator D1
;; Set a subgoal to add F1 and the converted fraction

The main differences are:

  • fractions are now variables;
  • salience of the rule is slightly higher to compensate for the overly general rule determine-lcd;
  • we test that the denominators are not equal to each other;
  • we test that one of the denominators divides evenly into the other with no remainder using the mod operation;
  • only one fraction must be converted; and
  • the two fractions to add are the fraction with the larger denominator and the new, converted fraction

6.3 Jess code version

You will now write the new Jess code for the rule by copy and paste.

(defrule one-denominator-multiple-of-other
   (declare (salience 200))
   ?problem <- (problem
      (given-fractions $? ?f1 $?)
      (subgoals))
   ?problem <- (problem (given-fractions $? ?f2 $?))
   (test (not (eq ?f1 ?f2)))
   ?f1 <- (fraction (denominator ?denom1))
   ?f2 <- (fraction (denominator ?denom2)
       (has-converted-form ?conv))
   ?denom1 <- (textArea (value ?d1&:(neq ?d1 nil)))
   ?denom2 <- (textArea (value ?d2&:(neq ?d2 nil)))
   (test (= 0 (mod ?d1 ?d2)))
=>
   (bind ?sub1 (assert (convert-fraction-goal  (fraction ?f2) 
      (denominator-value ?d1))))
   (bind ?sub2 (assert (add-fractions-goal (fractions ?f1 ?conv))))
   (modify ?problem (subgoals ?sub1 ?sub2))
   (construct-message
      "[ What denominator could you use to add the two fractions? ]"
      "[ Since " ?d1 " is a multiple of " ?d2 ", use " ?d1 " as the 
      denominator of the sum fraction.  ]"
      ))

Note: If using the Java version of fraction addition, replace textArea with textField above.

7. Evaluate and Test Rule

Save your new rule by selecting File > Save in Eclipse.

Reload the modified production rules file by switching to CTAT and clicking Graph > Jump to Start State. CTAT evaluates the validity of the file whenever you re-enter the start state of the graph.

With both the Conflict Tree and student interface visible, you are now ready to test your rule. As before, enter 2 as the numerator of the first converted fraction. The input should turn green. More importantly, the Conflict Tree should show that the rule you just added was activated.

Continue solving the problem, pausing to examine the Conflict Tree after each step. Be sure to skip conversion of the second fraction, as this is the test case that produced an error in the earlier model.

8. Why Not?

Often times while testing your production rules, a particular rule that you expect to fire won't. In this case, we can invoke the Why Not? window to tell us more. In the previous section, we had a functioning model where all rules that we expected to fire did. Let's augment the model with a logical error.

In Eclipse, replace the modulus operator mod with /, the operator for division. Save your production rules file and return to the start state.

Next, test the faulty model by entering 2/8 for the first converted fraction, and 5 as the numerator of the answer fraction. You'll notice that the input turns red, and the rule one-denominator-multiple-of-other did not fire. But why not?

To launch the Why Not? window, we first click the text Chain to show us a list of all production rules in the model. We then select one-denominator-multiple-of-other from this list (See Figure 3).

Figure 3. Asking Why Not? from the Conflict Tree

The Why Not? window appears with the following text in red:
In line 9: (= 0 (/ ?d1 ?d2)) did not return TRUE.

This means that the test on this line returned false, so the rule could not fully activate. This makes sense since the one denominator divided by the other equal denominator would yield 1, not 0. We can see that in this operation, the return value was FALSE; hovering over the highlighted portion of the rule in the top half of the window shows us this. This information is also displayed in the last line of the Why Not? window.

Why Not?

Figure 4. Why Not? one-denominator-multiple-of-other

Change your production rules back to use the mod operator. Test that your production rules work again by returning to the start state and evaluating the tutor in Test Tutor mode.

9. Conclusion

This concludes the Cognitive Tutor tutorial. If you'd like, you can expand the fraction addition model to account for different problem-solving strategies or student misconceptions.

Back to top