You Are Here


Tutor definition: A speech-enabled tutor exercise created by Jonathan Brown.

Tutor Interface: You Are Here

History: This tutor exercise was created by Jonathan Brown during the course “Language Technologies for Computer Assisted Language Technologies”, during the portion of the course taught by Maxine Eskenazi. This exercise uses the Sphinx2-CTAT integration tools that Jonathan Brown also developed. These tools allow tutor designers to add speech-recognition functionality to their tutors without any programming.

How the interface was built and how the CTAT tools were used to build the tutor: The interface was built in the NetBeans IDE by adding Java Interface Widgets to the interface template. Event handlers were added for the record and stop buttons, which use the Sphinx2-CTAT integration tools to allow students to speak their responses instead of typing them. A template that incorporates this functionality is available with the Sphinx2-CTAT integration tools. These tools are freely-available at the Sphinx2-CTAT Integration website given below. They are also described in the Technical Report and Book Chapter cited below.

What distinctive features this tutor has that not all tutors have: This tutor exercise employs speech recognition functionality. Using the Sphinx2-CTAT integration tools, this exercise makes use of the Sphinx-2 open source speech recognition system. Students are able to speak their responses, and the tutor will determine which of the expected student answers, if any, were spoken. The exercise then triggers an event which causes the CTAT tools to respond to the student’s input as if the student had typed it. This exercise also employs a technique to automatically load new content into the interface. That is, it has functionality for representing what would traditionally be multiple exercises as one exercise. This functionality is used to change the map image and sentence choices as the student progresses through the problem, while allowing the widget on the left to have the list of sentences the student has spoken thus far. The content to display at each step is defined in a text file, which the code automatically parses and displays.

How interface utilizes scaffolding (make easier for learners by make steps explicit for students instead of implicit) & differ from standard textbook notation: The map continuously updates as the student progresses, and helps to see what to do next. Each of the student’s utterances is also shown in the widget on the left. This assists with the more global objective of compiling a set of directions for the character to follow to the destination.

Scale of delivery: This tutor exercise was created for a class project, and has not been deployed to any learners. The speech recognition integration technology behind it is expected to be developed further, at which time additional tutors will be created.

Publications:

  1. Brown, J. Integrating Tools for the Creation of Speech-Enabled? Tutors. CMU LTI Technical Report CMU-LTI-04-186. 2004.
  2. Eskenazi, M., Brown, J. "Teaching the Creation of Software that Uses Speech Recognition." Teacher Education in CALL: An International Perspective. Ed. Phil Hubbard and Michael Levy. John Benjamins, Summer 2005.

For more information about this project, please contact: Jonathan Brown, Language Technologies Institute, Carnegie Mellon University. E-mail: jonbrown@cs.cmu.edu. Personal Website: http://www.cs.cmu.edu/~jonbrown/. Sphinx2-CTAT Integration Website: http://www.cs.cmu.edu/~jonbrown/Sphinx2-CTAT.