Video games at school?

Khan, a master’s degree student in applied computing, used the course Topics in Human-Computer Interaction (HCI) to explore the ways Kinect might be used in elementary school classrooms for gesture and speech recognition.

The course focused on user experiences with next-generation input and output technologies. Students read and discussed papers from leading researchers around the world, and had the opportunity to apply their teachings to building their own interactive system using exciting new technologies. Khan decided to use the Microsoft Kinect as new hardware.

Khan enjoyed developing some ways that the Kinect might be used for pedagogical purposes. She explained: “While we discussed various HCI papers in class, I constantly found myself applying the research ideas and techniques specifically to the user group of children… I thought that applications of these techniques could help tremendously in early childhood education.”

Inspired by course readings, Khan developed prototypes in which the user participates in activities by using voice commands and gestures – “pointing to objects on screen and using voice commands to select them.” In her research examples, the voice command could be a simple “this” or “that”, or be a more specific naming activity (e.g., “horse”), based on the activity. (Watch a related video.)

Khan said, “The power of using gesture and speech-based systems in classrooms could not only make an interactive and fun experience, but could also simplify a lot of complex learning.”

Khan tested out her prototypes on her daughters, ages four and seven, and found the use of the Kinect made the activities, such as counting, classification, patterning, and identification, very effective and entertaining for her “usability testers.” She was also able to test her work in her daughter’s Junior Kindergarten classroom.

“As a mother involved in [my daughters’] early years learning development,” Khan noted, “I definitely see the potential use of this technology in education.”

When asked about future plans for research in this area, the graduate student said, “I plan to continue developing more simple prototypes, demonstrating my ideas that can be adapted to classroom education. I also hope to explore this space in the development of assistive and rehabilitative technologies.”

Work with next-generation technologies offers a whole host of life-changing possibilities.

Former Olympian Bruce Kidd, a professor of Kinesiology and Physical Education, is the new interim Warden of Hart House. Join him for some informal conversation.