PopSign: Teaching American Sign Language Using Mobile Games

Faculty: 
Thad Starner
Students: 
Bianca Copello, Domino Weir, Ellie Goebel, Cheryl Wang

CopyCat and PopSign are two games that help deaf children and their parents acquire language skills in American Sign Language. 95% of deaf children are born to hearing parents, and most of those parents never learn enough sign language to teach their children. This puts them at high risk of suffering from language deprivation, which has emotional, mental, and cognitive effects that can be irreversible. For example, short-term memory skills are learned from acquiring a language, and many deaf children enter school with a short-term memory of fewer than 3 items, much less than hearing children of hearing parents or Deaf children of Deaf parents. Their language skills also suffer, and they may be unable to learn either English or ASL fluently if they pass the critical language acquisition stage of development without proper language stimulus. Our systems address this problem from two different angles. PopSign teaches parents ASL vocabulary, since parents are the main vehicle through which children learn language. CopyCat improves children's short-term memory skills through ASL games. 

Lab: 
Director: 
Thad Starner

The Contextual Computing Group (CCG) creates wearable and ubiquitous computing technologies using techniques from artificial intelligence (AI) and human-computer interaction (HCI). We focus on giving users superpowers through augmenting their senses, improving learning, and providing intelligent assistants in everyday life. Members' long-term projects have included creating wearable computers (Google Glass), teaching manual skills without attention (Passive Haptic Learning), improving hand sensation after traumatic injury (Passive Haptic Rehabilitation), educational technology for the Deaf community, and communicating with dogs and dolphins through computer interfaces (Animal Computer Interaction).