Research Projects
This page contains information about different research projects at the lab.
- Generating Animations of American Sign Language
- Text Simplification for Adults with Intellectual Disabilities
- Evaluating Parameters for ASL Animations
- Design and Evaluation of User-Interfaces for Deaf Users
- Educational Software for Deaf Users
- NLP for Low-Density Languages
Generating Animations of American Sign Language
The goal of this research is to develop technologies to generate animations of a virtual human character performing American Sign Language. The motivation for this work is to make more information and services accessible to ASL signers who may have lower levels of written English literacy. A particular focus of this work has been how the use of space around an ASL signer affects the 3D movement aspects of the sign language performance. This project will use motion-capture technology to collect a corpus of ASL from native signers; this corpus will allow us to study the way in which signers use the 3D space around them.
People:
Pengfei Lu, Matt Huenerfauth, Jonathan Lamberton, Lijun Feng, Amanda Krieger, Kelsey Gallagher, Wesley Clarke, Aaron Pagan
Funding:
Matt Huenerfauth, PI. June 2008 to May 2013. “CAREER: Learning to Generate American Sign Language Animation through Motion-Capture and Participation of Native ASL Signers.” National Science Foundation, Faculty Early Career Development (CAREER) Award Program, CISE Directorate, IIS Division, HCC Cluster. Amount: $581,496.
Matt Huenerfauth, PI. June 2007 to June 2011. “Generating Animations of American Sign Language.” Go PLM Grant Program. Siemens A&D UGS PLM Software. Amount: $633,150.
Visage Technologies software: Visage Life and Visage Interactive. This project uses character animation software from Visage Technologies AB (www.visagetechnologies.com) under the free Academic License.
Recent Papers:
Pengfei Lu, Matt Huenerfauth. (in press, 2009). “Accessible Motion-Capture Glove Calibration Protocol for Recording Sign Language Data from Deaf Subjects.” In Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2009), Pittsburgh, Pennsylvania, USA.
Matt Huenerfauth. 2009. "Improving Spatial Reference in American Sign Language Animation through Data Collection from Native ASL Signers." In Proceedings of the International Conference on Universal Access in Human-Computer Interaction. San Diego, CA.
Matt Huenerfauth. 2009. "A Linguistically Motivated Model for Speed and Pausing in Animations of American Sign Language." ACM Transactions on Accessible Computing.
Webpage for Project:
http://latlab.cs.qc.cuny.edu/nsf0746556/
Text Simplification for Adults with Intellectual Disabilities
Natural language processing technology has great potential to revise and re-generate textual information that meets the special needs of individual readers. In particular, text summarization and simplification/transformation technologies may benefit adults with intellectual disabilities. This project will explore how NLP techniques can make texts more accessible and enjoyable for these readers. An important first goal of this project is to develop software that can automatically detect the readability of a text for adults with intellectual disabilities.
People:
Lijun Feng, Noemie Elhadad (Columbia University), Matt Huenerfauth
Funding:
Matt Huenerfauth, PI. December 2008. “Text readability software for adults with intellectual disabilities.” Research Enhancement Committee, Queens College, The City University of New York. Amount: $10,000.
Recent Papers:
Matt Huenerfauth, Lijun Feng, Noemie Elhadad. (in press, 2009). “Comparing Evaluation Techniques for Text Readability Software for Adults with Intellectual Disabilities.” In Proceedings of the 11th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2009), Pittsburgh, Pennsylvania, USA.
Lijun Feng, Noemie Elhadad, Matt Huenerfauth. (in press, 2009). “Cognitively Motivated Features for Readability Assessment,” Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2009), Athens, Greece.
Evaluating Parameters for ASL Animations
This project will empirically determine guidelines for how to best set parameters of speed and timing needed to generate animations of American Sign Language. The goal of this work is to learn how to make these animations more understandable and natural-looking for ASL signers.
People:
Jonathan Lamberton, Matt Huenerfauth
Funding:
Matt Huenerfauth, PI. July 2007 to December 2008. “Evaluating Parameters for American Sign Language Animations.” Professional Staff Congress - City University of New York (PSC-CUNY) Research Award Program, Out-Of-Cycle Round 38. Amount: $4,095.
Recent Papers:
Matt Huenerfauth. (in press, 2009). "A Linguistically Motivated Model for Speed and Pausing in Animations of American Sign Language." ACM Transactions on Accessible Computing.
Matt Huenerfauth. 2008. "Evaluation of a Psycholinguistically Motivated Timing Model for Animations of American Sign Language." The 10th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS 2008), Halifax, Nova Scotia, Canada.
Design and Evaluation of User-Interfaces for Deaf Users
Users of computers who are deaf may benefit from specially designed user-interfaces that include textual captioning of audio information, animations of sign language, videos of sign language, and other multimodal features. The goal of this project will be to determine the best practices in designing and evaluating user-interfaces with these features.
People:
Allen Harper, Matt Huenerfauth
Funding:
Award to Computer Science Department (participating faculty: Jinlin Chen, Matt Huenerfauth, Christopher Vickery). 2009. "Eye-Tracking Analysis for User Interface Design." Graduate Investment Initiative 2009, Queens College, The City University of New York. Total Amount: $30,000.
Educational Software for Deaf Users
We are developing educational tools including animations of a human character for deaf or hard-of-hearing students to practice their communication skills.
People:
Matt Huenerfauth
Funding:
Matt Huenerfauth, PI. July 2008 to June 2009. “Educational Software for Deaf Users.” Professional Staff Congress - City University of New York (PSC-CUNY) Research Award Program, Regular-Cycle Round 39. Amount: $3,800.
Recent Papers:
Matt Huenerfauth. 2009. “Improving Spatial Reference in American Sign Language Animation through Data Collection from Native ASL Signers.” International Conference on Universal Access in Human-Computer Interaction (UAHCI). San Diego, CA. July 2009. In C. Stephanidis (Ed.), Universal Access in HCI, Part III, HCII 2009, LNCS 5616, pp. 530–539, 2009. Berlin/Heidelberg: Springer-Verlag.
NLP for Low-Density Languages
Low density languages are human languages (like ASL or Yiddish) that lack extensive electronic linguistic resources (such as parallel corpora, computational lexicons, etc.) that are used by traditional statistical natural language processing techniques. New approaches for developing computational linguistic software for these languages must be developed. This project will explore transliteration and translation technologies for such languages.
People:
Joshua Waxman, Matt Huenerfauth