Comprehensive Summary

To develop a model followed by a prototype application using XR technology that is capable in improving the learning experience of the learners with hearing and speech impairments.

Project Justification

There are about 385 million people who suffer from speech impairment. Meanwhile, the Malaysia Statistic Department recorded 66141 people with either hearing or speech impairments. One of the major challenges in interacting with these people are due to the communication approaches. Commonly, people communicate with one another via speech, especially when they are engaged in longer conversation. However, due to the impairments, the concerned group of people will not be able to communicate effectively with normal people. Their common communication approaches are either their sign language or in textual manner. It is a significant challenge as common people usually do not learn and understand this sign language. Thus, the alternative approach is to utilize paper and pen. However, it is time consuming and extra effort is required in converting a speech to text, especially when the audience comprises of both common people and the concerned people with impairment. By incorporating the extended reality (XR) technology, the superimposing of both the physical visuals such as the gesture movement and the virtually generated graphics such as the text or sign language may be achieved for a better learning experience of this focus group of learners while providing a usual and perhaps more convenient delivery mode to teachers or presenters.

Objectives

1. To identify, understand, and compare the current supporting models among learners with the hearing and speech impairments.
2. To propose an immersive model of XR application to support these learners with special needs in improving their learning experience
3. To validate the proposed XR model to ensure that all requirements have been met and fulfilled.

Phases Of Work

Estimated Budget

Project Timeline