Improving Speech Recognition for Japanese Deaf and Hard-of-Hearing People by Replacing Encoder Layers

EasyChair Preprint 15118

Abstract

Communication between hearing individuals and those with hearing impairments generally involves sign language, written communication, and speech. It has been reported that more than half of Japanese people with hearing impairments communicate using speech. Therefore, speech recognition systems available for individuals with hearing impairments are demanded. However, speech recognition systems trained on speech from hearing individuals do not achieve high recognition accuracy for speech from individuals with hearing impairments. In this study, we propose a method to replace the encoder layer of the speech recognition model based on SSL to achieve high-accuracy speech recognition for speech from individuals with hearing impairments. By this method, we improved the recognition performance for significantly speech from individuals with hearing impairments.

Keyphrases: Automatic Speech Recognition, Domain Adaptation, deaf speech, self-supervised learning

@booklet{EasyChair:15118,
  author    = {Kaito Takahashi and Yukoh Wakabayashi and Kengo Ohta and Akio Kobayashi and Norihide Kitaoka},
  title     = {Improving Speech Recognition for Japanese Deaf and Hard-of-Hearing People by Replacing Encoder Layers},
  howpublished = {EasyChair Preprint 15118},
  year      = {EasyChair, 2024}}