This paper outlines our efforts in developing Indonesian speech recognition for hearing and speaking impaired people. The lack of speech-enabling technology and research, as well as a shortage of data on the Indonesian language presents a major challenge for us to deal with. Difficulties arise in developing an Indonesian speech corpus since Indonesian is actually most people's second language after their own ethnic native language. Collecting all of the possible languages and dialects of the tribes recognized in Indonesia is still the biggest problem we face. In speech recognition, segmented utterances according to labels are usually used as a starting point for training speech models. This segmentation strategy is also one of the main issues. Initialization training utterances with flat segmentation would not give sufficient performance. Here, we used an English speech recognizer to set initial segmentation of Indonesian utterances. This method produced a significant improvement of up to 40% in performance.