A new syllable decomposition method, which uses the tone information of the main vowel in a syllable to distinguish the tone of the whole syllable, is proposed in this paper. Compared to the scheme, in which a syllable is decomposed into an initial and a final, and the tone information is carried on by the final, the new scheme reduces the number of phonemes in the phone set of a recognition system. It handles the syllabic languages especially the ones with complicated tonal phonology such as Cantonese successfully, and also can be generalized to other tonal languages. Experiments on both Cantonese and Mandarin to compare the performance of systems using these two schemes, lead to that the new scheme got a little bit better accuracy than the old one while reduces the number of phonemes dramatically in recognition system. Such a method is promising to be used in more real speech recognition system or product.