This study reports the development of a Myanmar word segmentation method using Unicode standard encoding. Word segmentation is an essential step prior to natural language processing in the Myanmar language, because a Myanmar text is a string of characters without explicit word boundary delimiters. The proposed method has two phases: syllable segmentation and syllable merging. A rule-based heuristic approach was adopted for syllable segmentation, and a dictionary-based statistical approach for syllable merging. Evaluation of test results showed that the method is very effective for the Myanmar language.
G.G. Chowdhury , Introduction to Modern Information Retrieval (Library Association, London, 1999).
3.
Myanmar NLP, Myanmar Unicode Reference Documents and Research Papers (2006). Available at: http://www.myanmars.net/unicode/doc/index.htm (accessed 1 January 2007).
4.
Unicode Consortium , The Unicode Standard 4.0: Southeast Asian Scripts (Addison Wesley, California, 2004).
5.
A. Wirot, Collocation and Thai Word Segmentation, Proceedings of Joint International Conference of SNLP-Oriental COCOSDA 2002 ( Thammasat University, Bangkok, Thailand, 2002).
6.
T. Theeramunkong and S. Usanavasin, Non-dictionary-based Thai word segmentation using decision trees, Human Language TechnologyConference: Proceedings of the first international conference on Human language technology research (Association for Computer Linguistics , NJ, 2001).
7.
A. Kawtrakul , C. Thumkanon, Y. Poovorawan, P. Varasrai and M. Suktarachan, Automatic Thai unknown word recognition , Proceedings of the nature language processing pacific rim symposium (Phuket, Thailand, 1997).
8.
V. Sornlertlamvanich , T. Potipiti and T. Charoenporn, Automatic corpus-based Thai word extraction with the c4.5 learning algorithm, Proceedings of the 18th Conference on Computational linguistics (Association for Computer Linguistics, NJ, 2000 ).
9.
J.R. Quinlan , C4.5 Programs for machine learning ( Morgan Kaufmann, California, 1993 ).
10.
S.F. Chen and J. Goodman, An empirical study of smoothing techniques for language modelling, Proceedings of the 34th Annual Meeting on Association for Computer Linguistics (Association for Computer Linguistics, NJ, 1996).
11.
C.D. Manning and H. Schutze, Foundations of Statistical Natural Language Processing (MIT Press, London, 2002).
12.
L.A. Ha, A method for word segmentation in Vietnamese, Proceedings of Corpus Linguistic 2003, (University of Lancaster , Lancaster, 2003).
13.
D. Dinh, Building a training corpus for word sense disambiguation in English-to-Vietnamese machine translation, Proceedings of the 19th International Conference on Computational Linguistics (Association for Computer Linguistics, NJ, 2002).
14.
D. Dinh, H. Kiem and N.V. Toan, Vietnamese word segmentation, Proceedings of Neural Networks and Natural Language Processing (Tokyo, Japan, 2001).
15.
T.V. Nguyen, H.K. Tran, T.T.T. Nguyen and H. Nguyen, Word segmentation for Vietnamese text categorization: an online corpus approach , Proceedings of 4th IEEE International Conference on Computer Science Research, Innovation and Vision of the Future, (HoChiMinh City, Vietnam, 2006).
16.
Z. Michalewicz , Genetic Algorithms + Data Structure = Evolution Programs ( Springer-Verlag, London, 1996).
17.
Myanmar Sar A Phwae, Myanmar Spelling Bible, Myanmar Thut Pon Kyan (Myanmar Sar A Phwae, Yangon , 2003).
18.
Z. Htut, Myanmar-Thai Co-workshop on Myanmar Language Implementation, Input Methods and Basic Encoding in Myanmar Language. Available at: http://www.myanmars.net/unicode/doc (accessed 1 January 2007).
19.
K.W. Church and P. Hanks, Word association norms, mutual information, and lexicography, Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics (Association for Computer Linguistics , Vancouver, Canada, 1989).
20.
C.J. Van Rijsbergen, The Geometry of Information Retrieval (Cambridge University Press, London , 2004).
