The use of Computational Linguistics (CL) to process Natural Languages (NL) is an important domain of Natural Language Processing (NLP). Our experimental results show that, Kneser-Ney outperforms Witten-Bell as a smoothing technique when used with n-gram LMs for checking grammatical correctness of Bangla sentences. We also provided an improved technique for calculating the optimum threshold which further enhanced the the results. In this work, we proposed an improved method using Kneser-Ney smoothing based n-gram language model for grammar checking and performed a comparative performance analysis between Kneser-Ney and Witten-Bell smoothing techniques for the same purpose. In our previous work, we presented a Witten-Bell smoothing based language modelling technique for checking grammatical correctness of Bangla sentences which showed promising results outperforming previous methods. Kneser-Ney is one of the most prominently used and successful smoothing technique for language modelling. Smoothing is one of the most effective techniques used in building a language model to deal with data sparsity problem. N-gram based language models are very popular and extensively used statistical methods for solving various natural language processing problems including grammar checking.
0 Comments
Leave a Reply. |