The Building Educational Applications (BEA) workshop at NAACL 2013 hosted the inaugural NLI shared task. However, it seems that character n-grams are the single best feature for the task. Surface level lexical features such as character, word and lemma n-grams have also been found to be quite useful for this task. These include syntactic features such as constituent parses, grammatical dependencies and part-of-speech tags. Various linguistic feature types have been applied for this task. Ī range of ensemble based systems have also been applied to the task and shown to improve performance over single classifier systems. Next, machine learning is applied to train classifiers, like support vector machines, for predicting the L1 of unseen texts. This is done using language learner data, usually from a learner corpus. Natural language processing methods are used to extract and identify language usage patterns common to speakers of an L1-group. This has already attracted interest and funding from intelligence agencies. an anonymous letter, is the key piece of evidence in an investigation and clues about the native language of a writer can help investigators in identifying the source. This is particularly useful in situations where a text, e.g. NLI methods can also be applied in forensic linguistics as a method of performing authorship profiling in order to infer the attributes of an author, including their linguistic background. This is useful for developing pedagogical material, teaching methods, L1-specific instructions and generating learner feedback that is tailored to their native language. This identification of L1-specific features has been used to study language transfer effects in second-language acquisition. This can be compared to a baseline of 9% for choosing randomly.Īpplications Pedagogy and language transfer Using large-scale English data, NLI methods achieve over 80% accuracy in predicting the native language of texts written by authors from 11 different L1 backgrounds. This relates to cross-linguistic influence (CLI), a key topic in the field of second-language acquisition (SLA) that analyzes transfer effects from the L1 on later learned languages.
The experimental results show that the proposed method has an accuracy of over 90% for small texts and over 99.8% for large texts.NLI works under the assumption that an author's L1 will dispose them towards particular language production patterns in their L2, as influenced by their native language.
Both corpora consists of UTF-8 encoded text, so the diacritics could be taken into account, in the case that the text has no diacritics only the stop words are used to determine the language of the text. We have tested our method using a Twitter corpus and a news article corpus. The languages taken into account were romance languages because they are very similar and usually it is hard to distinguish between them from a computational point of view. This method was chosen because stop words and diacritics are very specific to a language, although some languages have some similar words and special characters they are not all common. We propose different approaches that combine the two dictionaries to accurately determine the language of textual corpora. In this paper we present a statistical method for automatic language identification of written text using dictionaries containing stop words and diacritics.
Automatic language identification is a natural language processing problem that tries to determine the natural language of a given content.
0 kommentar(er)
