org.languagetool.language.identifier

Class DefaultLanguageIdentifier

java.lang.Object

org.languagetool.language.identifier.LanguageIdentifier

org.languagetool.language.identifier.DefaultLanguageIdentifier

public class DefaultLanguageIdentifier
extends LanguageIdentifier

Identify the language of a text. Note that some languages might never be detected because they are close to another language. Language variants like en-US or en-GB are not detected, the result will be en for those. By default, only the first 1000 characters of a text are considered. Email signatures that use \n-- \n as a delimiter are ignored.

Since:

2.9

Nested Class Summary

Nested classes/interfaces inherited from class org.languagetool.language.identifier.LanguageIdentifier

LanguageIdentifier.ParsedLanguageLists

Field Summary

Fields inherited from class org.languagetool.language.identifier.LanguageIdentifier

COMMON_WORDS_LANG_IDENTIFIER, maxLength, NON_LATIN_CHARS_LANGUAGES, REMOVE_EMAIL_SIGNATURE_FILTER, REMOVE_MENTION_FILTER, REMOVE_NON_BREAKING_SPACES_FILTER, REMOVE_URL_FILTER, SCORE_THRESHOLD, UNICODE_BASED_LANG_IDENTIFIER

Method Summary

Modifier and Type	Method and Description
Language	detectLanguage(String cleanText)
DetectedLanguage	detectLanguage(String cleanText, List<String> noopLangsTmp, List<String> preferredLangsTmp)
DetectedLanguage	detectLanguage(String cleanText, List<String> noopLangsTmp, List<String> preferredLangsTmp, boolean limitOnPreferredLangs)
AtomicInteger	getFasttextInitCounter() For test only
boolean	isFastTextEnabled()
void	setFastTextDetector(FastTextDetector fastTextDetector) For test only

Methods inherited from class org.languagetool.language.identifier.LanguageIdentifier

cleanAndShortenText, getHighestScoringResult, prepareDetectLanguage

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

setFastTextDetector

@TestOnly
public void setFastTextDetector(FastTextDetector fastTextDetector)

For test only

getFasttextInitCounter

@TestOnly
public AtomicInteger getFasttextInitCounter()

For test only

Returns:

a counter how often fasttext was already recreated after a failure

isFastTextEnabled

public boolean isFastTextEnabled()

Since:

5.2

detectLanguage

@Nullable
public Language detectLanguage(String cleanText)

Specified by:

detectLanguage in class LanguageIdentifier

Parameters:

cleanText - a cleanText as returned by LanguageIdentifier.cleanAndShortenText(String)

Returns:

language or null if language could not be identified

detectLanguage

public DetectedLanguage detectLanguage(String cleanText,
                                       List<String> noopLangsTmp,
                                       List<String> preferredLangsTmp)

Specified by:

detectLanguage in class LanguageIdentifier

Parameters:

cleanText - a cleanText as returned by LanguageIdentifier.cleanAndShortenText(String)

noopLangsTmp - list of codes that are detected but will lead to the NoopLanguage that has no rules

Returns:

language or null if language could not be identified

Since:

4.4 (new parameter noopLangs, changed return type to DetectedLanguage)

detectLanguage

@Nullable
public DetectedLanguage detectLanguage(String cleanText,
                                                 List<String> noopLangsTmp,
                                                 List<String> preferredLangsTmp,
                                                 boolean limitOnPreferredLangs)

Specified by:

detectLanguage in class LanguageIdentifier