org.languagetool.language.identifier

Class LanguageIdentifier

java.lang.Object

org.languagetool.language.identifier.LanguageIdentifier

Direct Known Subclasses:

DefaultLanguageIdentifier, SimpleLanguageIdentifier

public abstract class LanguageIdentifier
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	LanguageIdentifier.ParsedLanguageLists

Field Summary

Fields

Modifier and Type	Field and Description
protected static CommonWordsDetector	COMMON_WORDS_LANG_IDENTIFIER
protected static int	CONSIDER_ONLY_PREFERRED_THRESHOLD
protected int	maxLength
protected static List<String>	NON_LATIN_CHARS_LANGUAGES
protected static com.optimaize.langdetect.text.TextFilter	REMOVE_EMAIL_SIGNATURE_FILTER
protected static com.optimaize.langdetect.text.TextFilter	REMOVE_MENTION_FILTER
protected static com.optimaize.langdetect.text.TextFilter	REMOVE_NON_BREAKING_SPACES_FILTER
protected static com.optimaize.langdetect.text.TextFilter	REMOVE_URL_FILTER
protected static float	SCORE_THRESHOLD
protected static UnicodeBasedDetector	UNICODE_BASED_LANG_IDENTIFIER

Constructor Summary

Constructors

Constructor and Description
LanguageIdentifier(int maxLength)

Method Summary

Modifier and Type	Method and Description
String	cleanAndShortenText(String text)
abstract Language	detectLanguage(String cleanText)
abstract DetectedLanguage	detectLanguage(String cleanText, List<String> noopLangsTmp, List<String> preferredLangsTmp)
abstract DetectedLanguage	detectLanguage(String cleanText, List<String> noopLangsTmp, List<String> preferredLangsTmp, boolean limitOnPreferredLangs)
protected Map.Entry<String,Double>	getHighestScoringResult(Map<String,Double> probs)
protected LanguageIdentifier.ParsedLanguageLists	prepareDetectLanguage(String text, List<String> noopLangsTmp, List<String> preferredLangsTmp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SCORE_THRESHOLD

protected static final float SCORE_THRESHOLD

See Also:

Constant Field Values

CONSIDER_ONLY_PREFERRED_THRESHOLD

protected static final int CONSIDER_ONLY_PREFERRED_THRESHOLD

See Also:

Constant Field Values

NON_LATIN_CHARS_LANGUAGES

protected static final List<String> NON_LATIN_CHARS_LANGUAGES

REMOVE_EMAIL_SIGNATURE_FILTER

protected static final com.optimaize.langdetect.text.TextFilter REMOVE_EMAIL_SIGNATURE_FILTER

REMOVE_MENTION_FILTER

protected static final com.optimaize.langdetect.text.TextFilter REMOVE_MENTION_FILTER

REMOVE_NON_BREAKING_SPACES_FILTER

protected static final com.optimaize.langdetect.text.TextFilter REMOVE_NON_BREAKING_SPACES_FILTER

REMOVE_URL_FILTER

protected static final com.optimaize.langdetect.text.TextFilter REMOVE_URL_FILTER

UNICODE_BASED_LANG_IDENTIFIER

protected static final UnicodeBasedDetector UNICODE_BASED_LANG_IDENTIFIER

COMMON_WORDS_LANG_IDENTIFIER

protected static final CommonWordsDetector COMMON_WORDS_LANG_IDENTIFIER

maxLength

protected int maxLength

Constructor Detail

LanguageIdentifier

public LanguageIdentifier(int maxLength)

Method Detail

detectLanguage

@Nullable
public abstract DetectedLanguage detectLanguage(String cleanText,
                                                          List<String> noopLangsTmp,
                                                          List<String> preferredLangsTmp)

Parameters:

cleanText - a cleanText as returned by cleanAndShortenText(String)

noopLangsTmp - list of codes that are detected but will lead to the NoopLanguage that has no rules

Returns:

language or null if language could not be identified

Since:

4.4 (new parameter noopLangs, changed return type to DetectedLanguage)

detectLanguage

@Nullable
public abstract DetectedLanguage detectLanguage(String cleanText,
                                                          List<String> noopLangsTmp,
                                                          List<String> preferredLangsTmp,
                                                          boolean limitOnPreferredLangs)

detectLanguage

@Nullable
public abstract Language detectLanguage(String cleanText)

Parameters:

cleanText - a cleanText as returned by cleanAndShortenText(String)

Returns:

language or null if language could not be identified

Since:

4.4 (new parameter noopLangs, changed return type to DetectedLanguage)

cleanAndShortenText

public String cleanAndShortenText(String text)

Since:

5.8

prepareDetectLanguage

protected LanguageIdentifier.ParsedLanguageLists prepareDetectLanguage(String text,
                                                                       List<String> noopLangsTmp,
                                                                       List<String> preferredLangsTmp)

getHighestScoringResult

protected Map.Entry<String,Double> getHighestScoringResult(Map<String,Double> probs)