org.languagetool.tokenizers

Class PersianWordTokenizer

java.lang.Object

org.languagetool.tokenizers.WordTokenizer

org.languagetool.tokenizers.PersianWordTokenizer

All Implemented Interfaces:

Tokenizer

public class PersianWordTokenizer
extends WordTokenizer

Since:

2.7

Constructor Summary

Constructors

Constructor and Description
PersianWordTokenizer()

Method Summary

Modifier and Type	Method and Description
String	getTokenizingCharacters()

Methods inherited from class org.languagetool.tokenizers.WordTokenizer

getProtocols, isEMail, isUrl, joinEMails, joinEMailsAndUrls, joinUrls, tokenize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PersianWordTokenizer

public PersianWordTokenizer()

Method Detail

getTokenizingCharacters

public String getTokenizingCharacters()

Overrides:

getTokenizingCharacters in class WordTokenizer

Returns:

The string containing the characters used by the tokenizer to tokenize words.