org.languagetool.tokenizers

Class SRXSentenceTokenizer

java.lang.Object

org.languagetool.tokenizers.SRXSentenceTokenizer

All Implemented Interfaces:

SentenceTokenizer, Tokenizer

Direct Known Subclasses:

SimpleSentenceTokenizer

public class SRXSentenceTokenizer
extends Object
implements SentenceTokenizer

Class to tokenize sentences using rules from an SRX file.

Author:

Marcin Miłkowski, Jarek Lipski

Constructor Summary

Constructors

Constructor and Description
SRXSentenceTokenizer(Language language) Build a sentence tokenizer based on the rules in the segment.srx file that comes with LanguageTool.
SRXSentenceTokenizer(Language language, String srxInClassPath)

Method Summary

Modifier and Type	Method and Description
void	setSingleLineBreaksMarksParagraph(boolean lineBreakParagraphs)
boolean	singleLineBreaksMarksPara()
List<String>	tokenize(String text) Tokenize the given string to sentences.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SRXSentenceTokenizer

public SRXSentenceTokenizer(Language language)

Build a sentence tokenizer based on the rules in the segment.srx file that comes with LanguageTool.

SRXSentenceTokenizer

public SRXSentenceTokenizer(Language language,
                            String srxInClassPath)

Parameters:

srxInClassPath - the path to an SRX file in the classpath

Since:

3.2

Method Detail

tokenize

public final List<String> tokenize(String text)

Description copied from interface: SentenceTokenizer

Tokenize the given string to sentences.

Specified by:

tokenize in interface SentenceTokenizer

Specified by:

tokenize in interface Tokenizer

singleLineBreaksMarksPara

public final boolean singleLineBreaksMarksPara()

Specified by:

singleLineBreaksMarksPara in interface SentenceTokenizer

setSingleLineBreaksMarksParagraph

public final void setSingleLineBreaksMarksParagraph(boolean lineBreakParagraphs)

Specified by:

setSingleLineBreaksMarksParagraph in interface SentenceTokenizer

Parameters:

lineBreakParagraphs - if true, single lines breaks are assumed to end a paragraph; if false, only two ore more consecutive line breaks end a paragraph