Package org.apache.lucene.analysis.ngram
Class NGramTokenFilter
- java.lang.Object
-
- org.apache.lucene.util.AttributeSource
-
- org.apache.lucene.analysis.TokenStream
-
- org.apache.lucene.analysis.TokenFilter
-
- org.apache.lucene.analysis.ngram.NGramTokenFilter
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public final class NGramTokenFilter extends TokenFilter
Tokenizes the input into n-grams of the given size(s).
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.AttributeFactory, AttributeSource.State
-
-
Field Summary
Fields Modifier and Type Field Description static int
DEFAULT_MAX_NGRAM_SIZE
static int
DEFAULT_MIN_NGRAM_SIZE
-
Fields inherited from class org.apache.lucene.analysis.TokenFilter
input
-
-
Constructor Summary
Constructors Constructor Description NGramTokenFilter(TokenStream input)
Creates NGramTokenFilter with default min and max n-grams.NGramTokenFilter(TokenStream input, int minGram, int maxGram)
Creates NGramTokenFilter with given min and max n-grams.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description boolean
incrementToken()
Returns the next token in the stream, or null at EOS.void
reset()
Reset the filter as well as the input TokenStream.-
Methods inherited from class org.apache.lucene.analysis.TokenFilter
close, end
-
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
-
-
-
-
Field Detail
-
DEFAULT_MIN_NGRAM_SIZE
public static final int DEFAULT_MIN_NGRAM_SIZE
- See Also:
- Constant Field Values
-
DEFAULT_MAX_NGRAM_SIZE
public static final int DEFAULT_MAX_NGRAM_SIZE
- See Also:
- Constant Field Values
-
-
Constructor Detail
-
NGramTokenFilter
public NGramTokenFilter(TokenStream input, int minGram, int maxGram)
Creates NGramTokenFilter with given min and max n-grams.- Parameters:
input
-TokenStream
holding the input to be tokenizedminGram
- the smallest n-gram to generatemaxGram
- the largest n-gram to generate
-
NGramTokenFilter
public NGramTokenFilter(TokenStream input)
Creates NGramTokenFilter with default min and max n-grams.- Parameters:
input
-TokenStream
holding the input to be tokenized
-
-
Method Detail
-
incrementToken
public final boolean incrementToken() throws IOException
Returns the next token in the stream, or null at EOS.- Specified by:
incrementToken
in classTokenStream
- Returns:
- false for end of stream; true otherwise
- Throws:
IOException
-
reset
public void reset() throws IOException
Description copied from class:TokenFilter
Reset the filter as well as the input TokenStream.- Overrides:
reset
in classTokenFilter
- Throws:
IOException
-
-