Class NumericTokenStream
- java.lang.Object
-
- org.apache.lucene.util.AttributeSource
-
- org.apache.lucene.analysis.TokenStream
-
- org.apache.lucene.analysis.NumericTokenStream
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public final class NumericTokenStream extends TokenStream
Expert: This class provides aTokenStream
for indexing numeric values that can be used byNumericRangeQuery
orNumericRangeFilter
.Note that for simple usage,
NumericField
is recommended.NumericField
disables norms and term freqs, as they are not usually needed during searching. If you need to change these settings, you should use this class.See
NumericField
for capabilities of fields indexed numerically.Here's an example usage, for an
int
field:Field field = new Field(name, new NumericTokenStream(precisionStep).setIntValue(value)); field.setOmitNorms(true); field.setIndexOptions(IndexOptions.DOCS_ONLY); document.add(field);
For optimal performance, re-use the TokenStream and Field instance for more than one document:
NumericTokenStream stream = new NumericTokenStream(precisionStep); Field field = new Field(name, stream); field.setOmitNorms(true); field.setIndexOptions(IndexOptions.DOCS_ONLY); Document document = new Document(); document.add(field); for(all documents) { stream.setIntValue(value) writer.addDocument(document); }
This stream is not intended to be used in analyzers; it's more for iterating the different precisions during indexing a specific numeric value.
NOTE: as token streams are only consumed once the document is added to the index, if you index more than one numeric field, use a separate
NumericTokenStream
instance for each.See
NumericRangeQuery
for more details on theprecisionStep
parameter as well as how numeric fields work under the hood.- Since:
- 2.9
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource
AttributeSource.AttributeFactory, AttributeSource.State
-
-
Field Summary
Fields Modifier and Type Field Description static String
TOKEN_TYPE_FULL_PREC
The full precision token gets this token type assigned.static String
TOKEN_TYPE_LOWER_PREC
The lower precision tokens gets this token type assigned.
-
Constructor Summary
Constructors Constructor Description NumericTokenStream()
Creates a token stream for numeric values using the defaultprecisionStep
NumericUtils.PRECISION_STEP_DEFAULT
(4).NumericTokenStream(int precisionStep)
Creates a token stream for numeric values with the specifiedprecisionStep
.NumericTokenStream(AttributeSource.AttributeFactory factory, int precisionStep)
Expert: Creates a token stream for numeric values with the specifiedprecisionStep
using the givenAttributeSource.AttributeFactory
.NumericTokenStream(AttributeSource source, int precisionStep)
Expert: Creates a token stream for numeric values with the specifiedprecisionStep
using the givenAttributeSource
.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description int
getPrecisionStep()
Returns the precision step.boolean
incrementToken()
Consumers (i.e.,IndexWriter
) use this method to advance the stream to the next token.void
reset()
Resets this stream to the beginning.NumericTokenStream
setDoubleValue(double value)
Initializes the token stream with the supplieddouble
value.NumericTokenStream
setFloatValue(float value)
Initializes the token stream with the suppliedfloat
value.NumericTokenStream
setIntValue(int value)
Initializes the token stream with the suppliedint
value.NumericTokenStream
setLongValue(long value)
Initializes the token stream with the suppliedlong
value.String
toString()
Returns a string representation of the object.-
Methods inherited from class org.apache.lucene.analysis.TokenStream
close, end
-
Methods inherited from class org.apache.lucene.util.AttributeSource
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState
-
-
-
-
Field Detail
-
TOKEN_TYPE_FULL_PREC
public static final String TOKEN_TYPE_FULL_PREC
The full precision token gets this token type assigned.- See Also:
- Constant Field Values
-
TOKEN_TYPE_LOWER_PREC
public static final String TOKEN_TYPE_LOWER_PREC
The lower precision tokens gets this token type assigned.- See Also:
- Constant Field Values
-
-
Constructor Detail
-
NumericTokenStream
public NumericTokenStream()
Creates a token stream for numeric values using the defaultprecisionStep
NumericUtils.PRECISION_STEP_DEFAULT
(4). The stream is not yet initialized, before using set a value using the various set???Value() methods.
-
NumericTokenStream
public NumericTokenStream(int precisionStep)
Creates a token stream for numeric values with the specifiedprecisionStep
. The stream is not yet initialized, before using set a value using the various set???Value() methods.
-
NumericTokenStream
public NumericTokenStream(AttributeSource source, int precisionStep)
Expert: Creates a token stream for numeric values with the specifiedprecisionStep
using the givenAttributeSource
. The stream is not yet initialized, before using set a value using the various set???Value() methods.
-
NumericTokenStream
public NumericTokenStream(AttributeSource.AttributeFactory factory, int precisionStep)
Expert: Creates a token stream for numeric values with the specifiedprecisionStep
using the givenAttributeSource.AttributeFactory
. The stream is not yet initialized, before using set a value using the various set???Value() methods.
-
-
Method Detail
-
setLongValue
public NumericTokenStream setLongValue(long value)
Initializes the token stream with the suppliedlong
value.- Parameters:
value
- the value, for which this TokenStream should enumerate tokens.- Returns:
- this instance, because of this you can use it the following way:
new Field(name, new NumericTokenStream(precisionStep).setLongValue(value))
-
setIntValue
public NumericTokenStream setIntValue(int value)
Initializes the token stream with the suppliedint
value.- Parameters:
value
- the value, for which this TokenStream should enumerate tokens.- Returns:
- this instance, because of this you can use it the following way:
new Field(name, new NumericTokenStream(precisionStep).setIntValue(value))
-
setDoubleValue
public NumericTokenStream setDoubleValue(double value)
Initializes the token stream with the supplieddouble
value.- Parameters:
value
- the value, for which this TokenStream should enumerate tokens.- Returns:
- this instance, because of this you can use it the following way:
new Field(name, new NumericTokenStream(precisionStep).setDoubleValue(value))
-
setFloatValue
public NumericTokenStream setFloatValue(float value)
Initializes the token stream with the suppliedfloat
value.- Parameters:
value
- the value, for which this TokenStream should enumerate tokens.- Returns:
- this instance, because of this you can use it the following way:
new Field(name, new NumericTokenStream(precisionStep).setFloatValue(value))
-
reset
public void reset()
Description copied from class:TokenStream
Resets this stream to the beginning. This is an optional operation, so subclasses may or may not implement this method.TokenStream.reset()
is not needed for the standard indexing process. However, if the tokens of aTokenStream
are intended to be consumed more than once, it is necessary to implementTokenStream.reset()
. Note that if your TokenStream caches tokens and feeds them back again after a reset, it is imperative that you clone the tokens when you store them away (on the first pass) as well as when you return them (on future passes afterTokenStream.reset()
).- Overrides:
reset
in classTokenStream
-
incrementToken
public boolean incrementToken()
Description copied from class:TokenStream
Consumers (i.e.,IndexWriter
) use this method to advance the stream to the next token. Implementing classes must implement this method and update the appropriateAttributeImpl
s with the attributes of the next token.The producer must make no assumptions about the attributes after the method has been returned: the caller may arbitrarily change it. If the producer needs to preserve the state for subsequent calls, it can use
AttributeSource.captureState()
to create a copy of the current attribute state.This method is called for every token of a document, so an efficient implementation is crucial for good performance. To avoid calls to
AttributeSource.addAttribute(Class)
andAttributeSource.getAttribute(Class)
, references to allAttributeImpl
s that this stream uses should be retrieved during instantiation.To ensure that filters and consumers know which attributes are available, the attributes must be added during instantiation. Filters and consumers are not required to check for availability of attributes in
TokenStream.incrementToken()
.- Specified by:
incrementToken
in classTokenStream
- Returns:
- false for end of stream; true otherwise
-
toString
public String toString()
Description copied from class:AttributeSource
Returns a string representation of the object. In general, thetoString
method returns a string that "textually represents" this object.WARNING: For backwards compatibility this method is implemented as in Lucene 2.9/3.0. In Lucene 4.0 this default implementation will be removed.
It is recommeneded to use
AttributeSource.reflectAsString(boolean)
orAttributeSource.reflectWith(org.apache.lucene.util.AttributeReflector)
to get a well-defined output of AttributeSource's internals.- Overrides:
toString
in classAttributeSource
-
getPrecisionStep
public int getPrecisionStep()
Returns the precision step.
-
-