Class DelimitedPayloadTokenFilter

  • All Implemented Interfaces:
    Closeable, AutoCloseable

    public final class DelimitedPayloadTokenFilter
    extends org.apache.lucene.analysis.TokenFilter
    Characters before the delimiter are the "token", those after are the payload.

    For example, if the delimiter is '|', then for the string "foo|bar", foo is the token and "bar" is a payload.

    Note, you can also include a PayloadEncoder to convert the payload in an appropriate way (from characters to bytes).

    Note make sure your Tokenizer doesn't split on the delimiter, or this won't work

    See Also:
    PayloadEncoder
    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

        org.apache.lucene.util.AttributeSource.AttributeFactory, org.apache.lucene.util.AttributeSource.State
    • Field Summary

      Fields 
      Modifier and Type Field Description
      static char DEFAULT_DELIMITER  
      • Fields inherited from class org.apache.lucene.analysis.TokenFilter

        input
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      boolean incrementToken()  
      • Methods inherited from class org.apache.lucene.analysis.TokenFilter

        close, end, reset
      • Methods inherited from class org.apache.lucene.util.AttributeSource

        addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toString
    • Constructor Detail

      • DelimitedPayloadTokenFilter

        public DelimitedPayloadTokenFilter​(org.apache.lucene.analysis.TokenStream input,
                                           char delimiter,
                                           PayloadEncoder encoder)
    • Method Detail

      • incrementToken

        public boolean incrementToken()
                               throws IOException
        Specified by:
        incrementToken in class org.apache.lucene.analysis.TokenStream
        Throws:
        IOException