Lucene++ - a full-featured, c++ search engine
API Documentation


 All Data Structures Namespaces Files Functions Variables Typedefs Enumerations Enumerator Macros Pages
LowerCaseTokenizer.h
Go to the documentation of this file.
1 // Copyright (c) 2009-2014 Alan Wright. All rights reserved.
3 // Distributable under the terms of either the Apache License (Version 2.0)
4 // or the GNU Lesser General Public License.
6 
7 #ifndef LOWERCASETOKENIZER_H
8 #define LOWERCASETOKENIZER_H
9 
10 #include "LetterTokenizer.h"
11 
12 namespace Lucene {
13 
21 class LPPAPI LowerCaseTokenizer : public LetterTokenizer {
22 public:
24  LowerCaseTokenizer(const ReaderPtr& input);
25 
27  LowerCaseTokenizer(const AttributeSourcePtr& source, const ReaderPtr& input);
28 
30  LowerCaseTokenizer(const AttributeFactoryPtr& factory, const ReaderPtr& input);
31 
32  virtual ~LowerCaseTokenizer();
33 
35 
36 public:
38  virtual wchar_t normalize(wchar_t c);
39 };
40 
41 }
42 
43 #endif
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.
Definition: LowerCaseTokenizer.h:21
boost::shared_ptr< Reader > ReaderPtr
Definition: LuceneTypes.h:547
boost::shared_ptr< AttributeFactory > AttributeFactoryPtr
Definition: LuceneTypes.h:519
boost::shared_ptr< AttributeSource > AttributeSourcePtr
Definition: LuceneTypes.h:520
A LetterTokenizer is a tokenizer that divides text at non-letters. That's to say, it defines tokens a...
Definition: LetterTokenizer.h:19
#define LUCENE_CLASS(Name)
Definition: LuceneObject.h:24
Definition: AbstractAllTermDocs.h:12

clucene.sourceforge.net