The Lexer Iterator Interface Lexer迭代器接口

Introduction 簡介
Wave Lexer synopsis   Wave Lexer 概要
Public Typedefs 公有Typedef
Member functions 成員函數

Introduction 簡介

Every lexer, which should be used in conjunction with the Wave library, has to return tokens formed from the input stream. These tokens should conform to the synopsis described in the topic The Token Type. The lexer type should expose an interface which conforms at least to a forward_iterator (in the sense defined by the the C++ Standard) returning the token type. The code sample below does not show the definition of this forward iterator interface because this is highly implementation defined.
每一個要與 Wave 庫一起使用的 lexer,必須返回從輸入流形成的單詞。這些單詞應當符合在主題 單詞類型 中描述的規定概要。這些 lexer 類型應暴露一個至少符合 forward_iterator (按C++標準中的定義界定)並返回單詞類型的接口。以下代碼示例沒有示範這個前向迭代器接口的定義,因為它非常依賴於實現定義。

Wave lexer synopsis (header: wave/cpplexer/cpp_lexer_interface.hpp) 概要

    struct lex_iterator
{
typedef boost::wave::lex_token<> token_type;

// Every lex_iterator should implement at least a complete
// forward_iterator interface (not shown here)
typedef std::forward_iterator_tag iterator_category;

// additional requirements
lex_iterator();

template <typename Iterator>
lex_iterator(Iterator const &first, Iterator const &last
typename token_type::position_type const &pos,
boost::wave::language_support language)
};

Please note, that the lex_iterator defined in the library header wave/cpplexer/cpp_lexer_interface.hpp actually is a template class taking the token type to use as its template parameter. This is omitted in the synopsis above because it is an implementation detail of the Re2C lexer provided as part of the Wave library.
請注意,在本庫的頭文件 wave/cpplexer/cpp_lexer_interface.hpp 中定義的 lex_iterator 實際上是一個模板類,它以單詞類型作為其模板參數。這一點在以上概要中被忽略,因為它是作為Wave庫的組成部分的Re2C lexer的一個實現細節。

If you want to use Wave in conjunction with your own lexing component this will have to conform to the interface described above only.
如果你想把Wave與你自己的lexing組件一起使用,它只需滿足上述接口即可。

Public Typedefs 公有 Typedef

Besides the typedefs mandated for a forward_iterator by the C++ standard every lexer to be used with the Wave library should define the following typedefs:
除了C++標準中有關 forward_iterator 的typedef之外,每個用於 Wave 庫的 lexer 還要定義以下 typedef:

Public typedef's defined by the boost::wave::context class
boost::wave::context 類定義的公有 typedef
token_type

The token type returned by the lexer. This is type is used as the return value of the main iterators provided by the boost::wave::context object too.

由該lexer返回的 token 類型。該類型也被作為 boost::wave::context 對像所提供的主迭代器的返回值。

Member functions 成員函數

Besides the functions, which should be provided for forward_iterators as mandated by the C++ Standard, every lexer must implement the following functions to be used with the Wave library:
除了C++標準中的 forward_iterators 所提供的函數以外,每個用於 Wave 庫的 lexer 還要實現以下函數:

constructor 構造函數

    lex_iterator();

The default constructor should construct a lexer iterator, which may be used as the end iterator of the provided iterator range.
缺省構造函數應構造一個lexer迭代器,它被用作所提供的迭代器區間的結束迭代器。

    template <typename Iterator>
lex_iterator(Iterator const &first, Iterator const &last,
typename token_type::position_type const &pos,
boost::wave::language_support language);

The second constructor should construct a lexer iterator, which may be used as a iterator traversing over the token sequence, generated by the lexer class.
第二個構造函數應構造一個lexer迭代器,它被用作對由該lexer類生成的單詞序列進行遍歷的迭代器。

The pair of iterators first and last should represent the input stream to be tokenized by the given lexer class.
迭代器對 firstlast 應表示被給定的lexer類進行單詞化的輸入流。

The parameter pos contains the initial position information to be used for token generation.
參數 pos 含有用於單詞生成的初始化位置信息。

The parameter language controls the reuqired mode with which the lexer should be initialised.
參數 language 用於控制lexer被初始化為哪一種請求模式。