|
|||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--net.percederberg.grammatica.parser.Tokenizer
A character stream tokenizer. This class groups the characters read from the stream together into tokens ("words"). The grouping is controlled by token patterns that contain either a fixed string to search for, or a regular expression. If the stream of characters don't match any of the token patterns, a parse exception is thrown.
Constructor Summary | |
Tokenizer(java.io.Reader in)
Creates a new tokenizer for the specified input stream. |
Method Summary | |
void |
addPattern(TokenPattern pattern)
Adds a new token pattern to the tokenizer. |
int |
getCurrentColumn()
Returns the current column number. |
int |
getCurrentLine()
Returns the current line number. |
java.lang.String |
getPatternDescription(int id)
Returns a description of the token pattern with the specified id. |
Token |
next()
Finds the next token on the stream. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
public Tokenizer(java.io.Reader in)
in
- the input stream to readMethod Detail |
public java.lang.String getPatternDescription(int id)
id
- the token pattern id
public int getCurrentLine()
public int getCurrentColumn()
public void addPattern(TokenPattern pattern) throws ParserCreationException
pattern
- the pattern to add
ParserCreationException
- if the pattern couldn't be
added to the tokenizerpublic Token next() throws ParseException
ParseException
- if the input stream couldn't be read or
parsed correctly
|
|||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |