Document Token. More...
#include <gtoken.h>
Classes | |
struct | Search |
Token Search Structure. More... | |
Public Member Functions | |
GToken (const R::RString &token=R::RString::Null) | |
int | Compare (const GToken &token) const |
int | Compare (const Search &token) const |
size_t | HashCode (size_t max) const |
R::RString | GetToken (void) const |
tTokenType | GetType (void) const |
GConcept * | GetConcept (void) const |
void | SetConcept (GConcept *concept) |
bool | IsAlpha (void) const |
bool | IsUsed (GConcept *metaconcept) const |
bool | IsUsed (tConceptCat cat) const |
size_t | GetNbOccurs (void) const |
virtual | ~GToken (void) |
Private Attributes | |
R::RString | Token |
tTokenType | Type |
GConcept * | Concept |
R::RContainer< GTokenOccur, false, false > | Occurs |
size_t | Index |
Detailed Description
Document Token.
The GToken class represents a token extracted from a document. It is used when a document is analyzed : the tokenizer parses the document and create the corresponding tokens. These tokens are then treated by the different analyzers in a specific order.
It is only used by GDocAnalyze.
Constructor & Destructor Documentation
|
explicit |
Create a token.
- Parameters
-
token String containing the token.
|
virtual |
Destructor.
Member Function Documentation
int Compare | ( | const GToken & | token | ) | const |
Compare two tokens. They must match the type and the token.
- Parameters
-
token Token to compare with.
- Returns
- a number usable by R::RContainer.
int Compare | ( | const Search & | token | ) | const |
Compare a token with a string.
- Parameters
-
token Search to compare with.
- Returns
- a number usable by R::RContainer.
size_t HashCode | ( | size_t | max | ) | const |
Compute the hash code for a token.
- See also
- R::RHashContainer
- Parameters
-
max Maximal size of the hash code.
R::RString GetToken | ( | void | ) | const |
- Returns
- a R::RString representing the token.
tTokenType GetType | ( | void | ) | const |
- Returns
- the type of the token.
GConcept* GetConcept | ( | void | ) | const |
Get the concept associated with the token (if any).
- Returns
- a pointer to a GConcept (may be null).
void SetConcept | ( | GConcept * | concept | ) |
Associate a concept to the token.
- Parameters
-
concept Concept to assign.
bool IsAlpha | ( | void | ) | const |
Look if the token contains alphabetic characters only.
- Returns
- true if it is the case.
bool IsUsed | ( | GConcept * | metaconcept | ) | const |
Look if the token is associated to a particular meta-concept.
- Parameters
-
metaconcept Meta-concept to search for.
bool IsUsed | ( | tConceptCat | cat | ) | const |
Look if the token is associated to a meta-concept of a particular category.
- Parameters
-
cat Category to search for.
size_t GetNbOccurs | ( | void | ) | const |
Get the number of occurrences of the token.
Member Data Documentation
|
private |
Token.
|
private |
Type of the token.
|
private |
Concept associated with that token.
|
private |
Occurrences of the token.
|
private |
Index of the token in the main container.