Vocabulary
Vocabulary layer.
This is not meant to be used as a torch.nn.Module but subclassing
torch.nn.Module makes the instances appear when printing a model, which is nice.
Parameters
| PARAMETER | DESCRIPTION |
|---|---|
items |
Initial vocabulary elements if any. Specific elements such as padding and unk can be set here to enforce their index in the vocabulary.
TYPE:
|
default |
Default index to use for out of vocabulary elements Defaults to -100
TYPE:
|
Functions
initialization
Enters the initialization mode. Out of vocabulary elements will be assigned an index.
encode
Converts an element into its vocabulary index
If the layer is in its initialization mode (with vocab.initialization(): ...),
and the element is out of vocabulary, a new index will be created and returned.
Otherwise, any oov element will be encoded with the default index.
| PARAMETER | DESCRIPTION |
|---|---|
item |
|
| RETURNS | DESCRIPTION |
|---|---|
int
|
|
decode
Converts an index into its original value
| PARAMETER | DESCRIPTION |
|---|---|
idx |
|
| RETURNS | DESCRIPTION |
|---|---|
InputT
|
|