SinusoidalEmbedding
A position embedding lookup table that stores embeddings for a fixed number
of positions.
The value of each of the embedding_dim channels of the generated embedding
is generated according to a trigonometric function (sin for even channels,
cos for odd channels).
The frequency of the signal in each pair of channels varies according to the
temperature parameter.
Any input position above the maximum value num_embeddings will be capped to
num_embeddings - 1
Parameters
| PARAMETER | DESCRIPTION |
|---|---|
num_embeddings |
The maximum number of position embeddings store in this table
TYPE:
|
embedding_dim |
The embedding size
TYPE:
|
temperature |
The temperature controls the range of frequencies used by each channel of the embedding
TYPE:
|
forward
Forward pass of the SinusoidalEmbedding module
| PARAMETER | DESCRIPTION |
|---|---|
indices |
Shape: any
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
FloatTensor
|
Shape: |