Doc Pooler[source]

The eds.doc_pooler component aggregates the word embeddings of a document into a single embedding per document, to be consumed by a document-level component such as eds.doc_classifier. When the document is split into several contexts, they are pooled together into a single embedding.

Examples

import edsnlp, edsnlp.pipes as eds

embedding = eds.doc_pooler(
    pooling_mode="attention",
    embedding=eds.transformer(
        model="hf-internal-testing/tiny-random-bert",
        window=128,
        stride=96,
    ),
)

Parameters

PARAMETER	DESCRIPTION
`nlp`	The pipeline object
`name`	Name of the component
`embedding`	The word embedding component TYPE: `WordEmbeddingComponent`
`pooling_mode`	How the embeddings are aggregated into a single embedding per document: `"mean"`, `"max"`, `"sum"`: element-wise reduction over the embeddings of the document `"attention"`: weighted sum of the embeddings of the document, the weights being a softmax of scores given by a learned linear layer DEFAULT: `mean`
`use_cls`	Pool the embeddings of the first wordpiece of each transformer window (the `[CLS]` token) instead of the word embeddings. A document longer than the transformer `window` is split into several windows, whose `[CLS]` embeddings are aggregated with `pooling_mode`. This is mostly meant for short texts, and requires an embedding that returns a `cls` output, such as `eds.transformer`. DEFAULT: `False`