Skip to content

Trainable Biaffine Dependency Parser[source]

The eds.biaffine_dep_parser component is a trainable dependency parser based on a biaffine model (Dozat and Manning, 2017). For each token, the model predicts a score for each possible head in the document, and a score for each possible label for each head. The results are then decoded either greedily by picking the best scoring head for each token independently, or holistically by computing the Maximum Spanning Tree (MST) over the graph of token → head scores.

Experimental

This component is experimental. In particular, it expects the input to be sentences and not full documents, as it has not been optimized for memory efficiency yet and computed the full matrix of scores for all pairs of tokens in a document.

At the moment, it is mostly used for benchmarking and research purposes.

Examples

import edsnlp, edsnlp.pipes as eds

nlp = edsnlp.blank("eds")
nlp.add_pipe(
    eds.biaffine_dep_parser(
        embedding=eds.transformer(model="prajjwal1/bert-tiny"),
        hidden_size=128,
        dropout_p=0.1,
        # labels unset, will be inferred from the data in `post_init`
        decoding_mode="mst",
    ),
    name="dep_parser"
)

Dependency parsers are typically trained on CoNLL-formatted Universal Dependencies corpora, which you can load using the edsnlp.data.read_conll function.

To train the model, refer to the Training tutorial.

Parameters

PARAMETER DESCRIPTION
nlp

The pipeline object
name

Name of the component
embedding

The word embedding component
context_getter

What context to use when computing the span embeddings (defaults to the whole document). For example {"section": "conclusion"} to predict dependencies in the conclusion section of documents.

DEFAULT: None

use_attrs

The attributes to use as features for the model (ex. ["pos_"] to use the POS tag). By default, no attributes are used.

Note that if you train a model with attributes, you will need to provide the same attributes during inference, and the model might not work well if the attributes were not annotated accurately on the test data.

DEFAULT: None

attr_size

The size of the attribute embeddings.

DEFAULT: 32

hidden_size

The size of the hidden layer in the MLP.

DEFAULT: 128

dropout_p

The dropout probability to use in the MLP.

DEFAULT: 0.0

labels

The labels to predict. The labels can also be inferred from the data during nlp.post_init(...).

DEFAULT: ['root']

decoding_mode

Whether to decode the dependencies greedily or using the Maximum Spanning Tree algorithm.

DEFAULT: mst

Authors and citation

The eds.biaffine_dep_parser trainable pipe was developed by AP-HP's Data Science team, and heavily inspired by the implementation of Grobol and Crabbé, 2021. The biaffine architecture is based on the biaffine parser of Dozat and Manning, 2017.

  1. Dozat T. and Manning C.D., 2017. Deep Biaffine Attention for Neural Dependency Parsing. https://arxiv.org/abs/1611.01734

  2. Grobol L. and Crabbé B., 2021. Analyse en dépendances du français avec des plongements contextualisés. https://hal.archives-ouvertes.fr/hal-03223424