Trainable NER

The eds.ner_crf component is a general purpose trainable named entity recognizer. It can extract:

• flat entities
• overlapping entities of different labels

However, at the moment, the model cannot currently extract entities that are nested inside larger entities of the same label.

It is based on a CRF (Conditional Random Field) layer and should therefore work well on dataset composed of entities will ill-defined boundaries. We offer a compromise between speed and performance by allowing the user to specify a window size for the CRF layer. The smaller the window, the faster the model will be, but at the cost of degraded performance.

The pipeline assigns both doc.ents (in which overlapping entities are filtered out) and doc.spans. These destinations can be inferred from the target_span_getter parameter, combined with the post_init step.

Architecture

The model performs token classification using the BIOUL (Begin, Inside, Outside, Unary, Last) tagging scheme. To extract overlapping entities, each label has its own tag sequence, so the model predicts n_labels sequences of O, I, B, L, U tags. The architecture is displayed in the figure below.

To enforce the tagging scheme, (ex: I cannot follow O but only B, ...), we use a stack of CRF (Conditional Random Fields) layers, one per label during both training and prediction.

Examples

Let us define a pipeline composed of a transformer, and a NER component.

from pathlib import Path

import edsnlp, edsnlp.pipes as eds

nlp = edsnlp.blank("eds")
nlp.add_pipe(
    eds.ner_crf(
        embedding=eds.transformer(
            model="prajjwal1/bert-tiny",
            window=128,
            stride=96,
        ),
        mode="joint",
        target_span_getter=["ents", "ner-preds"],
        window=10,
    ),
    name="ner"
)

To train the model, refer to the Training tutorial.

Parameters

Authors and citation

The eds.ner_crf pipeline was developed by AP-HP's Data Science team.

The deep learning model was adapted from Wajsbürt, 2021.