TNM[source]

The eds.tnm component extracts TNM staging mentions from clinical documents and decomposes them into structured attributes.

Extraction logic

Matching happens in two stages: a regex whose leading lookahead (logic_filter) decides whether a candidate is a TNM mention, then a post-filter that drops known lookalike abbreviations.

A span is extracted when at least one of the following conditions holds:

• T + N/M/R present: the T component is followed by at least one of N (node), M (metastasis), or R (resection), with any delimiter (space, comma, slash, newline) between them.
• Standalone qualified T: the T component carries both a prefix (e.g. p, c, yp) and a specification (e.g. a, b, mi), even without an N/M/R component.

The pattern is case-insensitive (pT2N1M0, pt2n1m0 and PT2N1M0 are all matched). Delimiters between components can be spaces, commas, slashes, or newlines (e.g. pT2 / N1 / M0, pT2,N1,M0).

What is matched

What is deliberately not matched

The banned_words post-filter also drops any match of two characters or less that does not start with a lowercase letter, which removes the many T/PT fragments produced by uppercase headings and tables.

Decomposition

Each matched span is parsed into a TNM Pydantic model stored on span._.tnm. The following fields are extracted:

Each component carries its own prefix: in pT1 cN1 M0 the tumour is pathological while the node is clinical, and both are kept.

Normalised form

span._.tnm.norm() returns a compact canonical string that concatenates all non-None components, stripping delimiters and surrounding whitespace:

{tumour_prefix}T{tumour}{tumour_specification}{tumour_suffix}
{node_prefix}N{node}{node_specification}{node_suffix}
{metastasis_prefix}M{metastasis}{metastasis_specification}{metastasis_suffix}
PL{pleura}
{resection_prefix}R{resection}{resection_specification}{resection_loc}{resection_suffix}
({VERSION} {version_year})

This value is also stored on span.kb_id_ for downstream filtering.

Examples

import edsnlp, edsnlp.pipes as eds

nlp = edsnlp.blank("eds")
nlp.add_pipe(eds.sentences())
nlp.add_pipe(eds.tnm())

text = "Conclusion : pT2c N1mi M0 R0"

doc = nlp(text)
doc.ents
# Out: (pT2c N1mi M0 R0,)

ent = doc.ents[0]
ent._.tnm.norm()
# Out: 'pT2cN1miM0R0'

ent._.tnm.dict()
# Out: {
#   'tumour_prefix': 'p',
#   'tumour': '2',
#   'tumour_specification': 'c',
#   'tumour_suffix': None,
#   'node_prefix': None,
#   'node': '1',
#   'node_specification': 'mi',
#   'node_suffix': None,
#   'metastasis_prefix': None,
#   'metastasis': '0',
#   'metastasis_specification': None,
#   'metastasis_suffix': None,
#   'pleura': None,
#   'resection_prefix': None,
#   'resection': '0',
#   'resection_specification': None,
#   'resection_loc': None,
#   'resection_suffix': None,
#   'version': None,
#   'version_year': None,
# }

Migrating from the previous version

The regex and the TNM model were rewritten. Extracted spans and norm() values are broadly compatible, and the two renamed fields keep a deprecated alias, so most code reading span._.tnm.<field> keeps working.

Renamed fields

The old names still read, with a DeprecationWarning, so existing code keeps working. resection_completeness still returns an int for a numeric status, and a str for the x and + values the previous model could not represent.

tnm.prefix                 # -> deprecated, use tnm.tumour_prefix
tnm.resection_completeness # -> deprecated, use tnm.resection

Enums replaced by strings

Prefix, Tumour, Specification, Node, Metastasis and TnmEnum were removed from edsnlp.pipes.ner.tnm.model. Every field now holds the raw matched text as a str (except version_year, an int).

# Before -- fields were enum members
from edsnlp.pipes.ner.tnm.model import Tumour
tnm.tumour is Tumour.score_2

# Now -- fields are plain strings
tnm.tumour == "2"

The value space is not open, though. The pattern has always been the gate -- the previous one restricted stages just as narrowly ([0-4o]|is for T, [0-3o]|x for N, [01o]|x for M) and the enums merely duplicated that check in the model. Only the duplicate is gone, and the accepted values are those listed in the decomposition table above. What the enums could not represent -- N4, M2, M3, Rx, R+, metastasis site codes such as PUL -- is accepted now. Only the free-text fields are genuinely open: *_suffix, resection_loc, and node ratios such as (3/12).

New fields

node_prefix, metastasis_prefix, resection_prefix, metastasis_specification, metastasis_suffix, resection_specification, resection_loc, resection_suffix and pleura. They default to None, so existing code keeps working -- but norm() and span.kb_id_ now include them, which means a normalised value may be longer than before for the same text.

Behaviour changes to be aware of

• Matching is case-insensitive. pt2n1m0 and PT2N1M0 are now extracted; previously only certain case combinations were.
• A lone T is no longer extracted unless it carries both a prefix and a specification (pT2b), or is followed by an N, M or R component. pT2 and pTx used to match and no longer do. This is the main driver of the precision gain.
• o to 0 coercion is now restricted to the numeric stage fields, so M1OSS keeps its O. Previously every field was coerced.
• A component prefix binds to its own component. In pT1 cN1 M0 the c is now node_prefix; it used to land in tumour_specification.
• norm() no longer concatenates free-text suffixes verbatim -- pT1(grade 2)N1M0 used to give pT1grade 2N1M0. Only suffixes that read as a TNM qualifier are kept; the full text stays on the field.
• banned_words is a new parameter. Pass an empty list to restore the unfiltered regex output.

Evaluation

The pipe was qualified by two physicians before production use, on an initial sample of 20 million clinical notes stratified by year, restricted to the ~5 million documents belonging to patients followed for cancer. Sampling used Neyman allocation over strata, with a minimum of 5 documents per stratum; the figures below are the corresponding stratum-weighted estimates.

Document-level recall is the share of documents containing at least one TNM mention for which at least one mention is retrieved. It is much higher than the entity-level figure because staging is usually repeated within a report.

Strata were built on the document type (pathology report / tumour board report vs. other), the oncological density of the issuing care unit, and -- for precision -- whether the mention carried only a T component, the configuration most prone to false positives. Recall strata additionally split on whether the pipe found a mention and whether the raw text contained the word TNM.

Precision errors. 21 false positives out of 366 annotations. Most come from the rule accepting the letter o as a substitute for the digit 0 (p t o m, TOM, TOC, CS tox). The rest are interfering acronyms (CMT1A, RT 3D), numbering or temporal wording (Tour 1, au T1), and mentions whose format is a valid TNM but whose context is not.

Recall errors. 27 false negatives, matching the patterns listed under Known limitations above.

Parameters

Authors and citation

The eds.tnm component was originally developed by S. Priou, B. Rance and E. Kempf (Kempf et al., 2022), and later refined by AP-HP's Data Science team.