Skip to content

edsnlp.pipelines.core.normalizer.spaces.factory

create_component(nlp, name, newline=True)

Create a new component to update the tag_ attribute of tokens.

We assign "SPACE" to token.tag to be used by optimized components such as the EDSPhraseMatcher

PARAMETER DESCRIPTION
newline

Whether to update the newline tokens too

TYPE: bool DEFAULT: True

Source code in edsnlp/pipelines/core/normalizer/spaces/factory.py 
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@Language.factory(
    "eds.spaces",
    default_config=DEFAULT_CONFIG,
    assigns=["token.tag"],
)
def create_component(
    nlp: Language,
    name: str,
    newline: bool = True,
):
    """
    Create a new component to update the `tag_` attribute of tokens.

    We assign "SPACE" to `token.tag` to be used by optimized components
    such as the EDSPhraseMatcher

    Parameters
    ----------
    newline : bool
        Whether to update the newline tokens too
    """
    return Spaces(newline=newline)