Skip to content

Naive

matchlab.models.dedupers.naive

A deduplication methodology based on a deterministic set of conditions.

Classes:

  • NaiveDeduper

    Groups records that match exactly on every field in unique_fields.

NaiveDeduper

Bases: Deduper


              flowchart TD
              matchlab.models.dedupers.naive.NaiveDeduper[NaiveDeduper]
              matchlab.models.dedupers.base.Deduper[Deduper]

                              matchlab.models.dedupers.base.Deduper --> matchlab.models.dedupers.naive.NaiveDeduper
                


              click matchlab.models.dedupers.naive.NaiveDeduper href "" "matchlab.models.dedupers.naive.NaiveDeduper"
              click matchlab.models.dedupers.base.Deduper href "" "matchlab.models.dedupers.base.Deduper"
            

Groups records that match exactly on every field in unique_fields.

Methods:

  • prepare

    No preparation needed.

  • dedupe

    Deduplicate the dataframe. Every match scores 1.0.

Attributes:

version class-attribute

version: int = 1

unique_fields class-attribute instance-attribute

unique_fields: list[str] = Field(description='A list of fields that will form a unique, deduplicated record')

model_config class-attribute instance-attribute

model_config = ConfigDict(extra='forbid', frozen=True)

id class-attribute instance-attribute

id: Literal['id'] = Field(default='id', description='The unique ID field in the data to dedupe')

prepare

prepare(data: DataFrame) -> None

No preparation needed.

dedupe

dedupe(data: DataFrame) -> DataFrame

Deduplicate the dataframe. Every match scores 1.0.