Testkit

Generate test data whose answer you already know, then score matching against it. Used throughout matchlab's own test suite to implement to oracle pattern.

Start at matchlab.testkit below — it carries the decision table for choosing between these. linked is where most work happens.

matchlab.testkit ¶

Generate data whose answer you already know, then score matching against it.

This package makes entity resolution measurable. It plants entities you already know the answer for, so a test can assert precision and recall against truth rather than eyeball a fixture.

The package has one job, done in four steps: describe the data (features), generate it and plant the answer (sources, linked), match it (matchers), then score the result (compare). LinkedSources is the handle for all of it. It planted the entities, so nothing downstream has to be told the answer.

Which do I use?¶

I want…	Use
data whose answer I know	`linked_sources_factory()` → `LinkedSources`
one source only	`source_factory()` → `GeneratedSource`
rows I wrote by hand	`source_from_tuple()`
a plan to test	`linked.dedupe(...)` / `linked.link(..., through=...)`
to score a collected plan	`linked.diff_resolver_output(resolver_output, *sources)`
to score a methodology's edges	`linked.diff_model_edges(edges, left=...)`
different columns	`FeatureConfig` + `SuffixRule` / `ReplaceRule`
a matcher that already knows the answer	`PerfectDeduper` / `PerfectLinker`
to build or bend the answer it matches on	`AnswerKey.from_sources(...)`
to compose a comparison by hand	`resolver_output_to_clusters` + `diff_entities`

dedupe() and link() wire a perfect matcher up for you, so the last two rows are only for when you want a matcher that is deliberately wrong in a particular way.

A testkit exposes what it knows about the fixture. Anything you do to the plan goes through the node it wraps: source.source.dedupe(...), model.model.resolve(). There are no forwarding shortcuts, so it is always clear which of the two you are holding.

from matchlab.testkit import linked_sources_factory

linked = linked_sources_factory(
    n_true_entities=10, engine=warehouse
).write_to_location()
resolver_output = linked.link("crn", "cdms").model.resolve().collect().entities()

identical, report = linked.diff_resolver_output(resolver_output, "crn", "cdms")
assert identical, report

Two layers, two questions¶

Ground truth is expressed twice, in different ID spaces. Neither subsumes the other, and picking the wrong one is the easiest mistake to make here:

	Synthetic-ID layer	Value-keyed layer
Asks	does this methodology work?	does the plan carry grouping forward?
Built from	`GeneratedModel.predicted_clusters`	`AnswerKey` + `Perfect*`
Scored with	`diff_model_edges`	`diff_resolver_output`
Needs a warehouse	no, the record step read is mocked	yes, and a real `collect()`
Example	`test/methodologies/`	`test/plan/test_ground_truth.py`

The split exists because matchlab derives record identity by content-hashing rows at collect time. IDs are unknowable when a fixture is built, so anything asserted against a collected plan has to be keyed by row values instead. See matchers for the full reasoning. dedupe()/link() build both, so either is available from one call.

Names not re-exported below are still importable from their module, but they are plumbing rather than entry points.

Modules:

compare –

Scoring: turn results into the vocabulary, then compare them with the answer.
entities –

The vocabulary: what a planted answer is, and what a claimed answer is.
features –

What to generate: feature declarations and the variations applied to them.
linked –

Sources that share entities, and everything you do with them.
matchers –

Matchers that already know the answer.
models –

The generated model, and the expectation it is asserted against.
sources –

One generated source: a real plan node, its rows, and the partition they imply.

Classes:

Cluster –

A set of records claimed to be one entity, an answer.
TrueEntity –

One planted real-world thing, the answer key.
FeatureConfig –

Configuration for generating a feature with variations.
ReplaceRule –

Replace occurrences of a string with another.
SourceParameters –

Configuration for generating a source.
SuffixRule –

Add a suffix to a value.
LinkedSources –

A set of generated sources, plus the true entities planted across all of them.
AnswerKey –

The sheet a perfect matcher looks answers up in: row values → true entity.
PerfectDeduper –

A perfect deduper. Emits every within-entity pair it is given.
PerfectLinker –

A perfect linker. Emits every cross-side pair sharing a true entity.
GeneratedModel –

A generated model: the plan node, its inputs, and the answer expected of it.
GeneratedSource –

Generated rows, and the Source plan node that reads them.

Functions:

diff_entities –

Compare two lists of Cluster against each other, with a diff report.
resolver_output_to_clusters –

Convert a collected resolver output into entities comparable with truth.
linked_sources_factory –

Generate a set of linked sources with tracked entities.
source_factory –

Generate a complete source testkit from configured features.
source_from_tuple –

Generate a complete source testkit from dummy data.

Cluster ¶

Bases: BaseModel, EntityIDMixin, SourceKeyMixin


              flowchart TD
              matchlab.testkit.Cluster[Cluster]
              matchlab.testkit.entities.EntityIDMixin[EntityIDMixin]
              matchlab.testkit.entities.SourceKeyMixin[SourceKeyMixin]

                              matchlab.testkit.entities.EntityIDMixin --> matchlab.testkit.Cluster
                
                matchlab.testkit.entities.SourceKeyMixin --> matchlab.testkit.Cluster
                


              click matchlab.testkit.Cluster href "" "matchlab.testkit.Cluster"
              click matchlab.testkit.entities.EntityIDMixin href "" "matchlab.testkit.entities.EntityIDMixin"
              click matchlab.testkit.entities.SourceKeyMixin href "" "matchlab.testkit.entities.SourceKeyMixin"

A set of records claimed to be one entity, an answer.

This is the unit of comparison. Both sides of diff_entities are clusters: the expected side projected from TrueEntity.cluster(), and the actual side read back off a model or a resolver's output. It carries membership and nothing else, because membership is the only thing the two sides can agree on. A resolver output's IDs are minted at collect time and have no counterpart in generated data.

Equality and hashing are over keys alone. id is carried for bookkeeping and is deliberately ignored when comparing. Contrast TrueEntity, which is the answer key rather than an answer.

Methods:

get_keys –

Get keys for a specific source.
is_subset_of_true_entity –

Check if this Cluster's references are a subset of a TrueEntity's.
similarity_ratio –

Return ratio of shared keys to total keys across all sources.

Attributes:

model_config –
id (int) –
keys (EntityReference) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True, frozen=True)

id `class-attribute` `instance-attribute` ¶

id: int = Field(default_factory=lambda: getrandbits(63))

keys `instance-attribute` ¶

keys: EntityReference

get_keys ¶

get_keys(name: str) -> set[str]

Get keys for a specific source.

Parameters:

name ¶
(str) –

Name of the source.

Returns:

set[str] –

Set of keys, empty if the source is not found.

is_subset_of_true_entity ¶

is_subset_of_true_entity(source_entity: TrueEntity) -> bool

Check if this Cluster's references are a subset of a TrueEntity's.

similarity_ratio ¶

similarity_ratio(other: Cluster) -> float

Return ratio of shared keys to total keys across all sources.

TrueEntity ¶

Bases: BaseModel, EntityIDMixin, SourceKeyMixin


              flowchart TD
              matchlab.testkit.TrueEntity[TrueEntity]
              matchlab.testkit.entities.EntityIDMixin[EntityIDMixin]
              matchlab.testkit.entities.SourceKeyMixin[SourceKeyMixin]

                              matchlab.testkit.entities.EntityIDMixin --> matchlab.testkit.TrueEntity
                
                matchlab.testkit.entities.SourceKeyMixin --> matchlab.testkit.TrueEntity
                


              click matchlab.testkit.TrueEntity href "" "matchlab.testkit.TrueEntity"
              click matchlab.testkit.entities.EntityIDMixin href "" "matchlab.testkit.entities.EntityIDMixin"
              click matchlab.testkit.entities.SourceKeyMixin href "" "matchlab.testkit.entities.SourceKeyMixin"

One planted real-world thing, the answer key.

This is what the generator started from. It holds base_values, the feature values its rows were derived from, and accumulates the keys it landed under in every source it appears in. Equality is over base_values. Two entities are the same thing if they were generated from the same values.

It spans every source, so it is not directly comparable with a result. Project it onto the sources under test with cluster() to get something that is.

Methods:

get_keys –

Get keys for a specific source.
add_source_reference –

Add or update a source reference.
cluster –

Project this true entity onto the given sources, making it comparable.

Attributes:

model_config –
id (int) –
base_values (dict[str, Any]) –
keys (EntityReference) –
total_unique_variations (int) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True)

id `class-attribute` `instance-attribute` ¶

id: int = Field(default_factory=lambda: getrandbits(63))

base_values `class-attribute` `instance-attribute` ¶

base_values: dict[str, Any] = Field(description='Feature name -> base value')

keys `class-attribute` `instance-attribute` ¶

keys: EntityReference = Field(description='Source to keys mapping', default=EntityReference(mapping=frozenset()))

total_unique_variations `class-attribute` `instance-attribute` ¶

total_unique_variations: int = Field(default=0)

get_keys ¶

get_keys(name: str) -> set[str]

Get keys for a specific source.

Parameters:

name ¶
(str) –

Name of the source.

Returns:

set[str] –

Set of keys, empty if the source is not found.

add_source_reference ¶

add_source_reference(name: str, keys: list[str]) -> None

Add or update a source reference.

Parameters:

name ¶
(str) –

Source name.
keys ¶
(list[str]) –

List of primary keys for this source.

cluster ¶

cluster(*names: str) -> Cluster | None

Project this true entity onto the given sources, making it comparable.

Comparing equality of Cluster sets is a simpler, more reliable test than checking whether Cluster objects are subsets of TrueEntity objects. This method is what makes that comparison possible:

actual: set[Cluster] = ...
expected: set[Cluster] = {
    s.cluster("source1", "source2") for s in true_entities
}

is_identical = expected == actual
missing = expected - actual
extra = actual - expected

Parameters:

*names ¶
(str, default: () ) –

Names of sources to include in the Cluster.

Returns:

Cluster | None –

A Cluster with only the named sources' keys, or None if this entity has
Cluster | None –

no keys in any of them.

FeatureConfig ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.FeatureConfig[FeatureConfig]

              

              click matchlab.testkit.FeatureConfig href "" "matchlab.testkit.FeatureConfig"

Configuration for generating a feature with variations.

Methods:

add_variations –

Add a variation rule to the feature.
protected_names –

Ensure name is not a reserved keyword.

Attributes:

model_config –
name (str) –
base_generator (str) –
parameters (tuple | None) –
unique (bool) –
drop_base (bool) –
variations (tuple[VariationRule, ...]) –
datatype (DataType) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)

name `instance-attribute` ¶

name: str

base_generator `instance-attribute` ¶

base_generator: str

parameters `class-attribute` `instance-attribute` ¶

parameters: tuple | None = Field(default=None, description='Parameters for the generator. A tuple of tuples passed to the generator.')

unique `class-attribute` `instance-attribute` ¶

unique: bool = Field(default=True, description="Whether the generator enforces uniqueness in the generated data. For example, using unique=True with the 'boolean' generator will error if more than two values are generated.")

drop_base `class-attribute` `instance-attribute` ¶

drop_base: bool = Field(default=False, description='Whether the base case is dropped.')

variations `class-attribute` `instance-attribute` ¶

variations: tuple[VariationRule, ...] = Field(default_factory=tuple)

datatype `class-attribute` `instance-attribute` ¶

datatype: DataType = Field(default_factory=lambda data: infer_data_type(data['base_generator'], data['parameters']))

add_variations ¶

add_variations(*rule: VariationRule) -> FeatureConfig

Add a variation rule to the feature.

protected_names `classmethod` ¶

protected_names(value: str) -> str

Ensure name is not a reserved keyword.

ReplaceRule ¶

Bases: VariationRule[str]


              flowchart TD
              matchlab.testkit.ReplaceRule[ReplaceRule]
              matchlab.testkit.features.VariationRule[VariationRule]

                              matchlab.testkit.features.VariationRule --> matchlab.testkit.ReplaceRule
                


              click matchlab.testkit.ReplaceRule href "" "matchlab.testkit.ReplaceRule"
              click matchlab.testkit.features.VariationRule href "" "matchlab.testkit.features.VariationRule"

Replace occurrences of a string with another.

Methods:

apply –

Apply the variation to a value.

Attributes:

model_config –
old (str) –
new (str) –
type (type[str]) –

Python type this rule can be applied to.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True)

old `instance-attribute` ¶

old: str

new `instance-attribute` ¶

new: str

type `property` ¶

type: type[str]

Python type this rule can be applied to.

apply ¶

apply(value: str) -> str

Apply the variation to a value.

SourceParameters ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.SourceParameters[SourceParameters]

              

              click matchlab.testkit.SourceParameters href "" "matchlab.testkit.SourceParameters"

Configuration for generating a source.

Attributes:

model_config –
features (tuple[FeatureConfig, ...]) –
name (str) –
engine (Engine | Connection | None) –
n_true_entities (int | None) –
repetition (int) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True, extra='forbid')

features `class-attribute` `instance-attribute` ¶

features: tuple[FeatureConfig, ...] = Field(default_factory=tuple)

name `instance-attribute` ¶

name: str

engine `class-attribute` `instance-attribute` ¶

engine: Engine | Connection | None = Field(default=None)

n_true_entities `class-attribute` `instance-attribute` ¶

n_true_entities: int | None = Field(default=None)

repetition `class-attribute` `instance-attribute` ¶

repetition: int = Field(default=0)

SuffixRule ¶

Bases: VariationRule[str]


              flowchart TD
              matchlab.testkit.SuffixRule[SuffixRule]
              matchlab.testkit.features.VariationRule[VariationRule]

                              matchlab.testkit.features.VariationRule --> matchlab.testkit.SuffixRule
                


              click matchlab.testkit.SuffixRule href "" "matchlab.testkit.SuffixRule"
              click matchlab.testkit.features.VariationRule href "" "matchlab.testkit.features.VariationRule"

Add a suffix to a value.

Methods:

apply –

Apply the variation to a value.

Attributes:

model_config –
suffix (str) –
type (type[str]) –

Python type this rule can be applied to.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True)

suffix `instance-attribute` ¶

suffix: str

type `property` ¶

type: type[str]

Python type this rule can be applied to.

apply ¶

apply(value: str) -> str

Apply the variation to a value.

LinkedSources ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.LinkedSources[LinkedSources]

              

              click matchlab.testkit.LinkedSources href "" "matchlab.testkit.LinkedSources"

A set of generated sources, plus the true entities planted across all of them.

This is the object the module docstring describes. Build a plan from it with dedupe()/link(), then score one with diff_resolver_output()/diff_model_edges().

Methods:

find_entities –

Find entities matching appearance criteria.
true_entity_subset –

Return a subset of true entities that appear in the given sources.
diff_model_edges –

Diff a model's edges against the planted true entities.
diff_resolver_output –

Diff a collected resolver output against the planted true entities.
write_to_location –

Write every source's data to its location.
dedupe –

Build a perfect deduper over one of these sources.
link –

Build a perfect linker between two of these sources.

Attributes:

model_config –
true_entities (set[TrueEntity]) –
sources (dict[str, GeneratedSource]) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True)

true_entities `class-attribute` `instance-attribute` ¶

true_entities: set[TrueEntity] = Field(default_factory=set)

sources `instance-attribute` ¶

sources: dict[str, GeneratedSource]

find_entities ¶

find_entities(min_appearances: dict[str, int] | None = None, max_appearances: dict[str, int] | None = None) -> list[TrueEntity]

Find entities matching appearance criteria.

Parameters:

min_appearances ¶
(dict[str, int] | None, default: None ) –

For each named source, the minimum number of keys an entity must have there.
max_appearances ¶
(dict[str, int] | None, default: None ) –

For each named source, the maximum number of keys an entity may have there.

true_entity_subset ¶

true_entity_subset(*sources: str) -> list[Cluster]

Return a subset of true entities that appear in the given sources.

diff_model_edges ¶

diff_model_edges(edges: DataFrame, left: GeneratedSource, right: GeneratedSource | None = None, threshold: float = 0.0) -> tuple[bool, dict]

Diff a model's edges against the planted true entities.

This takes the sources rather than their innards. The clusters the model started from, and the names to compare over, are both properties of the sources you handed it, so re-supplying them separately was only a chance to pass the wrong ones.

Parameters:

edges ¶
(DataFrame) –

The model edge table to score.
left ¶
(GeneratedSource) –

The generated source the model read as its left input.
right ¶
(GeneratedSource | None, default: None ) –

Its right input, for a linker. None for a deduper.
threshold ¶
(float, default: 0.0 ) –

Score at or above which an edge counts as a match.

Returns:

tuple[bool, dict] –

(identical, report). See diff_entities() for the report format.

diff_resolver_output ¶

diff_resolver_output(resolver_output: DataFrame, *sources: str) -> tuple[bool, dict]

Diff a collected resolver output against the planted true entities.

The counterpart to diff_model_edges for a plan that has run. It scores the (root, key, source) table Resolver.entities() returns, whose IDs are content-derived at collect time and so have no counterpart here. Only the record membership is comparable, which is exactly what a cluster asserts.

Parameters:

resolver_output ¶
(DataFrame) –

The table returned by Resolver.entities().
*sources ¶
(str, default: () ) –

The source names to compare over, e.g. "crn", "cdms".

Returns:

tuple[bool, dict] –

(identical, report). See diff_entities() for the report format.

write_to_location ¶

write_to_location() -> Self

Write every source's data to its location.

Mirrors GeneratedSource.write_to_location, so callers don't need to loop over sources by hand.

dedupe ¶

dedupe(source: str, *, true_entities: Iterable[TrueEntity] | None = None, score_range: tuple[float, float] = (0.8, 1.0), seed: int = 42) -> GeneratedModel

Build a perfect deduper over one of these sources.

The truth is implied. It is what this testkit planted, so there is nothing to thread through by hand.

Parameters:

source ¶
(str) –

Name of the source to deduplicate.
true_entities ¶
(Iterable[TrueEntity] | None, default: None ) –

Restrict the truth to these entities. Defaults to all of them, which is what you want unless you are deliberately building a model that knows only part of the answer.
score_range ¶
(tuple[float, float], default: (0.8, 1.0) ) –

Range the emitted scores fall in.
seed ¶
(int, default: 42 ) –

Random seed for the generated scores.

link ¶

link(left: str, right: str, *, through: Resolver | None = None, true_entities: Iterable[TrueEntity] | None = None, score_range: tuple[float, float] = (0.8, 1.0), seed: int = 42) -> GeneratedModel

Build a perfect linker between two of these sources.

Parameters:

left ¶
(str) –

Name of the left source.
right ¶
(str) –

Name of the right source.
through ¶
(Resolver | None, default: None ) –

Read the left source through this resolver rather than raw, so the link sits on top of an upstream dedupe. That is the layered shape worth exercising: the apex must carry the upstream grouping forward as well as its own.
true_entities ¶
(Iterable[TrueEntity] | None, default: None ) –

Restrict the truth to these entities. Defaults to all.
score_range ¶
(tuple[float, float], default: (0.8, 1.0) ) –

Range the emitted scores fall in.
seed ¶
(int, default: 42 ) –

Random seed for the generated scores.

AnswerKey ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.AnswerKey[AnswerKey]

              

              click matchlab.testkit.AnswerKey href "" "matchlab.testkit.AnswerKey"

The sheet a perfect matcher looks answers up in: row values → true entity.

Three things happen to one of these, in order:

build: AnswerKey.from_sources() derives it from generated data. Pass a subset of the true entities to make a matcher that knows only part of the answer.
store: .register() puts it in a process-local registry and returns a content-addressed ID. A matcher's settings carry that ID rather than the table itself, because settings are JSON-serialised into a step's fingerprint and a lookup table is not. Hashing the content keeps the fingerprint honest. A different answer produces a different ID, and so a different model artifact.
use: .dedupe_edges() / .link_edges() are what PerfectDeduper and PerfectLinker call at match time, against whatever record step they were handed.

groups maps a tuple of column values to a true-entity ID. columns names the columns to read, in the same order they appear in the record step the model is given (source-qualified). Linkers carry a second set for the right-hand record step. Both sides map into the same entity-ID space, which is what lets them be joined.

Methods:

from_sources –

Derive the lookup from generated sources.
dedupe_edges –

All within-entity pairs among the record step's records.
link_edges –

All cross-side pairs whose records share a true entity.
register –

Store this key, returning a content-addressed ID for it.

Attributes:

model_config –
left_columns (tuple[str, ...]) –
left_groups (dict[tuple, int]) –
right_columns (tuple[str, ...] | None) –
right_groups (dict[tuple, int] | None) –
score (float) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)

left_columns `instance-attribute` ¶

left_columns: tuple[str, ...]

left_groups `instance-attribute` ¶

left_groups: dict[tuple, int]

right_columns `class-attribute` `instance-attribute` ¶

right_columns: tuple[str, ...] | None = None

right_groups `class-attribute` `instance-attribute` ¶

right_groups: dict[tuple, int] | None = None

score `class-attribute` `instance-attribute` ¶

score: float = 1.0

from_sources `classmethod` ¶

from_sources(left: GeneratedSource, true_entities: Iterable[TrueEntity], right: GeneratedSource | None = None, score: float = 1.0) -> AnswerKey

Derive the lookup from generated sources.

Each generated row is mapped to the true entity that owns its key, then keyed by its feature values under the names the model will see them by, source-qualified, because that is how they arrive in the record step a matcher is handed.

Parameters:

left ¶
(GeneratedSource) –

The generated source the matcher reads as its left input.
right ¶
(GeneratedSource | None, default: None ) –

Its right input, for a linker. None for a deduper.
true_entities ¶
(Iterable[TrueEntity]) –

The planted entities to answer for. Pass a subset to build a matcher that knows only part of the answer.
score ¶
(float, default: 1.0 ) –

The score to emit on every edge.

dedupe_edges ¶

dedupe_edges(data: DataFrame) -> DataFrame

All within-entity pairs among the record step's records.

link_edges ¶

link_edges(left: DataFrame, right: DataFrame) -> DataFrame

All cross-side pairs whose records share a true entity.

register ¶

register() -> str

Store this key, returning a content-addressed ID for it.

Settings are JSON-serialised into a step's fingerprint and a lookup table is not, so the key itself cannot live there. Hashing its content keeps the fingerprint honest. A different answer key produces a different ID, and therefore a different model artifact.

PerfectDeduper ¶

Bases: Deduper


              flowchart TD
              matchlab.testkit.PerfectDeduper[PerfectDeduper]
              matchlab.models.dedupers.base.Deduper[Deduper]

                              matchlab.models.dedupers.base.Deduper --> matchlab.testkit.PerfectDeduper
                


              click matchlab.testkit.PerfectDeduper href "" "matchlab.testkit.PerfectDeduper"
              click matchlab.models.dedupers.base.Deduper href "" "matchlab.models.dedupers.base.Deduper"

A perfect deduper. Emits every within-entity pair it is given.

Methods:

prepare –

No preparation required.
dedupe –

Emit edges between records sharing a true entity.

Attributes:

model_config –
id (Literal['id']) –
version (int) –
truth_id (str) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(extra='forbid', frozen=True)

id `class-attribute` `instance-attribute` ¶

id: Literal['id'] = Field(default='id', description='The unique ID field in the data to dedupe')

version `class-attribute` ¶

version: int = 1

truth_id `instance-attribute` ¶

truth_id: str

prepare ¶

prepare(data: DataFrame) -> None

No preparation required.

dedupe ¶

dedupe(data: DataFrame) -> DataFrame

Emit edges between records sharing a true entity.

PerfectLinker ¶

Bases: Linker


              flowchart TD
              matchlab.testkit.PerfectLinker[PerfectLinker]
              matchlab.models.linkers.base.Linker[Linker]

                              matchlab.models.linkers.base.Linker --> matchlab.testkit.PerfectLinker
                


              click matchlab.testkit.PerfectLinker href "" "matchlab.testkit.PerfectLinker"
              click matchlab.models.linkers.base.Linker href "" "matchlab.models.linkers.base.Linker"

A perfect linker. Emits every cross-side pair sharing a true entity.

Methods:

prepare –

No preparation required.
link –

Emit edges between records sharing a true entity.

Attributes:

model_config –
left_id (Literal['id']) –
right_id (Literal['id']) –
version (int) –
truth_id (str) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(extra='forbid', frozen=True)

left_id `class-attribute` `instance-attribute` ¶

left_id: Literal['id'] = Field(default='id', description='The unique ID field in the left data')

right_id `class-attribute` `instance-attribute` ¶

right_id: Literal['id'] = Field(default='id', description='The unique ID field in the right data')

version `class-attribute` ¶

version: int = 1

truth_id `instance-attribute` ¶

truth_id: str

prepare ¶

prepare(left: DataFrame, right: DataFrame) -> None

No preparation required.

link ¶

link(left: DataFrame, right: DataFrame) -> DataFrame

Emit edges between records sharing a true entity.

GeneratedModel ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.GeneratedModel[GeneratedModel]

              

              click matchlab.testkit.GeneratedModel href "" "matchlab.testkit.GeneratedModel"

A generated model: the plan node, its inputs, and the answer expected of it.

As with GeneratedSource, this exposes what it knows about the fixture and nothing else. To run the plan, go through .model, e.g. model.model.resolve().

Attributes:

model_config –
model (Model) –
left_source (GeneratedSource) –
right_source (GeneratedSource | None) –
scores (DataFrame) –
predicted_clusters (tuple[Cluster, ...]) –

What the model's scores imply, to compare against the planted answer.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True, validate_assignment=True)

model `instance-attribute` ¶

model: Model

left_source `instance-attribute` ¶

left_source: GeneratedSource

right_source `class-attribute` `instance-attribute` ¶

right_source: GeneratedSource | None = None

scores `class-attribute` `instance-attribute` ¶

scores: DataFrame = Field(frozen=True)

predicted_clusters `property` ¶

predicted_clusters: tuple[Cluster, ...]

What the model's scores imply, to compare against the planted answer.

GeneratedSource ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.GeneratedSource[GeneratedSource]

              

              click matchlab.testkit.GeneratedSource href "" "matchlab.testkit.GeneratedSource"

Generated rows, and the Source plan node that reads them.

Sources that share entities are what make linking testable. Use linked_sources_factory in linked instead when you need that.

This exposes what it knows about the fixture: the rows, the features they were generated from, and the partition they imply. Anything you do to the plan goes through .source, which is the node itself: source.source.clean(...), source.source.name, source.source.spec. There is deliberately no shortcut. A testkit that forwarded those would be indistinguishable from the node it wraps, and knowing which one you are holding is the whole point.

Methods:

write_to_location –

Write the data to the source's location.

Attributes:

model_config –
source (Source) –
features (tuple[FeatureConfig, ...] | None) –
data (Table) –
input_clusters (tuple[Cluster, ...]) –
field_names (list[str]) –

The non-key columns of this testkit's data, in generation order.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True)

source `class-attribute` `instance-attribute` ¶

source: Source = Field(description='The Source object containing the spec and convenience methods.')

features `class-attribute` `instance-attribute` ¶

features: tuple[FeatureConfig, ...] | None = Field(description='The features used to generate the data. If None, the source data was not generated, but set manually.', default=None)

data `class-attribute` `instance-attribute` ¶

data: Table = Field(description='Data corresponding to the output of queries, without leaf data.')

input_clusters `class-attribute` `instance-attribute` ¶

input_clusters: tuple[Cluster, ...] = Field(description='The partition a matcher starts from: generated rows grouped by identical content. Not the answer, that is LinkedSources.true_entities.')

field_names `property` ¶

field_names: list[str]

The non-key columns of this testkit's data, in generation order.

Taken from the testkit rather than from Source.index_fields, which would have to read the warehouse. These names are needed before a plan is even built. Falls back to the data's own columns when the source was set manually rather than generated from features.

write_to_location ¶

write_to_location() -> Self

Write the data to the source's location.

Pass engine to source_factory to choose which warehouse that is. Testkits are generated against an in-memory SQLite engine otherwise.

Returns:

Self –

This testkit.

diff_entities ¶

diff_entities(expected: list[Cluster], actual: list[Cluster]) -> tuple[bool, dict]

Compare two lists of Cluster against each other, with a diff report.

Parameters:

expected ¶
(list[Cluster]) –

The expected Cluster list.
actual ¶
(list[Cluster]) –

The actual Cluster list.

Returns:

bool –

(identical, report). identical is True if the two lists match exactly.
dict –

report counts how each actual cluster relates to the expected ones:
tuple[bool, dict] –
- perfect: matches an expected cluster exactly.
tuple[bool, dict] –
- subset: is a subset of an expected cluster.
tuple[bool, dict] –
- superset: is a superset of an expected cluster.
tuple[bool, dict] –
- wrong: does not overlap any expected cluster.
tuple[bool, dict] –
- invalid: contains keys absent from every expected cluster.

resolver_output_to_clusters ¶

resolver_output_to_clusters(resolver_output: DataFrame) -> set[Cluster]

Convert a collected resolver output into entities comparable with truth.

This is the other half of measuring a plan against generated data. The testkit plants known entities, the plan resolves records into clusters, and this turns those clusters back into the same currency the truth is expressed in. Pair it with LinkedSources.true_entity_subset() and diff_entities():

identical, report = diff_entities(
    expected=linked.true_entity_subset("crn", "cdms"),
    actual=list(resolver_output_to_clusters(resolver_output)),
)

A Cluster compares by its keys and never by its ID (Cluster.__eq__), which is what lets this work at all. The resolver output's root is a content-derived hash minted at collect time, and has no counterpart in the testkit's synthetic ID space. Only the (source, key) membership is comparable, and that is exactly what a cluster asserts.

Parameters:

resolver_output ¶
(DataFrame) –

A table conforming to SCHEMA_RESOLVER_OUTPUT, the table Resolver.entities() returns, with root, key and source columns.

Returns:

set[Cluster] –

One Cluster per distinct root.

Raises:

ValueError –

If the required columns are absent.

linked_sources_factory `cached` ¶

linked_sources_factory(source_parameters: tuple[SourceParameters, ...] | None = None, n_true_entities: int | None = None, engine: Engine | None = None, seed: int = 42) -> LinkedSources

Generate a set of linked sources with tracked entities.

Parameters:

source_parameters ¶
(tuple[SourceParameters, ...] | None, default: None ) –

Optional tuple of source testkit parameters
n_true_entities ¶
(int | None, default: None ) –

Optional number of true entities to generate. If provided, overrides any n_true_entities in source configs. If not provided, each SourceParameters must specify its own n_true_entities.
engine ¶
(Engine | None, default: None ) –

Optional SQLAlchemy engine to use for all sources. If provided, overrides any engine in source configs.
seed ¶
(int, default: 42 ) –

Random seed for reproducibility

source_factory `cached` ¶

source_factory(features: list[FeatureConfig] | list[dict] | None = None, name: str | None = None, location_name: str = 'dbname', engine: Engine | None = None, n_true_entities: int = 10, repetition: int = 0, seed: int = 42) -> GeneratedSource

Generate a complete source testkit from configured features.

Sources created with the factory system can only use a RelationalDB, and the data at that location will be stored in a single table.

Parameters:

features ¶
(list[FeatureConfig] | list[dict] | None, default: None ) –

List of FeatureConfig objects or dictionaries to use for generating the source data. If None, defaults to a set of common features.
name ¶
(str | None, default: None ) –

Name of the source. If None, a unique name is generated. This will be used as the name of the table in the RelationalDB, but also in the str for the source.
location_name ¶
(str, default: 'dbname' ) –

The param name the engine is bound to, which is what a dumped plan names and load must supply.
engine ¶
(Engine | None, default: None ) –

SQLAlchemy engine to use for the source's RelationalDB. If None, an in-memory SQLite engine is created.
n_true_entities ¶
(int, default: 10 ) –

Number of true entities to generate. Defaults to 10.
repetition ¶
(int, default: 0 ) –

Number of times to repeat the generated data. Defaults to 0.
seed ¶
(int, default: 42 ) –

Random seed for reproducibility. Defaults to 42.

source_from_tuple ¶

source_from_tuple(data_tuple: tuple[dict[str, Any], ...], data_keys: tuple[Any], name: str | None = None, location_name: str = 'dbname', engine: Engine | None = None, seed: int = 42) -> GeneratedSource

Generate a complete source testkit from dummy data.

matchlab.testkit.linked ¶

Sources that share entities, and everything you do with them.

This is the entry point. linked_sources_factory generates several sources from one pool of planted entities, so the same real-world thing appears in more than one of them under different keys. That is what makes linking testable at all.

The object it returns is the only one that knows the answer, so everything needing the answer hangs off it. Build a plan with dedupe()/link() and score one with diff_resolver_output()/diff_model_edges(), none of which need to be told the truth.

Classes:

LinkedSources –

A set of generated sources, plus the true entities planted across all of them.

Functions:

linked_sources_factory –

Generate a set of linked sources with tracked entities.

LinkedSources ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.linked.LinkedSources[LinkedSources]

              

              click matchlab.testkit.linked.LinkedSources href "" "matchlab.testkit.linked.LinkedSources"

A set of generated sources, plus the true entities planted across all of them.

This is the object the module docstring describes. Build a plan from it with dedupe()/link(), then score one with diff_resolver_output()/diff_model_edges().

Methods:

find_entities –

Find entities matching appearance criteria.
true_entity_subset –

Return a subset of true entities that appear in the given sources.
diff_model_edges –

Diff a model's edges against the planted true entities.
diff_resolver_output –

Diff a collected resolver output against the planted true entities.
write_to_location –

Write every source's data to its location.
dedupe –

Build a perfect deduper over one of these sources.
link –

Build a perfect linker between two of these sources.

Attributes:

model_config –
true_entities (set[TrueEntity]) –
sources (dict[str, GeneratedSource]) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True)

true_entities `class-attribute` `instance-attribute` ¶

true_entities: set[TrueEntity] = Field(default_factory=set)

sources `instance-attribute` ¶

sources: dict[str, GeneratedSource]

find_entities ¶

find_entities(min_appearances: dict[str, int] | None = None, max_appearances: dict[str, int] | None = None) -> list[TrueEntity]

Find entities matching appearance criteria.

Parameters:

min_appearances ¶
(dict[str, int] | None, default: None ) –

For each named source, the minimum number of keys an entity must have there.
max_appearances ¶
(dict[str, int] | None, default: None ) –

For each named source, the maximum number of keys an entity may have there.

true_entity_subset ¶

true_entity_subset(*sources: str) -> list[Cluster]

Return a subset of true entities that appear in the given sources.

diff_model_edges ¶

diff_model_edges(edges: DataFrame, left: GeneratedSource, right: GeneratedSource | None = None, threshold: float = 0.0) -> tuple[bool, dict]

Diff a model's edges against the planted true entities.

This takes the sources rather than their innards. The clusters the model started from, and the names to compare over, are both properties of the sources you handed it, so re-supplying them separately was only a chance to pass the wrong ones.

Parameters:

edges ¶
(DataFrame) –

The model edge table to score.
left ¶
(GeneratedSource) –

The generated source the model read as its left input.
right ¶
(GeneratedSource | None, default: None ) –

Its right input, for a linker. None for a deduper.
threshold ¶
(float, default: 0.0 ) –

Score at or above which an edge counts as a match.

Returns:

tuple[bool, dict] –

(identical, report). See diff_entities() for the report format.

diff_resolver_output ¶

diff_resolver_output(resolver_output: DataFrame, *sources: str) -> tuple[bool, dict]

Diff a collected resolver output against the planted true entities.

The counterpart to diff_model_edges for a plan that has run. It scores the (root, key, source) table Resolver.entities() returns, whose IDs are content-derived at collect time and so have no counterpart here. Only the record membership is comparable, which is exactly what a cluster asserts.

Parameters:

resolver_output ¶
(DataFrame) –

The table returned by Resolver.entities().
*sources ¶
(str, default: () ) –

The source names to compare over, e.g. "crn", "cdms".

Returns:

tuple[bool, dict] –

(identical, report). See diff_entities() for the report format.

write_to_location ¶

write_to_location() -> Self

Write every source's data to its location.

Mirrors GeneratedSource.write_to_location, so callers don't need to loop over sources by hand.

dedupe ¶

dedupe(source: str, *, true_entities: Iterable[TrueEntity] | None = None, score_range: tuple[float, float] = (0.8, 1.0), seed: int = 42) -> GeneratedModel

Build a perfect deduper over one of these sources.

The truth is implied. It is what this testkit planted, so there is nothing to thread through by hand.

Parameters:

source ¶
(str) –

Name of the source to deduplicate.
true_entities ¶
(Iterable[TrueEntity] | None, default: None ) –

Restrict the truth to these entities. Defaults to all of them, which is what you want unless you are deliberately building a model that knows only part of the answer.
score_range ¶
(tuple[float, float], default: (0.8, 1.0) ) –

Range the emitted scores fall in.
seed ¶
(int, default: 42 ) –

Random seed for the generated scores.

link ¶

link(left: str, right: str, *, through: Resolver | None = None, true_entities: Iterable[TrueEntity] | None = None, score_range: tuple[float, float] = (0.8, 1.0), seed: int = 42) -> GeneratedModel

Build a perfect linker between two of these sources.

Parameters:

left ¶
(str) –

Name of the left source.
right ¶
(str) –

Name of the right source.
through ¶
(Resolver | None, default: None ) –

Read the left source through this resolver rather than raw, so the link sits on top of an upstream dedupe. That is the layered shape worth exercising: the apex must carry the upstream grouping forward as well as its own.
true_entities ¶
(Iterable[TrueEntity] | None, default: None ) –

Restrict the truth to these entities. Defaults to all.
score_range ¶
(tuple[float, float], default: (0.8, 1.0) ) –

Range the emitted scores fall in.
seed ¶
(int, default: 42 ) –

Random seed for the generated scores.

linked_sources_factory `cached` ¶

linked_sources_factory(source_parameters: tuple[SourceParameters, ...] | None = None, n_true_entities: int | None = None, engine: Engine | None = None, seed: int = 42) -> LinkedSources

Generate a set of linked sources with tracked entities.

Parameters:

source_parameters ¶
(tuple[SourceParameters, ...] | None, default: None ) –

Optional tuple of source testkit parameters
n_true_entities ¶
(int | None, default: None ) –

Optional number of true entities to generate. If provided, overrides any n_true_entities in source configs. If not provided, each SourceParameters must specify its own n_true_entities.
engine ¶
(Engine | None, default: None ) –

Optional SQLAlchemy engine to use for all sources. If provided, overrides any engine in source configs.
seed ¶
(int, default: 42 ) –

Random seed for reproducibility

matchlab.testkit.sources ¶

One generated source: a real plan node, its rows, and the partition they imply.

Classes:

GeneratedSource –

Generated rows, and the Source plan node that reads them.

Functions:

make_features_hashable –

Let callers configure source_factory with plain dicts.
values_of –

Every unique value this entity took, per source and per feature.
source_factory –

Generate a complete source testkit from configured features.
source_from_tuple –

Generate a complete source testkit from dummy data.

GeneratedSource ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.sources.GeneratedSource[GeneratedSource]

              

              click matchlab.testkit.sources.GeneratedSource href "" "matchlab.testkit.sources.GeneratedSource"

Generated rows, and the Source plan node that reads them.

Sources that share entities are what make linking testable. Use linked_sources_factory in linked instead when you need that.

This exposes what it knows about the fixture: the rows, the features they were generated from, and the partition they imply. Anything you do to the plan goes through .source, which is the node itself: source.source.clean(...), source.source.name, source.source.spec. There is deliberately no shortcut. A testkit that forwarded those would be indistinguishable from the node it wraps, and knowing which one you are holding is the whole point.

Methods:

write_to_location –

Write the data to the source's location.

Attributes:

model_config –
source (Source) –
features (tuple[FeatureConfig, ...] | None) –
data (Table) –
input_clusters (tuple[Cluster, ...]) –
field_names (list[str]) –

The non-key columns of this testkit's data, in generation order.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True)

source `class-attribute` `instance-attribute` ¶

source: Source = Field(description='The Source object containing the spec and convenience methods.')

features `class-attribute` `instance-attribute` ¶

features: tuple[FeatureConfig, ...] | None = Field(description='The features used to generate the data. If None, the source data was not generated, but set manually.', default=None)

data `class-attribute` `instance-attribute` ¶

data: Table = Field(description='Data corresponding to the output of queries, without leaf data.')

input_clusters `class-attribute` `instance-attribute` ¶

input_clusters: tuple[Cluster, ...] = Field(description='The partition a matcher starts from: generated rows grouped by identical content. Not the answer, that is LinkedSources.true_entities.')

field_names `property` ¶

field_names: list[str]

The non-key columns of this testkit's data, in generation order.

Taken from the testkit rather than from Source.index_fields, which would have to read the warehouse. These names are needed before a plan is even built. Falls back to the data's own columns when the source was set manually rather than generated from features.

write_to_location ¶

write_to_location() -> Self

Write the data to the source's location.

Pass engine to source_factory to choose which warehouse that is. Testkits are generated against an in-memory SQLite engine otherwise.

Returns:

Self –

This testkit.

make_features_hashable ¶

make_features_hashable(func: Callable[P, R]) -> Callable[P, R]

Let callers configure source_factory with plain dicts.

Converts each dict to a FeatureConfig before the wrapped function runs, so source_factory stays hashable, which @cache needs, while its callers don't have to build FeatureConfig objects by hand.

values_of ¶

values_of(entity: TrueEntity | Cluster, sources: dict[str, GeneratedSource]) -> dict[str, dict[str, list[str]]]

Every unique value this entity took, per source and per feature.

Parameters:

entity ¶
(TrueEntity | Cluster) –

The entity whose rows to look up.
sources ¶
(dict[str, GeneratedSource]) –

The generated sources to look in, by name.

Returns:

dict[str, dict[str, list[str]]] –

{source: {feature: sorted unique values}}. Each source may vary the same base
dict[str, dict[str, list[str]]] –

value differently, so they are kept apart.

Raises:

ValueError –

If the entity references a source that was not supplied.

source_factory `cached` ¶

source_factory(features: list[FeatureConfig] | list[dict] | None = None, name: str | None = None, location_name: str = 'dbname', engine: Engine | None = None, n_true_entities: int = 10, repetition: int = 0, seed: int = 42) -> GeneratedSource

Generate a complete source testkit from configured features.

Sources created with the factory system can only use a RelationalDB, and the data at that location will be stored in a single table.

Parameters:

features ¶
(list[FeatureConfig] | list[dict] | None, default: None ) –

List of FeatureConfig objects or dictionaries to use for generating the source data. If None, defaults to a set of common features.
name ¶
(str | None, default: None ) –

Name of the source. If None, a unique name is generated. This will be used as the name of the table in the RelationalDB, but also in the str for the source.
location_name ¶
(str, default: 'dbname' ) –

The param name the engine is bound to, which is what a dumped plan names and load must supply.
engine ¶
(Engine | None, default: None ) –

SQLAlchemy engine to use for the source's RelationalDB. If None, an in-memory SQLite engine is created.
n_true_entities ¶
(int, default: 10 ) –

Number of true entities to generate. Defaults to 10.
repetition ¶
(int, default: 0 ) –

Number of times to repeat the generated data. Defaults to 0.
seed ¶
(int, default: 42 ) –

Random seed for reproducibility. Defaults to 42.

source_from_tuple ¶

source_from_tuple(data_tuple: tuple[dict[str, Any], ...], data_keys: tuple[Any], name: str | None = None, location_name: str = 'dbname', engine: Engine | None = None, seed: int = 42) -> GeneratedSource

Generate a complete source testkit from dummy data.

matchlab.testkit.models ¶

The generated model, and the expectation it is asserted against.

GeneratedModel.scores and .predicted_clusters express the expected result in the testkit's own entity ID space, which is what LinkedSources.diff_model_edges compares a methodology against without running a plan. The model itself matches on row values (see matchers), so a collected plan reproduces the same answer over the content-derived IDs the record step actually carries.

Build one with LinkedSources.dedupe() or .link(). They know the answer, so nothing has to be threaded through by hand.

Classes:

GeneratedModel –

A generated model: the plan node, its inputs, and the answer expected of it.

Functions:

generate_entity_scores –

Generate scores that recover the planted entity relationships.

GeneratedModel ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.models.GeneratedModel[GeneratedModel]

              

              click matchlab.testkit.models.GeneratedModel href "" "matchlab.testkit.models.GeneratedModel"

A generated model: the plan node, its inputs, and the answer expected of it.

As with GeneratedSource, this exposes what it knows about the fixture and nothing else. To run the plan, go through .model, e.g. model.model.resolve().

Attributes:

model_config –
model (Model) –
left_source (GeneratedSource) –
right_source (GeneratedSource | None) –
scores (DataFrame) –
predicted_clusters (tuple[Cluster, ...]) –

What the model's scores imply, to compare against the planted answer.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True, validate_assignment=True)

model `instance-attribute` ¶

model: Model

left_source `instance-attribute` ¶

left_source: GeneratedSource

right_source `class-attribute` `instance-attribute` ¶

right_source: GeneratedSource | None = None

scores `class-attribute` `instance-attribute` ¶

scores: DataFrame = Field(frozen=True)

predicted_clusters `property` ¶

predicted_clusters: tuple[Cluster, ...]

What the model's scores imply, to compare against the planted answer.

generate_entity_scores ¶

generate_entity_scores(left_entities: frozenset[Cluster], right_entities: frozenset[Cluster] | None, true_entities: frozenset[TrueEntity], score_range: tuple[float, float] = (0.8, 1.0), seed: int = 42) -> DataFrame

Generate scores that recover the planted entity relationships.

Maps each Cluster to the TrueEntity it is a subset of, then emits a score for every same-entity pair. The result is a fully connected, perfectly correct score graph. This function does not generate partial, wrong, or missing matches.

Parameters:

left_entities ¶
(frozenset[Cluster]) –

Cluster objects from the left input.
right_entities ¶
(frozenset[Cluster] | None) –

Cluster objects from the right input. None to deduplicate left_entities instead.
true_entities ¶
(frozenset[TrueEntity]) –

The planted true entities to score against.
score_range ¶
(tuple[float, float], default: (0.8, 1.0) ) –

Assign each matching pair a random score in this range.
seed ¶
(int, default: 42 ) –

Random seed for reproducibility.

Returns:

DataFrame –

A Polars DataFrame with left_id, right_id, and score columns.

Raises:

ValueError –

If score_range is not increasing within [0, 1], or a Cluster is a subset of more than one true entity.

matchlab.testkit.entities ¶

The vocabulary: what a planted answer is, and what a claimed answer is.

Two types exist, and the difference between them matters:

TrueEntity is the answer: one planted real-world thing, identified by the values it was generated from, spanning every source it landed in.
Cluster is an answer: a set of records claimed to be one entity, identified by that membership alone.

TrueEntity.cluster() projects the first onto the second, so the two become comparable. Only membership can be compared this way. A resolver output's IDs are minted at collect time and have no counterpart in generated data.

Classes:

EntityReference –

Reference to an entity's presence in specific sources.
EntityIDMixin –

Shared ID behaviour for entity classes.
SourceKeyMixin –

Shared source-key behaviour for entity classes.
Cluster –

A set of records claimed to be one entity, an answer.
TrueEntity –

One planted real-world thing, the answer key.

EntityReference ¶

EntityReference(mapping: dict[str, frozenset[str]] | None = None)

Bases: frozendict


              flowchart TD
              matchlab.testkit.entities.EntityReference[EntityReference]

              

              click matchlab.testkit.entities.EntityReference href "" "matchlab.testkit.entities.EntityReference"

Reference to an entity's presence in specific sources.

Maps source names to sets of primary keys.

Initialise the EntityReference.

EntityIDMixin ¶

Shared ID behaviour for entity classes.

Provides integer conversion and comparison operators, so entities sort by id.

Attributes:

id (int) –

id `instance-attribute` ¶

id: int

SourceKeyMixin ¶

Shared source-key behaviour for entity classes.

Provides get_keys(), for reading back the keys held for one source.

Methods:

get_keys –

Get keys for a specific source.

Attributes:

keys (EntityReference) –

keys `instance-attribute` ¶

keys: EntityReference

get_keys ¶

get_keys(name: str) -> set[str]

Get keys for a specific source.

Parameters:

name ¶
(str) –

Name of the source.

Returns:

set[str] –

Set of keys, empty if the source is not found.

Cluster ¶

Bases: BaseModel, EntityIDMixin, SourceKeyMixin


              flowchart TD
              matchlab.testkit.entities.Cluster[Cluster]
              matchlab.testkit.entities.EntityIDMixin[EntityIDMixin]
              matchlab.testkit.entities.SourceKeyMixin[SourceKeyMixin]

                              matchlab.testkit.entities.EntityIDMixin --> matchlab.testkit.entities.Cluster
                
                matchlab.testkit.entities.SourceKeyMixin --> matchlab.testkit.entities.Cluster
                


              click matchlab.testkit.entities.Cluster href "" "matchlab.testkit.entities.Cluster"
              click matchlab.testkit.entities.EntityIDMixin href "" "matchlab.testkit.entities.EntityIDMixin"
              click matchlab.testkit.entities.SourceKeyMixin href "" "matchlab.testkit.entities.SourceKeyMixin"

A set of records claimed to be one entity, an answer.

This is the unit of comparison. Both sides of diff_entities are clusters: the expected side projected from TrueEntity.cluster(), and the actual side read back off a model or a resolver's output. It carries membership and nothing else, because membership is the only thing the two sides can agree on. A resolver output's IDs are minted at collect time and have no counterpart in generated data.

Equality and hashing are over keys alone. id is carried for bookkeeping and is deliberately ignored when comparing. Contrast TrueEntity, which is the answer key rather than an answer.

Methods:

is_subset_of_true_entity –

Check if this Cluster's references are a subset of a TrueEntity's.
similarity_ratio –

Return ratio of shared keys to total keys across all sources.
get_keys –

Get keys for a specific source.

Attributes:

model_config –
id (int) –
keys (EntityReference) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True, frozen=True)

id `class-attribute` `instance-attribute` ¶

id: int = Field(default_factory=lambda: getrandbits(63))

keys `instance-attribute` ¶

keys: EntityReference

is_subset_of_true_entity ¶

is_subset_of_true_entity(source_entity: TrueEntity) -> bool

Check if this Cluster's references are a subset of a TrueEntity's.

similarity_ratio ¶

similarity_ratio(other: Cluster) -> float

Return ratio of shared keys to total keys across all sources.

get_keys ¶

get_keys(name: str) -> set[str]

Get keys for a specific source.

Parameters:

name ¶
(str) –

Name of the source.

Returns:

set[str] –

Set of keys, empty if the source is not found.

TrueEntity ¶

Bases: BaseModel, EntityIDMixin, SourceKeyMixin


              flowchart TD
              matchlab.testkit.entities.TrueEntity[TrueEntity]
              matchlab.testkit.entities.EntityIDMixin[EntityIDMixin]
              matchlab.testkit.entities.SourceKeyMixin[SourceKeyMixin]

                              matchlab.testkit.entities.EntityIDMixin --> matchlab.testkit.entities.TrueEntity
                
                matchlab.testkit.entities.SourceKeyMixin --> matchlab.testkit.entities.TrueEntity
                


              click matchlab.testkit.entities.TrueEntity href "" "matchlab.testkit.entities.TrueEntity"
              click matchlab.testkit.entities.EntityIDMixin href "" "matchlab.testkit.entities.EntityIDMixin"
              click matchlab.testkit.entities.SourceKeyMixin href "" "matchlab.testkit.entities.SourceKeyMixin"

One planted real-world thing, the answer key.

This is what the generator started from. It holds base_values, the feature values its rows were derived from, and accumulates the keys it landed under in every source it appears in. Equality is over base_values. Two entities are the same thing if they were generated from the same values.

It spans every source, so it is not directly comparable with a result. Project it onto the sources under test with cluster() to get something that is.

Methods:

add_source_reference –

Add or update a source reference.
cluster –

Project this true entity onto the given sources, making it comparable.
get_keys –

Get keys for a specific source.

Attributes:

model_config –
id (int) –
base_values (dict[str, Any]) –
keys (EntityReference) –
total_unique_variations (int) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(arbitrary_types_allowed=True)

id `class-attribute` `instance-attribute` ¶

id: int = Field(default_factory=lambda: getrandbits(63))

base_values `class-attribute` `instance-attribute` ¶

base_values: dict[str, Any] = Field(description='Feature name -> base value')

keys `class-attribute` `instance-attribute` ¶

keys: EntityReference = Field(description='Source to keys mapping', default=EntityReference(mapping=frozenset()))

total_unique_variations `class-attribute` `instance-attribute` ¶

total_unique_variations: int = Field(default=0)

add_source_reference ¶

add_source_reference(name: str, keys: list[str]) -> None

Add or update a source reference.

Parameters:

name ¶
(str) –

Source name.
keys ¶
(list[str]) –

List of primary keys for this source.

cluster ¶

cluster(*names: str) -> Cluster | None

Project this true entity onto the given sources, making it comparable.

Comparing equality of Cluster sets is a simpler, more reliable test than checking whether Cluster objects are subsets of TrueEntity objects. This method is what makes that comparison possible:

actual: set[Cluster] = ...
expected: set[Cluster] = {
    s.cluster("source1", "source2") for s in true_entities
}

is_identical = expected == actual
missing = expected - actual
extra = actual - expected

Parameters:

*names ¶
(str, default: () ) –

Names of sources to include in the Cluster.

Returns:

Cluster | None –

A Cluster with only the named sources' keys, or None if this entity has
Cluster | None –

no keys in any of them.

get_keys ¶

get_keys(name: str) -> set[str]

Get keys for a specific source.

Parameters:

name ¶
(str) –

Name of the source.

Returns:

set[str] –

Set of keys, empty if the source is not found.

matchlab.testkit.compare ¶

Scoring: turn results into the vocabulary, then compare them with the answer.

The functions here convert a resolver's or model's output into Cluster objects, and diff two Cluster lists. None of that needs the planted answer, which is why they stay as plain functions rather than methods. LinkedSources.diff_resolver_output() and .diff_model_edges() do need the planted answer, so those are methods on the one object that holds it.

Every comparison returns (identical, report), where the report classifies each cluster as perfect, subset, superset, wrong or invalid. That breakdown is the point. "Eight clusters were subsets" tells you the matcher is too strict, in a way a number cannot.

Functions:

resolver_output_to_clusters –

Convert a collected resolver output into entities comparable with truth.
scores_to_clusters –

Merge clusters connected by a score at or above threshold.
diff_entities –

Compare two lists of Cluster against each other, with a diff report.

resolver_output_to_clusters ¶

resolver_output_to_clusters(resolver_output: DataFrame) -> set[Cluster]

Convert a collected resolver output into entities comparable with truth.

This is the other half of measuring a plan against generated data. The testkit plants known entities, the plan resolves records into clusters, and this turns those clusters back into the same currency the truth is expressed in. Pair it with LinkedSources.true_entity_subset() and diff_entities():

identical, report = diff_entities(
    expected=linked.true_entity_subset("crn", "cdms"),
    actual=list(resolver_output_to_clusters(resolver_output)),
)

A Cluster compares by its keys and never by its ID (Cluster.__eq__), which is what lets this work at all. The resolver output's root is a content-derived hash minted at collect time, and has no counterpart in the testkit's synthetic ID space. Only the (source, key) membership is comparable, and that is exactly what a cluster asserts.

Parameters:

resolver_output ¶
(DataFrame) –

A table conforming to SCHEMA_RESOLVER_OUTPUT, the table Resolver.entities() returns, with root, key and source columns.

Returns:

set[Cluster] –

One Cluster per distinct root.

Raises:

ValueError –

If the required columns are absent.

scores_to_clusters ¶

scores_to_clusters(scores: DataFrame, left_clusters: tuple[Cluster, ...], right_clusters: tuple[Cluster, ...] | None = None, threshold: float = 0.0) -> tuple[Cluster, ...]

Merge clusters connected by a score at or above threshold.

Left and right clusters that share an edge scoring at least threshold merge into one Cluster. With no right_clusters, this merges within left_clusters alone (deduplication).

Parameters:

scores ¶
(DataFrame) –

A left_id/right_id/score edge table.
left_clusters ¶
(tuple[Cluster, ...]) –

Clusters the model read as its left input.
right_clusters ¶
(tuple[Cluster, ...] | None, default: None ) –

Clusters read as the right input, for a linker. None for a dedupe.
threshold ¶
(float, default: 0.0 ) –

Score at or above which an edge counts as a match.

Returns:

tuple[Cluster, ...] –

One merged Cluster per connected component.

diff_entities ¶

diff_entities(expected: list[Cluster], actual: list[Cluster]) -> tuple[bool, dict]

Compare two lists of Cluster against each other, with a diff report.

Parameters:

expected ¶
(list[Cluster]) –

The expected Cluster list.
actual ¶
(list[Cluster]) –

The actual Cluster list.

Returns:

bool –

(identical, report). identical is True if the two lists match exactly.
dict –

report counts how each actual cluster relates to the expected ones:
tuple[bool, dict] –
- perfect: matches an expected cluster exactly.
tuple[bool, dict] –
- subset: is a subset of an expected cluster.
tuple[bool, dict] –
- superset: is a superset of an expected cluster.
tuple[bool, dict] –
- wrong: does not overlap any expected cluster.
tuple[bool, dict] –
- invalid: contains keys absent from every expected cluster.

matchlab.testkit.features ¶

What to generate: feature declarations and the variations applied to them.

Pure declaration. Nothing here generates or compares anything. FeatureConfig describes one column's values. Its variations are what let one true entity produce more than one form of the same value, which _generate.generate_rows turns into rows.

Classes:

VariationRule –

Abstract base class for variation rules.
SuffixRule –

Add a suffix to a value.
ReplaceRule –

Replace occurrences of a string with another.
FeatureConfig –

Configuration for generating a feature with variations.
SourceParameters –

Configuration for generating a source.

Functions:

infer_data_type –

Infer the Polars data type a Faker configuration produces.

VariationRule ¶

Bases: BaseModel, Generic[T], ABC


              flowchart TD
              matchlab.testkit.features.VariationRule[VariationRule]

              

              click matchlab.testkit.features.VariationRule href "" "matchlab.testkit.features.VariationRule"

Abstract base class for variation rules.

Methods:

apply –

Apply the variation to a value.

Attributes:

model_config –
type (type[T]) –

Python type this rule can be applied to.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True)

type `abstractmethod` `property` ¶

type: type[T]

Python type this rule can be applied to.

apply `abstractmethod` ¶

apply(value: T) -> T

Apply the variation to a value.

SuffixRule ¶

Bases: VariationRule[str]


              flowchart TD
              matchlab.testkit.features.SuffixRule[SuffixRule]
              matchlab.testkit.features.VariationRule[VariationRule]

                              matchlab.testkit.features.VariationRule --> matchlab.testkit.features.SuffixRule
                


              click matchlab.testkit.features.SuffixRule href "" "matchlab.testkit.features.SuffixRule"
              click matchlab.testkit.features.VariationRule href "" "matchlab.testkit.features.VariationRule"

Add a suffix to a value.

Methods:

apply –

Apply the variation to a value.

Attributes:

suffix (str) –
type (type[str]) –

Python type this rule can be applied to.
model_config –

suffix `instance-attribute` ¶

suffix: str

type `property` ¶

type: type[str]

Python type this rule can be applied to.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True)

apply ¶

apply(value: str) -> str

Apply the variation to a value.

ReplaceRule ¶

Bases: VariationRule[str]


              flowchart TD
              matchlab.testkit.features.ReplaceRule[ReplaceRule]
              matchlab.testkit.features.VariationRule[VariationRule]

                              matchlab.testkit.features.VariationRule --> matchlab.testkit.features.ReplaceRule
                


              click matchlab.testkit.features.ReplaceRule href "" "matchlab.testkit.features.ReplaceRule"
              click matchlab.testkit.features.VariationRule href "" "matchlab.testkit.features.VariationRule"

Replace occurrences of a string with another.

Methods:

apply –

Apply the variation to a value.

Attributes:

old (str) –
new (str) –
type (type[str]) –

Python type this rule can be applied to.
model_config –

old `instance-attribute` ¶

old: str

new `instance-attribute` ¶

new: str

type `property` ¶

type: type[str]

Python type this rule can be applied to.

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True)

apply ¶

apply(value: str) -> str

Apply the variation to a value.

FeatureConfig ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.features.FeatureConfig[FeatureConfig]

              

              click matchlab.testkit.features.FeatureConfig href "" "matchlab.testkit.features.FeatureConfig"

Configuration for generating a feature with variations.

Methods:

add_variations –

Add a variation rule to the feature.
protected_names –

Ensure name is not a reserved keyword.

Attributes:

model_config –
name (str) –
base_generator (str) –
parameters (tuple | None) –
unique (bool) –
drop_base (bool) –
variations (tuple[VariationRule, ...]) –
datatype (DataType) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)

name `instance-attribute` ¶

name: str

base_generator `instance-attribute` ¶

base_generator: str

parameters `class-attribute` `instance-attribute` ¶

parameters: tuple | None = Field(default=None, description='Parameters for the generator. A tuple of tuples passed to the generator.')

unique `class-attribute` `instance-attribute` ¶

unique: bool = Field(default=True, description="Whether the generator enforces uniqueness in the generated data. For example, using unique=True with the 'boolean' generator will error if more than two values are generated.")

drop_base `class-attribute` `instance-attribute` ¶

drop_base: bool = Field(default=False, description='Whether the base case is dropped.')

variations `class-attribute` `instance-attribute` ¶

variations: tuple[VariationRule, ...] = Field(default_factory=tuple)

datatype `class-attribute` `instance-attribute` ¶

datatype: DataType = Field(default_factory=lambda data: infer_data_type(data['base_generator'], data['parameters']))

add_variations ¶

add_variations(*rule: VariationRule) -> FeatureConfig

Add a variation rule to the feature.

protected_names `classmethod` ¶

protected_names(value: str) -> str

Ensure name is not a reserved keyword.

SourceParameters ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.features.SourceParameters[SourceParameters]

              

              click matchlab.testkit.features.SourceParameters href "" "matchlab.testkit.features.SourceParameters"

Configuration for generating a source.

Attributes:

model_config –
features (tuple[FeatureConfig, ...]) –
name (str) –
engine (Engine | Connection | None) –
n_true_entities (int | None) –
repetition (int) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True, extra='forbid')

features `class-attribute` `instance-attribute` ¶

features: tuple[FeatureConfig, ...] = Field(default_factory=tuple)

name `instance-attribute` ¶

name: str

engine `class-attribute` `instance-attribute` ¶

engine: Engine | Connection | None = Field(default=None)

n_true_entities `class-attribute` `instance-attribute` ¶

n_true_entities: int | None = Field(default=None)

repetition `class-attribute` `instance-attribute` ¶

repetition: int = Field(default=0)

infer_data_type ¶

infer_data_type(base: str, parameters: tuple | None) -> DataType

Infer the Polars data type a Faker configuration produces.

Parameters:

base ¶
(str) –

Faker generator type.
parameters ¶
(tuple | None) –

Parameters for the generator.

Returns:

DataType –

The Polars data type of the generated values.

matchlab.testkit.matchers ¶

Matchers that already know the answer.

A perfect matcher makes no mistakes, so pointing one at generated data leaves the plan as the only variable. That is what makes an end-to-end assertion mean something. Build them through LinkedSources.dedupe() / .link(). They are public here only so the registry can name them.

The subtlety is how a matcher identifies a record. Pre-generating edges between record IDs does not work, because matchlab derives cluster IDs by content-hashing rows at collect time. The IDs a model actually receives are unknowable when a fixture is built, which is why an AnswerKey is keyed by the row's values instead. At match time the matcher reads those columns out of the record step it was handed, looks up each row's true entity, and emits edges between whatever IDs are actually present. That makes it independent of how identity is assigned, and lets it survive any future change to ID minting.

Classes:

AnswerKey –

The sheet a perfect matcher looks answers up in: row values → true entity.
PerfectDeduper –

A perfect deduper. Emits every within-entity pair it is given.
PerfectLinker –

A perfect linker. Emits every cross-side pair sharing a true entity.

AnswerKey ¶

Bases: BaseModel


              flowchart TD
              matchlab.testkit.matchers.AnswerKey[AnswerKey]

              

              click matchlab.testkit.matchers.AnswerKey href "" "matchlab.testkit.matchers.AnswerKey"

The sheet a perfect matcher looks answers up in: row values → true entity.

Three things happen to one of these, in order:

build: AnswerKey.from_sources() derives it from generated data. Pass a subset of the true entities to make a matcher that knows only part of the answer.
store: .register() puts it in a process-local registry and returns a content-addressed ID. A matcher's settings carry that ID rather than the table itself, because settings are JSON-serialised into a step's fingerprint and a lookup table is not. Hashing the content keeps the fingerprint honest. A different answer produces a different ID, and so a different model artifact.
use: .dedupe_edges() / .link_edges() are what PerfectDeduper and PerfectLinker call at match time, against whatever record step they were handed.

groups maps a tuple of column values to a true-entity ID. columns names the columns to read, in the same order they appear in the record step the model is given (source-qualified). Linkers carry a second set for the right-hand record step. Both sides map into the same entity-ID space, which is what lets them be joined.

Methods:

from_sources –

Derive the lookup from generated sources.
dedupe_edges –

All within-entity pairs among the record step's records.
link_edges –

All cross-side pairs whose records share a true entity.
register –

Store this key, returning a content-addressed ID for it.

Attributes:

model_config –
left_columns (tuple[str, ...]) –
left_groups (dict[tuple, int]) –
right_columns (tuple[str, ...] | None) –
right_groups (dict[tuple, int] | None) –
score (float) –

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(frozen=True, arbitrary_types_allowed=True)

left_columns `instance-attribute` ¶

left_columns: tuple[str, ...]

left_groups `instance-attribute` ¶

left_groups: dict[tuple, int]

right_columns `class-attribute` `instance-attribute` ¶

right_columns: tuple[str, ...] | None = None

right_groups `class-attribute` `instance-attribute` ¶

right_groups: dict[tuple, int] | None = None

score `class-attribute` `instance-attribute` ¶

score: float = 1.0

from_sources `classmethod` ¶

from_sources(left: GeneratedSource, true_entities: Iterable[TrueEntity], right: GeneratedSource | None = None, score: float = 1.0) -> AnswerKey

Derive the lookup from generated sources.

Each generated row is mapped to the true entity that owns its key, then keyed by its feature values under the names the model will see them by, source-qualified, because that is how they arrive in the record step a matcher is handed.

Parameters:

left ¶
(GeneratedSource) –

The generated source the matcher reads as its left input.
right ¶
(GeneratedSource | None, default: None ) –

Its right input, for a linker. None for a deduper.
true_entities ¶
(Iterable[TrueEntity]) –

The planted entities to answer for. Pass a subset to build a matcher that knows only part of the answer.
score ¶
(float, default: 1.0 ) –

The score to emit on every edge.

dedupe_edges ¶

dedupe_edges(data: DataFrame) -> DataFrame

All within-entity pairs among the record step's records.

link_edges ¶

link_edges(left: DataFrame, right: DataFrame) -> DataFrame

All cross-side pairs whose records share a true entity.

register ¶

register() -> str

Store this key, returning a content-addressed ID for it.

Settings are JSON-serialised into a step's fingerprint and a lookup table is not, so the key itself cannot live there. Hashing its content keeps the fingerprint honest. A different answer key produces a different ID, and therefore a different model artifact.

PerfectDeduper ¶

Bases: Deduper


              flowchart TD
              matchlab.testkit.matchers.PerfectDeduper[PerfectDeduper]
              matchlab.models.dedupers.base.Deduper[Deduper]

                              matchlab.models.dedupers.base.Deduper --> matchlab.testkit.matchers.PerfectDeduper
                


              click matchlab.testkit.matchers.PerfectDeduper href "" "matchlab.testkit.matchers.PerfectDeduper"
              click matchlab.models.dedupers.base.Deduper href "" "matchlab.models.dedupers.base.Deduper"

A perfect deduper. Emits every within-entity pair it is given.

Methods:

prepare –

No preparation required.
dedupe –

Emit edges between records sharing a true entity.

Attributes:

version (int) –
truth_id (str) –
model_config –
id (Literal['id']) –

version `class-attribute` ¶

version: int = 1

truth_id `instance-attribute` ¶

truth_id: str

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(extra='forbid', frozen=True)

id `class-attribute` `instance-attribute` ¶

id: Literal['id'] = Field(default='id', description='The unique ID field in the data to dedupe')

prepare ¶

prepare(data: DataFrame) -> None

No preparation required.

dedupe ¶

dedupe(data: DataFrame) -> DataFrame

Emit edges between records sharing a true entity.

PerfectLinker ¶

Bases: Linker


              flowchart TD
              matchlab.testkit.matchers.PerfectLinker[PerfectLinker]
              matchlab.models.linkers.base.Linker[Linker]

                              matchlab.models.linkers.base.Linker --> matchlab.testkit.matchers.PerfectLinker
                


              click matchlab.testkit.matchers.PerfectLinker href "" "matchlab.testkit.matchers.PerfectLinker"
              click matchlab.models.linkers.base.Linker href "" "matchlab.models.linkers.base.Linker"

A perfect linker. Emits every cross-side pair sharing a true entity.

Methods:

prepare –

No preparation required.
link –

Emit edges between records sharing a true entity.

Attributes:

version (int) –
truth_id (str) –
model_config –
left_id (Literal['id']) –
right_id (Literal['id']) –

version `class-attribute` ¶

version: int = 1

truth_id `instance-attribute` ¶

truth_id: str

model_config `class-attribute` `instance-attribute` ¶

model_config = ConfigDict(extra='forbid', frozen=True)

left_id `class-attribute` `instance-attribute` ¶

left_id: Literal['id'] = Field(default='id', description='The unique ID field in the left data')

right_id `class-attribute` `instance-attribute` ¶

right_id: Literal['id'] = Field(default='id', description='The unique ID field in the right data')

prepare ¶

prepare(left: DataFrame, right: DataFrame) -> None

No preparation required.

link ¶

link(left: DataFrame, right: DataFrame) -> DataFrame

Emit edges between records sharing a true entity.

Testkit

matchlab.testkit ¶

Which do I use?¶

Two layers, two questions¶

Cluster ¶

model_config class-attribute instance-attribute ¶

id class-attribute instance-attribute ¶

keys instance-attribute ¶

get_keys ¶

name ¶

is_subset_of_true_entity ¶

similarity_ratio ¶

TrueEntity ¶

model_config class-attribute instance-attribute ¶

id class-attribute instance-attribute ¶

base_values class-attribute instance-attribute ¶

keys class-attribute instance-attribute ¶

total_unique_variations class-attribute instance-attribute ¶

get_keys ¶

name ¶

add_source_reference ¶

name ¶

keys ¶

cluster ¶

*names ¶

FeatureConfig ¶

model_config class-attribute instance-attribute ¶

name instance-attribute ¶

base_generator instance-attribute ¶

parameters class-attribute instance-attribute ¶

unique class-attribute instance-attribute ¶

drop_base class-attribute instance-attribute ¶

variations class-attribute instance-attribute ¶

datatype class-attribute instance-attribute ¶

add_variations ¶

protected_names classmethod ¶

ReplaceRule ¶

model_config class-attribute instance-attribute ¶

old instance-attribute ¶

new instance-attribute ¶

type property ¶

apply ¶

SourceParameters ¶

model_config class-attribute instance-attribute ¶

features class-attribute instance-attribute ¶

name instance-attribute ¶

engine class-attribute instance-attribute ¶

n_true_entities class-attribute instance-attribute ¶

repetition class-attribute instance-attribute ¶

SuffixRule ¶

model_config class-attribute instance-attribute ¶

suffix instance-attribute ¶

type property ¶

apply ¶

LinkedSources ¶

model_config class-attribute instance-attribute ¶

true_entities class-attribute instance-attribute ¶

sources instance-attribute ¶

find_entities ¶

min_appearances ¶

max_appearances ¶

true_entity_subset ¶

diff_model_edges ¶

edges ¶

left ¶

right ¶

threshold ¶

diff_resolver_output ¶

resolver_output ¶

*sources ¶

write_to_location ¶

dedupe ¶

source ¶

true_entities ¶

score_range ¶

seed ¶

link ¶

left ¶

right ¶

through ¶

model_config `class-attribute` `instance-attribute` ¶

id `class-attribute` `instance-attribute` ¶

keys `instance-attribute` ¶

`name` ¶

model_config `class-attribute` `instance-attribute` ¶

id `class-attribute` `instance-attribute` ¶

base_values `class-attribute` `instance-attribute` ¶

keys `class-attribute` `instance-attribute` ¶

total_unique_variations `class-attribute` `instance-attribute` ¶

`name` ¶

`name` ¶

`keys` ¶

**`*names`** ¶

model_config `class-attribute` `instance-attribute` ¶

name `instance-attribute` ¶

base_generator `instance-attribute` ¶

parameters `class-attribute` `instance-attribute` ¶

unique `class-attribute` `instance-attribute` ¶

drop_base `class-attribute` `instance-attribute` ¶

variations `class-attribute` `instance-attribute` ¶

datatype `class-attribute` `instance-attribute` ¶

protected_names `classmethod` ¶

model_config `class-attribute` `instance-attribute` ¶

old `instance-attribute` ¶

new `instance-attribute` ¶

type `property` ¶

model_config `class-attribute` `instance-attribute` ¶

features `class-attribute` `instance-attribute` ¶

name `instance-attribute` ¶

engine `class-attribute` `instance-attribute` ¶

n_true_entities `class-attribute` `instance-attribute` ¶

repetition `class-attribute` `instance-attribute` ¶

model_config `class-attribute` `instance-attribute` ¶

suffix `instance-attribute` ¶

type `property` ¶

model_config `class-attribute` `instance-attribute` ¶

true_entities `class-attribute` `instance-attribute` ¶

sources `instance-attribute` ¶

`min_appearances` ¶

`max_appearances` ¶

`edges` ¶

`left` ¶

`right` ¶

`threshold` ¶

`resolver_output` ¶

**`*sources`** ¶

`source` ¶

`true_entities` ¶

`score_range` ¶

`seed` ¶

`left` ¶

`right` ¶

`through` ¶

`true_entities` ¶

`score_range` ¶

`seed` ¶

model_config `class-attribute` `instance-attribute` ¶

left_columns `instance-attribute` ¶

left_groups `instance-attribute` ¶

right_columns `class-attribute` `instance-attribute` ¶

right_groups `class-attribute` `instance-attribute` ¶

score `class-attribute` `instance-attribute` ¶

from_sources `classmethod` ¶

`left` ¶

`right` ¶

`true_entities` ¶

`score` ¶

model_config `class-attribute` `instance-attribute` ¶

id `class-attribute` `instance-attribute` ¶

version `class-attribute` ¶

truth_id `instance-attribute` ¶

model_config `class-attribute` `instance-attribute` ¶

left_id `class-attribute` `instance-attribute` ¶

right_id `class-attribute` `instance-attribute` ¶

version `class-attribute` ¶

truth_id `instance-attribute` ¶

model_config `class-attribute` `instance-attribute` ¶

model `instance-attribute` ¶

left_source `instance-attribute` ¶

right_source `class-attribute` `instance-attribute` ¶