Resolvers¶

The Resolver step. A resolver's methodology takes the output of a model and resolves it into entities. It is a RecordStep that can be used in further entity resolution processes, or output as your final lookup.

matchlab.resolvers.resolvers ¶

Resolver collapses model edges into clusters and materialises the resolver output.

Classes:

Resolver –

Clusters computed over one or more models' edges.

Functions:

add_resolver_class –

Register a resolver methodology so it can be named in a plan.

Resolver ¶

Resolver(*models: Model, resolver_class: type[ResolverMethod] | str = 'Components', resolver_settings: dict[str, Any] | None = None, resolver_resources: dict[str, Any] | None = None)

Bases: RecordStep


              flowchart TD
              matchlab.resolvers.resolvers.Resolver[Resolver]
              matchlab.recordstep.RecordStep[RecordStep]
              matchlab.steps.Step[Step]

                              matchlab.recordstep.RecordStep --> matchlab.resolvers.resolvers.Resolver
                                matchlab.steps.Step --> matchlab.recordstep.RecordStep
                



              click matchlab.resolvers.resolvers.Resolver href "" "matchlab.resolvers.resolvers.Resolver"
              click matchlab.recordstep.RecordStep href "" "matchlab.recordstep.RecordStep"
              click matchlab.steps.Step href "" "matchlab.steps.Step"

Clusters computed over one or more models' edges.

A resolver is also a RecordStep. Read as records, its rows are the sources it covers, with id set to the entity root, so matching on top of a resolver is how a plan layers (deduped.link(dh, …)). The resolver output it stores is a separate artifact, and the record step itself is derived on read.

Define a resolver.

Parameters:

*models ¶
(Model, default: () ) –

The models whose edges to resolve. At least one.
resolver_class ¶
(type[ResolverMethod] | str, default: 'Components' ) –

A ResolverMethod subclass or its registered name. Defaults to connected components.
resolver_settings ¶
(dict[str, Any] | None, default: None ) –

Settings for that methodology.
resolver_resources ¶
(dict[str, Any] | None, default: None ) –

Resources the methodology needs that cannot be serialised, keyed by field name. See matchlab.resources.

Raises:

ValueError –

If an input is not a model, if there are none, or if the models bring two different sources under one name.
ResourceError –

If a field was passed in the wrong one of resolver_settings and resolver_resources, or if the models bring one resource name over two different objects.

Methods:

publish –

Point a label at this resolver's output, so it can be found without the plan.
entities –

Return (root, leaf, key, source). Collects the plan first if needed.
get_lookup –

Return root plus one qualified-key column per source.
leaf_sets –

Return each entity as a sorted list of record identities.
view_entity –

Return the stored rows for every record in one entity.
lookup_key –

Find the keys in to_sources that resolve to the same entity as key.
collect –

Materialise this step and everything it depends on.
lineage –

Return this step and all its inputs, upstream-first.
draw –

Render this step's sub-plan as a tree.
fingerprints –

Address every artifact this plan is made of, its own and its inputs'.
identifiers –

Return (id, source, key, leaf) for every record this record step reads.
data –

Return this record step's records, collecting the plan first if needed.
transform –

Reshape this record step with a transformer.
select –

Keep only the named columns, plus id.
clean –

Derive columns with DuckDB SQL, keeping the rest.
group –

Collapse each id to one row using aggregate SQL.
dedupe –

Deduplicate this record step.
link –

Link this record step to another. A Source is one, needing no wrapping.

Attributes:

kind (StepKind) –
resolver_class (type[ResolverMethod]) –
resolver_settings (dict[str, Any]) –
resolver_resources (dict[str, Resource]) –
resolver_instance (ResolverMethod) –
parents (tuple[Model, ...]) –

The models this resolver reads.
spec (ResolverSpec) –

The serialisable spec for this resolver.
sources (tuple[Source, ...]) –

The sources reachable through this resolver, in lineage order.
is_collected (bool) –

Whether this step has been materialised.

kind `class-attribute` ¶

kind: StepKind = RESOLVER

resolver_class `instance-attribute` ¶

resolver_class: type[ResolverMethod]

resolver_settings `instance-attribute` ¶

resolver_settings: dict[str, Any]

resolver_resources `instance-attribute` ¶

resolver_resources: dict[str, Resource]

resolver_instance `instance-attribute` ¶

resolver_instance: ResolverMethod

parents `property` ¶

parents: tuple[Model, ...]

The models this resolver reads.

spec `property` ¶

spec: ResolverSpec

The serialisable spec for this resolver.

sources `property` ¶

sources: tuple[Source, ...]

The sources reachable through this resolver, in lineage order.

is_collected `property` ¶

is_collected: bool

Whether this step has been materialised.

publish ¶

publish(label: str, overwrite: bool = False) -> Self

Point a label at this resolver's output, so it can be found without the plan.

Publishing is an act, not a property of the plan. A label changes nothing about what gets computed, and there is nothing to point at until the resolver output exists. Publish after collection, for example resolver.collect().publish("x"). A plan that is never published is still perfectly runnable, just unlabelled.

A label rather than a name, because a name is something else here. A source's name is part of its output, prefixing every column it contributes. A label belongs to the store, and points at whichever resolver output you last aimed it at.

Re-publishing the same label for the same resolver output is a no-op, so re-running an unchanged pipeline is safe. Aiming an existing label at a different resolver output needs overwrite=True, because that is how you lose track of what a label used to mean.

Parameters:

label ¶
(str) –

The label to publish under.
overwrite ¶
(bool, default: False ) –

Move the label if it already points somewhere else.

Returns:

Self –

This resolver, so it chains off collect().

Raises:

RuntimeError –

If this resolver has not been collected.
ValueError –

If label already points at a different resolver output and overwrite is not set.

entities ¶

entities(sources: list[str] | None = None) -> DataFrame

Return (root, leaf, key, source). Collects the plan first if needed.

One row per source record. root is the entity it resolved to, leaf its content-addressed record identity, and key its key in the original source.

Parameters:

sources ¶
(list[str] | None, default: None ) –

Restrict to these source names. Defaults to all of them.

Raises:

StepNotFound –

If sources names something this resolver does not read.

get_lookup ¶

get_lookup(sources: list[str] | None = None) -> DataFrame

Return root plus one qualified-key column per source.

The wide form of entities() is a row per entity, joined across sources, with nulls where a source has no record in it. This is the table you hand to someone who just wants their identifiers lined up.

Parameters:

sources ¶
(list[str] | None, default: None ) –

Restrict to these source names. Defaults to all of them.

leaf_sets ¶

leaf_sets(sources: list[str] | None = None) -> list[list[int]]

Return each entity as a sorted list of record identities.

Cluster IDs are dropped, so two resolver outputs over the same records can be compared by structure alone. Leaves are deduplicated, so each one appears once per key it holds.

Parameters:

sources ¶
(list[str] | None, default: None ) –

Restrict to these source names. Defaults to all of them.

view_entity ¶

view_entity(root: int, merge_fields: bool = False) -> DataFrame

Return the stored rows for every record in one entity.

Values come from the extract cached when each source was collected, not from a fresh warehouse read. That is the data the matching actually saw, the same rows the reviewer puts on screen, and it means looking at an entity needs no warehouse connection.

Parameters:

root ¶
(int) –

The entity to look at.
merge_fields ¶
(bool, default: False ) –

Collapse the source qualifier on index fields, so a field two sources share lands in one column. Key fields stay qualified, since they are what tells you which source a row came from.

Raises:

KeyError –

If no source has a record in that entity.

lookup_key ¶

lookup_key(from_source: str, to_sources: list[str], key: str) -> dict[str, list[str]]

Find the keys in to_sources that resolve to the same entity as key.

Parameters:

from_source ¶
(str) –

The source key belongs to.
to_sources ¶
(list[str]) –

The sources to find matching keys in.
key ¶
(str) –

The key to look up.

Returns:

dict[str, list[str]] –

Source name → matching keys, including from_source itself.

collect ¶

collect(store: Store | None = None, interactive: bool | None = None) -> Self

Materialise this step and everything it depends on.

Steps whose artifact is already stored are skipped without being run, so re-collecting after adding a downstream step only does the new work.

Reports as it goes: the plan, a record per step, and a closing summary of what ran, what was cached, how long it took and what the store now holds. No logging setup is needed for any of that — a collection lends the matchlab logger a console handler where the application hasn't configured one, and leaves an application that has entirely alone. See matchlab.core.logging.audible.

Parameters:

store ¶
(Store | None, default: None ) –

Where to read and write artifacts. Defaults to the module-level store (a DuckDB store in the user cache directory).
interactive ¶
(bool | None, default: None ) –

Whether someone is watching. None, the default, takes a terminal or a notebook as a yes. When they are, the plan is drawn as a live tree redrawn in place, and not logged. The tree on screen is the key those [step N] records need, and it stays there. When they are not, the plan is logged instead. See matchlab.progress.

Returns:

Self –

This step, now collected.

lineage ¶

lineage() -> list[Step]

Return this step and all its inputs, upstream-first.

draw ¶

draw() -> str

Render this step's sub-plan as a tree.

fingerprints ¶

fingerprints() -> set[Fingerprint]

Address every artifact this plan is made of, its own and its inputs'.

Which artifacts belong to a plan is the plan's own business, so this is where a store gets told: store.prune(keep=plan.fingerprints()) hands storage a set of addresses it already understands, rather than a graph it would have to learn to walk.

Returns:

set[Fingerprint] –

One fingerprint per step in lineage(). A set, because two steps in one
set[Fingerprint] –

plan can address the same artifact. Identical specs over identical
set[Fingerprint] –

inputs is the same bytes, and it is stored once.

Raises:

RuntimeError –

If any step has not been collected. An uncollected plan names no artifacts at all, so answering with a smaller set would quietly tell a caller that less is worth keeping than they think.

identifiers ¶

identifiers(store: Store) -> DataFrame

Return (id, source, key, leaf) for every record this record step reads.

id is the resolver's entity root when reading through one, otherwise the source leaf. This is the upstream resolver output a downstream resolver needs to carry every reachable leaf forward, including records no model matched.

data ¶

data(return_type: DataFrameType = POLARS) -> DataFrameClass

Return this record step's records, collecting the plan first if needed.

transform ¶

transform(transformer: Transformer | type[Transformer] | str, transformer_settings: dict | None = None, transformer_resources: dict | None = None) -> Transform

Reshape this record step with a transformer.

select ¶

select(*columns: str) -> Transform

Keep only the named columns, plus id.

clean ¶

clean(cleaning: dict[str, SQLExpression]) -> Transform

Derive columns with DuckDB SQL, keeping the rest.

group ¶

group(aggregates: dict[str, SQLExpression]) -> Transform

Collapse each id to one row using aggregate SQL.

dedupe ¶

dedupe(model_class: type[Deduper] | str, model_settings: dict | None = None, model_resources: dict | None = None) -> Model

Deduplicate this record step.

link ¶

link(other: RecordStep, model_class: type[Linker] | str, model_settings: dict | None = None, model_resources: dict | None = None) -> Model

Link this record step to another. A Source is one, needing no wrapping.

add_resolver_class ¶

add_resolver_class(resolver_class: type[ResolverMethod]) -> None

Register a resolver methodology so it can be named in a plan.

Resolvers¶

matchlab.resolvers.resolvers ¶

Resolver ¶

*models ¶

resolver_class ¶

resolver_settings ¶

resolver_resources ¶

kind class-attribute ¶

resolver_class instance-attribute ¶

resolver_settings instance-attribute ¶

resolver_resources instance-attribute ¶

resolver_instance instance-attribute ¶

parents property ¶

spec property ¶

sources property ¶

is_collected property ¶

publish ¶

label ¶

overwrite ¶

entities ¶

sources ¶

get_lookup ¶

sources ¶

leaf_sets ¶

sources ¶

view_entity ¶

root ¶

merge_fields ¶

lookup_key ¶

from_source ¶

to_sources ¶

key ¶

collect ¶

store ¶

interactive ¶

lineage ¶

draw ¶

fingerprints ¶

identifiers ¶

data ¶

transform ¶

select ¶

clean ¶

group ¶

dedupe ¶

link ¶

add_resolver_class ¶

**`*models`** ¶

`resolver_class` ¶

`resolver_settings` ¶

`resolver_resources` ¶

kind `class-attribute` ¶

resolver_class `instance-attribute` ¶

resolver_settings `instance-attribute` ¶

resolver_resources `instance-attribute` ¶

resolver_instance `instance-attribute` ¶

parents `property` ¶

spec `property` ¶

sources `property` ¶

is_collected `property` ¶

`label` ¶

`overwrite` ¶

`sources` ¶

`sources` ¶

`sources` ¶

`root` ¶

`merge_fields` ¶

`from_source` ¶

`to_sources` ¶

`key` ¶

`store` ¶

`interactive` ¶