Specs
matchlab.specs
¶
Step specs — what a fingerprint hashes.
One model per step kind, and Step._spec_key hashes it to address that step's artifact.
What belongs in a spec is therefore decided entirely by the fingerprint invariant.
It must carry everything the step's output depends on, and nothing else. Omit
something that changes the output, and collect hands back a stale artifact without
re-running. Include something that doesn't, and the step re-runs for nothing, along
with everything below it.
That rules out two things. Edges, because _fingerprint already folds in the
parents' fingerprints. Describing an input here would record it twice, and a rename
would then invalidate a subtree producing identical bytes. Names are also excluded,
with one exception: SourceSpec.name is in the output it prefixes. A setting that must
point at an input points at its position instead (ComponentsSettings.thresholds).
A spec is therefore not the serialisation format. Rebuilding a plan needs the edges
that identifying a step must exclude. matchlab.document carries them alongside it.
Classes:
-
ModelType–Whether a model is a deduper or a linker.
-
SourceSpec–Specification of a source: its name, its key, and the location it reads.
-
TransformSpec–Specification of a transform, naming the transformer that reshapes the step.
-
ModelSpec–Specification of a deduper or linker.
-
ResolverSpec–Specification of a resolver over one or more models.
ModelType
¶
Bases: StrEnum
flowchart TD
matchlab.specs.ModelType[ModelType]
click matchlab.specs.ModelType href "" "matchlab.specs.ModelType"
Whether a model is a deduper or a linker.
Attributes:
SourceSpec
¶
Bases: BaseModel
flowchart TD
matchlab.specs.SourceSpec[SourceSpec]
click matchlab.specs.SourceSpec href "" "matchlab.specs.SourceSpec"
Specification of a source: its name, its key, and the location it reads.
There is no separate list of indexed fields. What the location returns is the single declaration of what a source is. Every column is part of the record, and therefore part of that record's identity. A column you do not want to affect identity is a column that should not come back.
Attributes:
-
model_config– -
location_class(str) – -
location_settings(dict[str, Any]) – -
name(str) – -
key_field(str) –
location_class
class-attribute
instance-attribute
¶
location_class: str = Field(description='The registered name of the Location subclass to read through.')
location_settings
class-attribute
instance-attribute
¶
location_settings: dict[str, Any] = Field(default_factory=dict, description="That location's configuration, dumped to JSON — the query, for a `RelationalDB`. Never its resources, which a document names separately and a fingerprint never sees.")
name
class-attribute
instance-attribute
¶
name: str = Field(description="The source's name within the plan. Part of the spec because it is part of the output: it prefixes every column this source contributes and tags its rows in a resolver's output.")
key_field
class-attribute
instance-attribute
¶
key_field: str = Field(description=dedent("\n The name of the key field. This is the source's key for unique\n entities, such as a primary key in a relational database.\n\n Keys are always read as strings, whatever the location returns.\n\n For example, if the source describes companies, it may have used\n a Companies House number as its key.\n\n This key is ALWAYS correct. It should be something generated and\n owned by the source being indexed.\n\n For example, your organisation's CRM ID is a key field within the CRM.\n\n A CRM ID entered by hand in another dataset shouldn't be used\n as a key field.\n "))
TransformSpec
¶
Bases: BaseModel
flowchart TD
matchlab.specs.TransformSpec[TransformSpec]
click matchlab.specs.TransformSpec href "" "matchlab.specs.TransformSpec"
Specification of a transform, naming the transformer that reshapes the step.
Attributes:
-
model_config– -
transformer_class(str) – -
transformer_settings(dict[str, Any]) –
ModelSpec
¶
Bases: BaseModel
flowchart TD
matchlab.specs.ModelSpec[ModelSpec]
click matchlab.specs.ModelSpec href "" "matchlab.specs.ModelSpec"
Specification of a deduper or linker.
Attributes:
-
model_config– -
model_type(ModelType) – -
model_class(str) – -
model_settings(dict[str, Any]) –
ResolverSpec
¶
Bases: BaseModel
flowchart TD
matchlab.specs.ResolverSpec[ResolverSpec]
click matchlab.specs.ResolverSpec href "" "matchlab.specs.ResolverSpec"
Specification of a resolver over one or more models.
Attributes:
-
model_config– -
resolver_class(str) – -
resolver_settings(dict[str, Any]) –