Skip to content

Overview

matchbox.server

Matchbox server.

Includes the API, and database adapters for various backends.

Modules:

  • api –

    Matchbox API.

  • base –

    Base classes and utilities for Matchbox database adapters.

  • postgresql –

    PostgreSQL adapter for Matchbox server.

  • uploads –

    Worker logic to process user uploads.

Classes:

MatchboxDBAdapter

Bases: ABC


              flowchart TD
              matchbox.server.MatchboxDBAdapter[MatchboxDBAdapter]

              

              click matchbox.server.MatchboxDBAdapter href "" "matchbox.server.MatchboxDBAdapter"

An abstract base class for Matchbox database adapters.

By default the database should contain the users, groups and permissions found in DEFAULT_GROUPS and DEFAULT_PERMISSIONS.

Methods:

Attributes:

settings instance-attribute 

settings: MatchboxServerSettings

sources instance-attribute 

sources: ListableAndCountable

models instance-attribute 

models: Countable

resolvers instance-attribute 

resolvers: Countable

source_clusters instance-attribute 

source_clusters: Countable

model_clusters instance-attribute 

model_clusters: Countable

all_clusters instance-attribute 

all_clusters: Countable

creates instance-attribute 

creates: Countable

merges instance-attribute 

merges: Countable

proposes instance-attribute 

proposes: Countable

source_steps instance-attribute 

source_steps: Countable

users instance-attribute 

users: Countable

query abstractmethod 

query(source: SourceStepPath, resolver: ResolverStepPath | None = None, return_leaf_id: bool = False, limit: int | None = None) -> Table

Queries the database from an optional resolution.

Parameters:

  • source
    (SourceStepPath) –

    The step path identifying the source to query.

  • resolver
    (optional, default: None ) –

    The resolver path to use for filtering results. If not specified, the source step is used for the queried source.

  • return_leaf_id
    (optional, default: False ) –

    whether to return cluster ID of leaves

  • limit
    (optional, default: None ) –

    the number to use in a limit clause. Useful for testing

Returns:

  • Table –

    The resulting matchbox IDs in Arrow format

match abstractmethod 

match(key: str, source: SourceStepPath, targets: list[SourceStepPath], resolver: ResolverStepPath) -> list[Match]

Match an ID in a source step and return the keys in the targets.

Parameters:

create_collection abstractmethod 

create_collection(name: CollectionName, permissions: list[PermissionGrant]) -> Collection

Create a new collection.

Parameters:

Returns:

  • Collection –

    A Collection object containing its metadata, versions, and steps.

get_collection abstractmethod 

get_collection(name: CollectionName) -> Collection

Get collection metadata.

Parameters:

Returns:

  • Collection –

    A Collection object containing its metadata, versions, and steps.

list_collections abstractmethod 

list_collections() -> list[CollectionName]

List all collection names.

Returns:

delete_collection abstractmethod 

delete_collection(name: CollectionName, certain: bool) -> None

Delete a collection and all its versions.

Parameters:

  • name
    (CollectionName) –

    The name of the collection to delete.

  • certain
    (bool) –

    Whether to delete the collection without confirmation.

create_run abstractmethod 

create_run(collection: CollectionName) -> Run

Create a new run.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection to create the run in.

Returns:

  • Run –

    A Run object containing its metadata and steps.

set_run_mutable abstractmethod 

set_run_mutable(collection: CollectionName, run_id: RunID, mutable: bool) -> Run

Set the mutability of a run.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to update.

  • mutable
    (bool) –

    Whether the run should be mutable.

Returns:

  • Run –

    The updated Run object.

set_run_default abstractmethod 

set_run_default(collection: CollectionName, run_id: RunID, default: bool) -> Run

Set the default status of a run.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to update.

  • default
    (bool) –

    Whether the run should be the default run.

Returns:

  • Run –

    The updated Run object.

get_run abstractmethod 

get_run(collection: CollectionName, run_id: RunID) -> Run

Get run metadata and steps.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to get.

Returns:

  • Run –

    A Run object containing its metadata and steps.

delete_run abstractmethod 

delete_run(collection: CollectionName, run_id: RunID, certain: bool) -> None

Delete a run and all its steps.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to delete.

  • certain
    (bool) –

    Whether to delete the run without confirmation.

create_step abstractmethod 

create_step(step: Step, path: StepPath) -> None

Write a step to Matchbox.

Parameters:

  • step
    (Step) –

    Step object with a source, model, or resolver config

  • path
    (StepPath) –

    The step path

get_step abstractmethod 

get_step(path: StepPath) -> Step

Get a step from its path.

Parameters:

Returns:

  • Step –

    A Step object.

update_step abstractmethod 

update_step(step: Step, path: StepPath) -> None

Update step metadata.

It cannot be used to update a step’s fingerprint.

Parameters:

  • step
    (Step) –

    Step object with a source, model, or resolver config

  • path
    (StepPath) –

    The step path

delete_step abstractmethod 

delete_step(path: StepPath, certain: bool) -> None

Delete a step from the database.

Parameters:

  • path
    (StepPath) –

    The path of the step to delete.

  • certain
    (bool) –

    Whether to delete without confirmation.

lock_step_data abstractmethod 

lock_step_data(path: StepPath) -> None

Change step upload stage to PROCESSING.

This will lock uploading data.

Parameters:

  • path
    (StepPath) –

    The path of the step to target.

unlock_step_data abstractmethod 

unlock_step_data(path: StepPath, complete: bool) -> None

Change step upload stage to READY.

This will unlock uploading data.

Parameters:

  • path
    (StepPath) –

    The path of the step to target.

  • complete
    (bool) –

    Whether to label the step stage as COMPLETE.

get_step_stage abstractmethod 

get_step_stage(path: StepPath) -> UploadStage

Retrieve upload stage of step data.

Parameters:

  • path
    (StepPath) –

    The path of the step to target.

insert_source_data abstractmethod 

insert_source_data(path: SourceStepPath, data_hashes: Table) -> None

Insert hash data for a source step.

Only possible if data fingerprint matches fingerprint declared when the step was created. Data can only be set once on a step.

Parameters:

  • path
    (SourceStepPath) –

    The path of the source step to index.

  • data_hashes
    (Table) –

    The Arrow table with the hash of each data row

insert_model_data abstractmethod 

insert_model_data(path: ModelStepPath, results: Table) -> None

Insert results data for a model step.

Only possible if data fingerprint matches fingerprint declared when the step was created. Data can only be set once on a step.

insert_resolver_data abstractmethod 

insert_resolver_data(path: ResolverStepPath, data: Table) -> None

Insert resolver cluster assignments for a resolver step.

get_model_data abstractmethod 

get_model_data(path: ModelStepPath) -> Table

Get the results for a model step.

get_resolver_data abstractmethod 

get_resolver_data(path: ResolverStepPath) -> Table

Get cluster assignments for a resolver step.

validate_ids abstractmethod 

validate_ids(ids: list[int]) -> bool

Validates a list of IDs exist in the database.

Parameters:

  • ids
    (list[int]) –

    A list of IDs to validate.

Raises:

  • MatchboxDataNotFound –

    If some items don’t exist in the target table.

dump abstractmethod 

dump() -> MatchboxSnapshot

Dumps the entire database to a snapshot.

Returns:

  • MatchboxSnapshot –

    A MatchboxSnapshot object of type “postgres” with the database’s current state.

drop abstractmethod 

drop(certain: bool) -> None

Hard clear the database by dropping all tables and re-creating.

Parameters:

  • certain
    (bool) –

    Whether to drop the database without confirmation.

clear abstractmethod 

clear(certain: bool) -> None

Soft clear the database by deleting all rows but retaining tables.

Parameters:

  • certain
    (bool) –

    Whether to delete the database without confirmation.

restore abstractmethod 

restore(snapshot: MatchboxSnapshot) -> None

Restores the database from a snapshot.

Parameters:

  • snapshot
    (MatchboxSnapshot) –

    A MatchboxSnapshot object of type “postgres” with the database’s state

Raises:

  • TypeError –

    If the snapshot is not compatible with PostgreSQL

delete_orphans abstractmethod 

delete_orphans() -> int

Deletes orphan clusters.

Orphan clusters are clusters recorded in the Clusters table but that are not referenced in other tables.

login abstractmethod 

login(user: User) -> LoginResponse

Upserts the user to the database.

  • If it’s the first user, will add them to the admins group
  • For all new users, are added to the public group

Parameters:

  • user
    (User) –

    A User with a username and optionally an email

get_user_groups abstractmethod 

get_user_groups(user_name: str) -> list[GroupName]

Get names of all groups a user belongs to.

Parameters:

  • user_name
    (str) –

    The username to get the groups of

list_groups abstractmethod 

list_groups() -> list[Group]

List all groups.

get_group abstractmethod 

get_group(name: GroupName) -> Group

Get group details including members.

Parameters:

  • name
    (GroupName) –

    The name of the group to fetch.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

create_group abstractmethod 

create_group(group: Group) -> None

Create a new group.

Arg

group: The group to create

Raises:

  • MatchboxGroupAlreadyExistsError –

    If group name taken.

delete_group abstractmethod 

delete_group(name: GroupName, certain: bool = False) -> None

Delete a group.

Parameters:

  • name
    (GroupName) –

    The name of the group to delete

  • certain
    (bool, default: False ) –

    Whether to delete the group without confirmation.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

  • MatchboxSystemGroupError –

    If attempting to delete a system group.

  • MatchboxDeletionNotConfirmed –

    If certain=False.

add_user_to_group abstractmethod 

add_user_to_group(user_name: str, group_name: GroupName) -> None

Add a user to a group. Creates user if they don’t exist.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

remove_user_from_group abstractmethod 

remove_user_from_group(user_name: str, group_name: GroupName) -> None

Remove a user from a group.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

  • MatchboxUserNotFoundError –

    If user doesn’t exist.

check_permission abstractmethod 

check_permission(user_name: str, permission: PermissionType, resource: Literal[SYSTEM] | CollectionName) -> bool

Check user permission against a resource.

Parameters:

get_permissions abstractmethod 

get_permissions(resource: Literal[SYSTEM] | CollectionName) -> list[PermissionGrant]

Get all granted permissions against a resource.

Parameters:

grant_permission abstractmethod 

grant_permission(group_name: GroupName, permission: PermissionType, resource: Literal[SYSTEM] | CollectionName) -> None

Grants a permission on a resource.

Parameters:

revoke_permission abstractmethod 

revoke_permission(group_name: GroupName, permission: PermissionType, resource: Literal[SYSTEM] | CollectionName) -> None

Revoke permission on a resource.

Parameters:

insert_judgement abstractmethod 

insert_judgement(user_name: str, judgement: Judgement) -> None

Adds an evaluation judgement to the database.

Parameters:

  • user_name
    (str) –

    Name of user inserting the judgement

  • judgement
    (Judgement) –

    Representation of the proposed clusters.

get_judgements abstractmethod 

get_judgements(tag: str | None = None) -> tuple[Table, Table]

Retrieves all evaluation judgements.

Parameters:

  • tag
    (str | None, default: None ) –

    optional string by which to filter judgements

Returns:

  • Table –

    Two PyArrow tables with the judgments and their expansion.

  • Table –

    See matchbox.common.arrow for information on the schema.

sample_for_eval abstractmethod 

sample_for_eval(n: int, path: ResolverStepPath, user_name: str) -> Table

Sample a cluster to validate.

Parameters:

  • n
    (int) –

    Number of clusters to sample

  • path
    (ResolverStepPath) –

    Path of resolver step from which to sample

  • user_name
    (str) –

    Name of user requesting the sample

Returns:

  • Table –

    An Arrow table with the same schema as returned by query()

MatchboxServerSettings

Bases: BaseSettings


              flowchart TD
              matchbox.server.MatchboxServerSettings[MatchboxServerSettings]

              

              click matchbox.server.MatchboxServerSettings href "" "matchbox.server.MatchboxServerSettings"

Settings for the Matchbox application.

Methods:

Attributes:

batch_size class-attribute instance-attribute 

batch_size: int = Field(default=250000)

backend_type instance-attribute 

backend_type: MatchboxBackends

datastore instance-attribute 

datastore: MatchboxDatastoreSettings

task_runner instance-attribute 

task_runner: Literal['api', 'celery']

redis_uri instance-attribute 

redis_uri: str | None

uploads_expiry_minutes instance-attribute 

uploads_expiry_minutes: int | None

authorisation class-attribute instance-attribute 

authorisation: bool = True

public_key class-attribute instance-attribute 

public_key: SecretBytes | None = Field(default=None)

log_level class-attribute instance-attribute 

log_level: LogLevelType = 'INFO'

validate_public_key classmethod 

validate_public_key(v: str | bytes | None) -> bytes | None

Validate and normalise PEM public key format.

check_settings 

check_settings() -> Self

Check that legal combinations of settings are provided.

matchbox.server.base

Base classes and utilities for Matchbox database adapters.

Classes:

Functions:

Attributes:

PERMISSION_GRANTS module-attribute 

PERMISSION_GRANTS: dict[PermissionType, list[PermissionType]] = {READ: [READ, WRITE, ADMIN], WRITE: [WRITE, ADMIN], ADMIN: [ADMIN]}

A global variable that defines the permission hierarchy.

Keys are the permission, values are a list of permissions that would grant the permission.

For example, only PermissionType.ADMIN can grant PermissionType.ADMIN, but any permission implies PermissionType.READ.

DEFAULT_GROUPS module-attribute 

DEFAULT_GROUPS: list[Group] = [Group(name=PUBLIC, description='Unauthenticated users.', is_system=True, members=[User(user_name=PUBLIC, email=None)]), Group(name='admins', description='System administrators.', is_system=True)]

The default groups and users that should be in any fresh Matchbox backend.

DEFAULT_PERMISSIONS module-attribute 

DEFAULT_PERMISSIONS: list[tuple[PermissionGrant, BackendResourceType, str | None]] = [(PermissionGrant(group_name=ADMINS, permission=ADMIN), SYSTEM, None)]

The default permissions that should be granted in any fresh Matchbox backend.

A list of tuples in the form:

  • The permission to grant
  • The resource type to grant it on
  • The resource name to grant it on, if applicable

MatchboxBackends

Bases: StrEnum


              flowchart TD
              matchbox.server.base.MatchboxBackends[MatchboxBackends]

              

              click matchbox.server.base.MatchboxBackends href "" "matchbox.server.base.MatchboxBackends"

The available backends for Matchbox.

Attributes:

POSTGRES class-attribute instance-attribute 

POSTGRES = 'postgres'

MatchboxSnapshot

Bases: BaseModel


              flowchart TD
              matchbox.server.base.MatchboxSnapshot[MatchboxSnapshot]

              

              click matchbox.server.base.MatchboxSnapshot href "" "matchbox.server.base.MatchboxSnapshot"

A snapshot of the Matchbox database.

Methods:

Attributes:

backend_type instance-attribute 

backend_type: MatchboxBackends

data instance-attribute 

data: dict[str, Any]

check_serialisable classmethod 

check_serialisable(value: dict[str, Any]) -> dict[str, Any]

Validate that the value can be serialised to JSON.

MatchboxDatastoreSettings

Bases: BaseSettings


              flowchart TD
              matchbox.server.base.MatchboxDatastoreSettings[MatchboxDatastoreSettings]

              

              click matchbox.server.base.MatchboxDatastoreSettings href "" "matchbox.server.base.MatchboxDatastoreSettings"

Settings specific to the datastore configuration.

Methods:

  • get_client –

    Returns an S3 client for the datastore.

Attributes:

host class-attribute instance-attribute 

host: str | None = None

port class-attribute instance-attribute 

port: int | None = None

access_key_id class-attribute instance-attribute 

access_key_id: SecretStr | None = None

secret_access_key class-attribute instance-attribute 

secret_access_key: SecretStr | None = None

default_region class-attribute instance-attribute 

default_region: str | None = None

cache_bucket_name instance-attribute 

cache_bucket_name: str

get_client 

get_client() -> S3Client

Returns an S3 client for the datastore.

Creates S3 buckets if they don’t exist.

MatchboxServerSettings

Bases: BaseSettings


              flowchart TD
              matchbox.server.base.MatchboxServerSettings[MatchboxServerSettings]

              

              click matchbox.server.base.MatchboxServerSettings href "" "matchbox.server.base.MatchboxServerSettings"

Settings for the Matchbox application.

Methods:

Attributes:

batch_size class-attribute instance-attribute 

batch_size: int = Field(default=250000)

backend_type instance-attribute 

backend_type: MatchboxBackends

datastore instance-attribute 

datastore: MatchboxDatastoreSettings

task_runner instance-attribute 

task_runner: Literal['api', 'celery']

redis_uri instance-attribute 

redis_uri: str | None

uploads_expiry_minutes instance-attribute 

uploads_expiry_minutes: int | None

authorisation class-attribute instance-attribute 

authorisation: bool = True

public_key class-attribute instance-attribute 

public_key: SecretBytes | None = Field(default=None)

log_level class-attribute instance-attribute 

log_level: LogLevelType = 'INFO'

validate_public_key classmethod 

validate_public_key(v: str | bytes | None) -> bytes | None

Validate and normalise PEM public key format.

check_settings 

check_settings() -> Self

Check that legal combinations of settings are provided.

BackendManager

Manages the Matchbox backend instance and settings.

Methods:

initialise classmethod 

initialise(settings: MatchboxServerSettings) -> None

Initialise the backend with the given settings.

get_backend classmethod 

get_backend() -> MatchboxDBAdapter

Get the backend instance.

get_settings classmethod 

get_settings() -> MatchboxServerSettings

Get the backend settings.

Countable

Bases: Protocol


              flowchart TD
              matchbox.server.base.Countable[Countable]

              

              click matchbox.server.base.Countable href "" "matchbox.server.base.Countable"

A protocol for objects that can be counted.

Methods:

  • count –

    Counts the number of items in the object.

count 

count() -> int

Counts the number of items in the object.

Listable

Bases: Protocol


              flowchart TD
              matchbox.server.base.Listable[Listable]

              

              click matchbox.server.base.Listable href "" "matchbox.server.base.Listable"

A protocol for objects that can be listed.

Methods:

  • list_all –

    Lists the items in the object.

list_all 

list_all() -> list[str]

Lists the items in the object.

ListableAndCountable

Bases: Countable, Listable


              flowchart TD
              matchbox.server.base.ListableAndCountable[ListableAndCountable]
              matchbox.server.base.Countable[Countable]
              matchbox.server.base.Listable[Listable]

                              matchbox.server.base.Countable --> matchbox.server.base.ListableAndCountable
                
                matchbox.server.base.Listable --> matchbox.server.base.ListableAndCountable
                


              click matchbox.server.base.ListableAndCountable href "" "matchbox.server.base.ListableAndCountable"
              click matchbox.server.base.Countable href "" "matchbox.server.base.Countable"
              click matchbox.server.base.Listable href "" "matchbox.server.base.Listable"

A protocol for objects that can be counted and listed.

Methods:

  • list_all –

    Lists the items in the object.

  • count –

    Counts the number of items in the object.

list_all 

list_all() -> list[str]

Lists the items in the object.

count 

count() -> int

Counts the number of items in the object.

MatchboxDBAdapter

Bases: ABC


              flowchart TD
              matchbox.server.base.MatchboxDBAdapter[MatchboxDBAdapter]

              

              click matchbox.server.base.MatchboxDBAdapter href "" "matchbox.server.base.MatchboxDBAdapter"

An abstract base class for Matchbox database adapters.

By default the database should contain the users, groups and permissions found in DEFAULT_GROUPS and DEFAULT_PERMISSIONS.

Methods:

Attributes:

settings instance-attribute 

settings: MatchboxServerSettings

sources instance-attribute 

sources: ListableAndCountable

models instance-attribute 

models: Countable

resolvers instance-attribute 

resolvers: Countable

source_clusters instance-attribute 

source_clusters: Countable

model_clusters instance-attribute 

model_clusters: Countable

all_clusters instance-attribute 

all_clusters: Countable

creates instance-attribute 

creates: Countable

merges instance-attribute 

merges: Countable

proposes instance-attribute 

proposes: Countable

source_steps instance-attribute 

source_steps: Countable

users instance-attribute 

users: Countable

query abstractmethod 

query(source: SourceStepPath, resolver: ResolverStepPath | None = None, return_leaf_id: bool = False, limit: int | None = None) -> Table

Queries the database from an optional resolution.

Parameters:

  • source
    (SourceStepPath) –

    The step path identifying the source to query.

  • resolver
    (optional, default: None ) –

    The resolver path to use for filtering results. If not specified, the source step is used for the queried source.

  • return_leaf_id
    (optional, default: False ) –

    whether to return cluster ID of leaves

  • limit
    (optional, default: None ) –

    the number to use in a limit clause. Useful for testing

Returns:

  • Table –

    The resulting matchbox IDs in Arrow format

match abstractmethod 

match(key: str, source: SourceStepPath, targets: list[SourceStepPath], resolver: ResolverStepPath) -> list[Match]

Match an ID in a source step and return the keys in the targets.

Parameters:

create_collection abstractmethod 

create_collection(name: CollectionName, permissions: list[PermissionGrant]) -> Collection

Create a new collection.

Parameters:

Returns:

  • Collection –

    A Collection object containing its metadata, versions, and steps.

get_collection abstractmethod 

get_collection(name: CollectionName) -> Collection

Get collection metadata.

Parameters:

Returns:

  • Collection –

    A Collection object containing its metadata, versions, and steps.

list_collections abstractmethod 

list_collections() -> list[CollectionName]

List all collection names.

Returns:

delete_collection abstractmethod 

delete_collection(name: CollectionName, certain: bool) -> None

Delete a collection and all its versions.

Parameters:

  • name
    (CollectionName) –

    The name of the collection to delete.

  • certain
    (bool) –

    Whether to delete the collection without confirmation.

create_run abstractmethod 

create_run(collection: CollectionName) -> Run

Create a new run.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection to create the run in.

Returns:

  • Run –

    A Run object containing its metadata and steps.

set_run_mutable abstractmethod 

set_run_mutable(collection: CollectionName, run_id: RunID, mutable: bool) -> Run

Set the mutability of a run.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to update.

  • mutable
    (bool) –

    Whether the run should be mutable.

Returns:

  • Run –

    The updated Run object.

set_run_default abstractmethod 

set_run_default(collection: CollectionName, run_id: RunID, default: bool) -> Run

Set the default status of a run.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to update.

  • default
    (bool) –

    Whether the run should be the default run.

Returns:

  • Run –

    The updated Run object.

get_run abstractmethod 

get_run(collection: CollectionName, run_id: RunID) -> Run

Get run metadata and steps.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to get.

Returns:

  • Run –

    A Run object containing its metadata and steps.

delete_run abstractmethod 

delete_run(collection: CollectionName, run_id: RunID, certain: bool) -> None

Delete a run and all its steps.

Parameters:

  • collection
    (CollectionName) –

    The name of the collection containing the run.

  • run_id
    (RunID) –

    The ID of the run to delete.

  • certain
    (bool) –

    Whether to delete the run without confirmation.

create_step abstractmethod 

create_step(step: Step, path: StepPath) -> None

Write a step to Matchbox.

Parameters:

  • step
    (Step) –

    Step object with a source, model, or resolver config

  • path
    (StepPath) –

    The step path

get_step abstractmethod 

get_step(path: StepPath) -> Step

Get a step from its path.

Parameters:

Returns:

  • Step –

    A Step object.

update_step abstractmethod 

update_step(step: Step, path: StepPath) -> None

Update step metadata.

It cannot be used to update a step’s fingerprint.

Parameters:

  • step
    (Step) –

    Step object with a source, model, or resolver config

  • path
    (StepPath) –

    The step path

delete_step abstractmethod 

delete_step(path: StepPath, certain: bool) -> None

Delete a step from the database.

Parameters:

  • path
    (StepPath) –

    The path of the step to delete.

  • certain
    (bool) –

    Whether to delete without confirmation.

lock_step_data abstractmethod 

lock_step_data(path: StepPath) -> None

Change step upload stage to PROCESSING.

This will lock uploading data.

Parameters:

  • path
    (StepPath) –

    The path of the step to target.

unlock_step_data abstractmethod 

unlock_step_data(path: StepPath, complete: bool) -> None

Change step upload stage to READY.

This will unlock uploading data.

Parameters:

  • path
    (StepPath) –

    The path of the step to target.

  • complete
    (bool) –

    Whether to label the step stage as COMPLETE.

get_step_stage abstractmethod 

get_step_stage(path: StepPath) -> UploadStage

Retrieve upload stage of step data.

Parameters:

  • path
    (StepPath) –

    The path of the step to target.

insert_source_data abstractmethod 

insert_source_data(path: SourceStepPath, data_hashes: Table) -> None

Insert hash data for a source step.

Only possible if data fingerprint matches fingerprint declared when the step was created. Data can only be set once on a step.

Parameters:

  • path
    (SourceStepPath) –

    The path of the source step to index.

  • data_hashes
    (Table) –

    The Arrow table with the hash of each data row

insert_model_data abstractmethod 

insert_model_data(path: ModelStepPath, results: Table) -> None

Insert results data for a model step.

Only possible if data fingerprint matches fingerprint declared when the step was created. Data can only be set once on a step.

insert_resolver_data abstractmethod 

insert_resolver_data(path: ResolverStepPath, data: Table) -> None

Insert resolver cluster assignments for a resolver step.

get_model_data abstractmethod 

get_model_data(path: ModelStepPath) -> Table

Get the results for a model step.

get_resolver_data abstractmethod 

get_resolver_data(path: ResolverStepPath) -> Table

Get cluster assignments for a resolver step.

validate_ids abstractmethod 

validate_ids(ids: list[int]) -> bool

Validates a list of IDs exist in the database.

Parameters:

  • ids
    (list[int]) –

    A list of IDs to validate.

Raises:

  • MatchboxDataNotFound –

    If some items don’t exist in the target table.

dump abstractmethod 

dump() -> MatchboxSnapshot

Dumps the entire database to a snapshot.

Returns:

  • MatchboxSnapshot –

    A MatchboxSnapshot object of type “postgres” with the database’s current state.

drop abstractmethod 

drop(certain: bool) -> None

Hard clear the database by dropping all tables and re-creating.

Parameters:

  • certain
    (bool) –

    Whether to drop the database without confirmation.

clear abstractmethod 

clear(certain: bool) -> None

Soft clear the database by deleting all rows but retaining tables.

Parameters:

  • certain
    (bool) –

    Whether to delete the database without confirmation.

restore abstractmethod 

restore(snapshot: MatchboxSnapshot) -> None

Restores the database from a snapshot.

Parameters:

  • snapshot
    (MatchboxSnapshot) –

    A MatchboxSnapshot object of type “postgres” with the database’s state

Raises:

  • TypeError –

    If the snapshot is not compatible with PostgreSQL

delete_orphans abstractmethod 

delete_orphans() -> int

Deletes orphan clusters.

Orphan clusters are clusters recorded in the Clusters table but that are not referenced in other tables.

login abstractmethod 

login(user: User) -> LoginResponse

Upserts the user to the database.

  • If it’s the first user, will add them to the admins group
  • For all new users, are added to the public group

Parameters:

  • user
    (User) –

    A User with a username and optionally an email

get_user_groups abstractmethod 

get_user_groups(user_name: str) -> list[GroupName]

Get names of all groups a user belongs to.

Parameters:

  • user_name
    (str) –

    The username to get the groups of

list_groups abstractmethod 

list_groups() -> list[Group]

List all groups.

get_group abstractmethod 

get_group(name: GroupName) -> Group

Get group details including members.

Parameters:

  • name
    (GroupName) –

    The name of the group to fetch.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

create_group abstractmethod 

create_group(group: Group) -> None

Create a new group.

Arg

group: The group to create

Raises:

  • MatchboxGroupAlreadyExistsError –

    If group name taken.

delete_group abstractmethod 

delete_group(name: GroupName, certain: bool = False) -> None

Delete a group.

Parameters:

  • name
    (GroupName) –

    The name of the group to delete

  • certain
    (bool, default: False ) –

    Whether to delete the group without confirmation.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

  • MatchboxSystemGroupError –

    If attempting to delete a system group.

  • MatchboxDeletionNotConfirmed –

    If certain=False.

add_user_to_group abstractmethod 

add_user_to_group(user_name: str, group_name: GroupName) -> None

Add a user to a group. Creates user if they don’t exist.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

remove_user_from_group abstractmethod 

remove_user_from_group(user_name: str, group_name: GroupName) -> None

Remove a user from a group.

Raises:

  • MatchboxGroupNotFoundError –

    If group doesn’t exist.

  • MatchboxUserNotFoundError –

    If user doesn’t exist.

check_permission abstractmethod 

check_permission(user_name: str, permission: PermissionType, resource: Literal[SYSTEM] | CollectionName) -> bool

Check user permission against a resource.

Parameters:

get_permissions abstractmethod 

get_permissions(resource: Literal[SYSTEM] | CollectionName) -> list[PermissionGrant]

Get all granted permissions against a resource.

Parameters:

grant_permission abstractmethod 

grant_permission(group_name: GroupName, permission: PermissionType, resource: Literal[SYSTEM] | CollectionName) -> None

Grants a permission on a resource.

Parameters:

revoke_permission abstractmethod 

revoke_permission(group_name: GroupName, permission: PermissionType, resource: Literal[SYSTEM] | CollectionName) -> None

Revoke permission on a resource.

Parameters:

insert_judgement abstractmethod 

insert_judgement(user_name: str, judgement: Judgement) -> None

Adds an evaluation judgement to the database.

Parameters:

  • user_name
    (str) –

    Name of user inserting the judgement

  • judgement
    (Judgement) –

    Representation of the proposed clusters.

get_judgements abstractmethod 

get_judgements(tag: str | None = None) -> tuple[Table, Table]

Retrieves all evaluation judgements.

Parameters:

  • tag
    (str | None, default: None ) –

    optional string by which to filter judgements

Returns:

  • Table –

    Two PyArrow tables with the judgments and their expansion.

  • Table –

    See matchbox.common.arrow for information on the schema.

sample_for_eval abstractmethod 

sample_for_eval(n: int, path: ResolverStepPath, user_name: str) -> Table

Sample a cluster to validate.

Parameters:

  • n
    (int) –

    Number of clusters to sample

  • path
    (ResolverStepPath) –

    Path of resolver step from which to sample

  • user_name
    (str) –

    Name of user requesting the sample

Returns:

  • Table –

    An Arrow table with the same schema as returned by query()

get_backend_settings 

get_backend_settings(backend_type: MatchboxBackends) -> type[MatchboxServerSettings]

Get the appropriate settings class based on the backend type.

get_backend_class 

get_backend_class(backend_type: MatchboxBackends) -> type[MatchboxDBAdapter]

Get the appropriate backend class based on the backend type.

settings_to_backend 

settings_to_backend(settings: MatchboxServerSettings) -> MatchboxDBAdapter

Create backend adapter with injected settings.

initialise_matchbox 

initialise_matchbox() -> None

Initialise the Matchbox backend based on environment variables.