dapla_metadata.variable_definitions package¶

dapla_metadata.variable_definitions.exceptions module¶

Vardef client exceptions.

exception PublishingBlockedError[source]¶

Bases: RuntimeError

Exception raised when publishing variable definitions is blocked in the production environment.

default_message = 'Publishing blocked: Publishing variable definitions is not allowed until further notice.'¶

exception VardefClientError(response_body)[source]¶

Bases: Exception

Custom exception to represent errors encountered in the Vardef client.

This exception extracts and formats error details from a JSON response body provided by the Vardef API, enabling more descriptive error messages. If the response body cannot be parsed as JSON or lacks expected keys, default values are used to provide meaningful feedback.

Parameters:: response_body (str)
Return type:: None

exception VardefFileError(message, *args)[source]¶

Bases: Exception

Custom exception for catching errors related to variable definition file handling.

Parameters:: message (str)
Return type:: None

message¶

Message describing the error.

Type:: str

exception VariableNotFoundError(message)[source]¶

Bases: Exception

Custom exception for when a variable is not found.

Parameters:: message (str)
Return type:: None

message¶

Message describing the error.

Type:: str

publishing_blocked_error_handler(method)[source]¶

Decorator that blocks publishing variable definitions in production.

If the environment is production:
If variable_status is present in the arguments and set to PUBLISHED_INTERNAL or PUBLISHED_EXTERNAL,
publishing is blocked by raising a PublishingBlockedError.
If variable_status is set to DRAFT, the method is allowed to proceed.
If no arguments are provided, all publishing attempts are blocked.

vardef_exception_handler(method)[source]¶: Decorator for handling exceptions in Vardef.

vardef_file_error_handler(method)[source]¶: Decorator for handling exceptions when generating yaml files for variable definitions.

dapla_metadata.variable_definitions.vardef module¶

class Vardef[source]¶

Bases: object

Create, maintain and read Variable Definitions.

Variable Definitions¶

Variable Definitions are centralized definitions of concrete variables which are typically present in multiple datasets. Variable Definitions support standardization of data and metadata and facilitate sharing and joining of data by clarifying when variables have an identical definition.

The methods in this class allow for creation, maintenance and access of Variable Definitions.

Creation and maintenance of variables may only be performed by Statistics Norway employees representing a specific Dapla team, who are defined as the owners of a given Variable Definition. The group a user represents is chosen when starting a service in Dapla Lab. This class will seamlessly detect this and send it to the API. All maintenance is to be performed by the owners, with no intervention from administrators.

Status¶

All Variable Definitions have an associated status. The possible values for status are DRAFT, PUBLISHED_INTERNAL and PUBLISHED_EXTERNAL.

Draft¶

When a Variable Definition is created it is assigned the status DRAFT. Under this status the Variable Definition is: * Only visible to Statistics Norway employees. * Mutable (it may be changed directly without need for versioning). * Not suitable to refer to from other systems. This status may be changed to PUBLISHED_INTERNAL or PUBLISHED_EXTERNAL with a direct update.

Published Internal¶

Under this status the Variable Definition is: * Only visible to Statistics Norway employees. * Immutable (all changes are versioned). * Suitable to refer to in internal systems for statistics production. * Not suitable to refer to for external use (for example in Statistikkbanken).

This status may be changed to PUBLISHED_EXTERNAL by creating a Patch version.

Published External¶

Under this status the Variable Definition is: * Visible to the general public. * Immutable (all changes are versioned). * Suitable to refer to from any system.

This status may not be changed as it would break immutability. If a Variable Definition is no longer relevant then its period of validity should be ended by specifying a valid_until date in a Patch version.

Immutability¶

Variable Definitions are immutable. This means that any changes must be performed in a strict versioning system. Consumers can avoid being exposed to breaking changes by specifying a date_of_validity when they request a Variable Definition.

classmethod create_draft(cls, draft)[source]¶

Create a Draft Variable Definition.

Return type:: VariableDefinition
Parameters:: draft (Draft)

classmethod create_draft_from_file(cls, file_path=None)[source]¶

Create a Draft Variable Definition from a stored yaml file.

By default the latest template file in the default directory is chosen, this may be overridden by providing a value for the optional file_path parameter.

Parameters:: file_path (PathLike[str], optional) – Supply a file path to override the automatic one. Defaults to None.
Raises:: FileNotFoundError – When a file can’t be found.
Returns:: The created draft variable definition.
Return type:: VariableDefinition

classmethod does_short_name_exist(cls, short_name)[source]¶

Return True if the short name exists in Vardef, otherwise False.

Return type:: bool
Parameters:: short_name (str)

classmethod get_vardok_id_by_short_name(cls, short_name)[source]¶

Retrieve a Vardok id by short name.

Parameters:: short_name (str) – The short name of the desired Variable Definition
Raises:: TypeError – If the incorrect type is returned.
Return type:: VardokId

classmethod get_variable_definition_by_id(cls, variable_definition_id, date_of_validity=None)[source]¶

Get a Variable Definition by ID.

Parameters:

variable_definition_id (str) – The ID of the desired Variable Definition
date_of_validity (date | None, optional) – List only variable definitions which are valid on this date. Defaults to None.

Returns:

The Variable Definition.

Return type:

VariableDefinition

Raises:

NotFoundException when the given ID is not found –

classmethod get_variable_definition_by_shortname(cls, short_name, date_of_validity=None)[source]¶

Retrieve a Variable Definition by short name.

Parameters:

short_name (str) – The short name of the Variable Definition.
date_of_validity (date | None, optional) – Filter by validity date. Defaults to None.

Returns:

The retrieved Variable Definition.

Return type:

VariableDefinition

Raises:

VariableNotFoundError – If no matching Variable Definition is found.
ValueError – If multiple variables with the same shortname is found.

classmethod get_variable_definition_by_vardok_id(cls, vardok_id)[source]¶

Get a Variable Definition by its Vardok ID.

Parameters:: vardok_id (str) – The Vardok ID of the desired Variable Definition
Returns:: The Variable Definition.
Return type:: VariableDefinition
Raises:: TypeError – If the incorrect type is returned.

classmethod list_vardok_vardef_mapping(cls)[source]¶

List the mapping between vardok and vardef.

Returns:: The list with mappings between Vardok and Vardef
Return type:: List[VardokVardefIdPair]

classmethod list_variable_definitions(cls, date_of_validity=None)[source]¶

List variable definitions.

Filtering¶

If no filter arguments are provided then all Variable Definitions are returned. See the documentation for the individual arguments to understand their effect. Filter arguments are combined with AND logic.

type date_of_validity:: Optional[date]
param date_of_validity:: List only variable definitions which are valid on this date. Defaults to None.
type date_of_validity:: date | None, optional
returns:: The list of Variable Definitions.
rtype:: list[VariableDefinition]

Parameters:: date_of_validity (date | None)
Return type:: list[VariableDefinition]

classmethod migrate_from_vardok(cls, vardok_id)[source]¶

Migrate a Variable Definition from Vardok to Vardef.

Each Vardok Variable Definition may only be migrated once.
The Dapla team of the person who performs the migration will be set as the owner.
All metadata should be checked for correctness before publication.

Parameters:: vardok_id (str) – The ID of a Variable Definition in Vardok.
Returns:: The migrated Variable Definition in Vardef.
Return type:: VariableDefinition

classmethod write_template_to_file(cls, custom_file_path=None)[source]¶

Write template with default values to a yaml file.

Return type:: Path
Parameters:: custom_file_path (str | None)

dapla_metadata.variable_definitions.vardok_id module¶

class VardokId(**data)[source]¶

Bases: VardokIdResponse

A Vardok id.

Provides access to the Vardok id filed.
Provides methods allowing maintenance for nicer output of the Vardok id.

Parameters:

VardokIdResponse – The Pydantic model superclass, representing a Vardok id response.
vardok_id (Annotated[str, Strict(strict=True)])

static from_model(model)[source]¶

Create a VariableDefinition instance from a CompleteResponse.

Return type:: VardokId
Parameters:: model (VardokIdResponse)

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context, /)¶

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

self (BaseModel) – The BaseModel instance.
context (Any) – The context.

Return type:

None

dapla_metadata.variable_definitions.vardok_vardef_id_pair module¶

class VardokVardefIdPair(**data)[source]¶

Bases: VardokVardefIdPairResponse

A Vardok id.

Provides access to the fields of a Vardok Vardef id pair.
Provides methods allowing for nicer output of the Vardok Vardef id pair.

Parameters:

VardokVardefIdPairResponse – The Pydantic model superclass, representing a Vardok Vardef id pair response.
vardok_id (Annotated[str, Strict(strict=True)])
vardef_id (Annotated[str, Strict(strict=True)])

static from_model(model)[source]¶

Create a VardokVardefIdPair instance from a VardokVardefIdPairResponse.

Return type:: VardokVardefIdPair
Parameters:: model (VardokVardefIdPairResponse)

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context, /)¶

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

self (BaseModel) – The BaseModel instance.
context (Any) – The context.

Return type:

None

dapla_metadata.variable_definitions.variable_definition module¶

class VariableDefinition(**data)[source]¶

Bases: CompleteResponse

A Variable Definition.

Provides access to the fields of the specific Variable Definition.
Provides methods to access Patches and Validity Periods of this Variable Definition.
Provides methods allowing maintenance of this Variable Definition.

Parameters:

CompleteResponse – The Pydantic model superclass, representing a Variable Definition.
id (Annotated[str, Strict(strict=True)])
patch_id (Annotated[int, Strict(strict=True)])
name (LanguageStringType)
short_name (Annotated[str, Strict(strict=True)])
definition (LanguageStringType)
classification_reference (Annotated[str, Strict(strict=True)] | None)
unit_types (list[Annotated[str, Strict(strict=True)]])
subject_fields (list[Annotated[str, Strict(strict=True)]])
contains_special_categories_of_personal_data (Annotated[bool, Strict(strict=True)])
variable_status (VariableStatus | None)
measurement_type (Annotated[str, Strict(strict=True)] | None)
valid_from (date)
valid_until (date | None)
external_reference_uri (Annotated[str, Strict(strict=True)] | None)
comment (LanguageStringType | None)
related_variable_definition_uris (list[Annotated[str, Strict(strict=True)]] | None)
owner (Owner)
contact (Contact)
created_at (datetime)
created_by (Annotated[str, Strict(strict=True)])
last_updated_at (datetime)
last_updated_by (Annotated[str, Strict(strict=True)])

create_patch(patch, valid_from=None)[source]¶

Create a new Patch for this Variable Definition.

Patches are to be used for minor changes which don’t require a new Validity Period. Examples of reasons for creating a new Patch: - Correcting a typo - Adding a translation - Adding a subject field

Supply only the fields to be changed. Other fields will retain their current values.

Parameters:

patch (Patch) – The input for a new patch.
valid_from (Optional[date]) – Optional date for selecting a Validity Period to create patch in. The date must exactly match the Validity Period valid_from. If value is None the patch is created in the last validity period.

Returns:

Variable Definition with all details.

Return type:

VariableDefinition

create_patch_from_file(file_path=None, valid_from=None)[source]¶

Create a new Patch for this Variable Definition from a file.

Will automatically read the relevant file pertaining to this variable definition. Can be overridden by specifying the file_path parameter.

Patches are to be used for minor changes which don’t require a new Validity Period. Examples of reasons for creating a new Patch: - Correcting a typo - Adding a translation - Adding a subject field

Supply only the fields to be changed. Other fields will retain their current values.

Parameters:

file_path (Optional[PathLike]) – Optionally specify the path to read from.
valid_from (Optional[date]) – Optional date for selecting a Validity Period to create patch in. The date must exactly match the Validity Period valid_from. If value is None the patch is created in the last validity period.

Returns:

Variable Definition with all details.

Return type:

VariableDefinition

create_validity_period(validity_period)[source]¶

Create a new Validity Period for this Variable Definition.

In order to create a new Validity Period input must contain updated ‘definition’ text for all present languages and a new valid from.

A new Validity Period should be created only when the fundamental definition of the variable has changed. This way the previous definition can be preserved for use in historical data.

Parameters:: validity_period (ValidityPeriod) – The input for new Validity Period
Returns:: Variable Definition with all details.
Return type:: VariableDefinition

create_validity_period_from_file(file_path=None)[source]¶

Create a new ValidityPeriod for this Variable Definition from a file.

In order to create a new Validity Period the input file must contain updated ‘definition’ text for all present languages and a new valid from.

Parameters:: file_path (Optional[PathLike]) – Optionally specify the path to read from.
Returns:: Variable Definition with all details.
Return type:: VariableDefinition

delete_draft()[source]¶

Delete this Variable definition.

Variable definition must have status ‘DRAFT’.

Returns:: A message if the operation was succsessful.
Return type:: str

static from_model(model)[source]¶

Create a VariableDefinition instance from a CompleteResponse.

Return type:: VariableDefinition
Parameters:: model (CompleteResponse)

get_file_path()[source]¶

Get the file path where the variable definition has been written to for editing.

Return type:: Optional[Path]

get_patch(patch_id)[source]¶

Get a single Patch by ID.

Parameters:: patch_id (int) – The ID of the patch.
Returns:: The desired patch.
Return type:: VariableDefinition

list_patches()[source]¶

List all Patches for this Variable Definition.

Return type:: list[VariableDefinition]

list_validity_periods()[source]¶

List all Validity Periods for this Variable Definition.

Return type:: list[VariableDefinition]

model_config: ClassVar[ConfigDict] = {'populate_by_name': True, 'protected_namespaces': (), 'str_strip_whitespace': True, 'use_enum_values': True, 'validate_assignment': True, 'validate_by_alias': True, 'validate_by_name': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context, /)¶

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Parameters:

self (BaseModel) – The BaseModel instance.
context (Any) – The context.

Return type:

None

publish_external()[source]¶

Publish this variable definition externally.

Return type:: VariableDefinition

publish_internal()[source]¶

Publish this variable definition internally.

Return type:: VariableDefinition

set_file_path(file_path)[source]¶

Set the file path where the variable definition has been written to for editing.

Return type:: None
Parameters:: file_path (Path | None)

to_dict()[source]¶

Return as dictionary.

Return type:: dict

to_file()[source]¶

Write this variable definition to file.

Return type:: VariableDefinition

update_draft(update_draft)[source]¶

Update this Variable Definition.

Variable definition must have status ‘DRAFT’.
Supply only the fields to be changed. Other fields will retain their current values.

Parameters:: update_draft (UpdateDraft) – The input with updated values.
Returns:: Updated Variable definition with all details.
Return type:: VariableDefinition

update_draft_from_file(file_path=None)[source]¶

Update this Variable Definition.

Will automatically read the relevant file pertaining to this variable definition. Can be overridden by specifying the file_path parameter.

Variable definition must have status ‘DRAFT’.
Supply only the fields to be changed. Other fields will retain their current values.

Parameters:: file_path (Optional[PathLike]) – Optionally specify the path to read from.
Returns:: Updated Variable definition with all details.
Return type:: VariableDefinition