nudb_use.variables package

Subpackages

• nudb_use.variables.derive package
  • nudb_use.variables.derive.all_data_helpers module
    • enforce_datetime_s()
    • get_source_data()
    • join_variable_data()
  • nudb_use.variables.derive.derive_decorator module
    • WrappedDerive
    • WrappedDeriveJoinAllData
    • fillna_by_priority()
    • get_derive_function()
    • wrap_derive()
    • wrap_derive_join_all_data()
  • nudb_use.variables.derive.fullfoert module
    • gr_ergrunnskole_fullfoert()
    • uh_erbachelor_fullfoert()
    • uh_erdoktorgrad_fullfoert()
    • uh_erhoeyskolekandidat_fullfoert()
    • uh_ermaster_fullfoert()
    • vg_erstudiespess_fullfoert()
    • vg_ervgo_fullfoert()
    • vg_eryrkesfag_fullfoert()
  • nudb_use.variables.derive.fullfoert_foerste module
    • gr_foerste_fullfoert_dato()
    • uh_bachelor_foerste_fullfoert_dato()
    • uh_doktorgrad_foerste_fullfoert_dato()
    • uh_hoeyskolekandidat_foerste_fullfoert_dato()
    • uh_master_foerste_fullfoert_dato()
    • vg_foerste_fullfoert_dato()
    • vg_studiespess_foerste_fullfoert_dato()
    • vg_yrkesfag_foerste_fullfoert_dato()
  • nudb_use.variables.derive.klass_correspondences_and_variants module
    • fa_erfagskole_nokut_nus()
    • fa_studiepoeng_nus()
    • uh_gradmerke_nus()
    • uh_gruppering_nus()
    • uh_studiepoeng_nus()
    • utd_foreldet_kode_nus()
    • utd_isced2011_attainment_nus()
    • utd_isced2011_programmes_nus()
    • utd_isced2013_fagfelt_nus()
    • utd_klassetrinn_lav_hoey_nus()
    • utd_samle_eller_enkeltutd_nus()
    • utd_utdanningsprogram_nus()
    • utd_varighet_antall_mnd_nus()
    • vg_kompetanse_nus()
    • vg_kurstrinn_nus()
  • nudb_use.variables.derive.klass_labels module
    • fa_erfagskole_nokut_nus_label()
    • fa_studiepoeng_nus_label()
    • fuh_nett_eller_stedbasert_label()
    • fuh_opptaksgrunnlag_label()
    • fuh_utvekslingsland_label()
    • gro_elevstatus_label()
    • nus2000_label()
    • pers_bokommune_16aar_label()
    • pers_bokommune_nr_label()
    • pers_bydel_nr_label()
    • pers_foedeland_label()
    • pers_fra_land_label()
    • pers_gkrets_nr_label()
    • pers_inngrunn1_label()
    • pers_invkat_label()
    • pers_kjoenn_label()
    • pers_landbak3gen_label()
    • pers_statsborgerskap_label()
    • uh_campus_kommune_label()
    • uh_foerste_nus2000_label()
    • uh_gradmerke_nus_label()
    • uh_gruppering_nus_label()
    • uh_institusjon_id_label()
    • uh_statsborgerskap_inn_label()
    • uh_studgrunnlagsland_label()
    • uh_studiepoeng_nus_label()
    • uh_univ_eller_hoegskole_label()
    • utd_aktivitetsnivaa_heltid_deltid_label()
    • utd_datakilde_label()
    • utd_foreldet_kode_nus_label()
    • utd_foreldres_utdnivaa_16aar_label()
    • utd_fullfoertkode_label()
    • utd_hoeyeste_nus2000_label()
    • utd_isced2011_attainment_nus_label()
    • utd_isced2011_programmes_nus_label()
    • utd_isced2013_fagfelt_nus_label()
    • utd_klassetrinn_lav_hoey_nus_label()
    • utd_samle_eller_enkeltutd_nus_label()
    • utd_skolekom_label()
    • utd_studieland_label()
    • utd_utdanningsprogram_nus_label()
    • utd_utdanningstype_label()
    • utd_utveksling_label()
    • utd_varighet_antall_mnd_nus_label()
    • utd_viderutd_nettbasert_label()
    • vg_bevistype_label()
    • vg_fullfoertkode_detaljert_label()
    • vg_kompetanse_nus_label()
    • vg_kontraktstype_label()
    • vg_kurstrinn_nus_label()
    • vg_proevekandtype_label()
    • vg_rettstype_inntak_label()
    • vg_utdprogram_label()
    • vg_yrkes_og_studiekompetanse_label()
  • nudb_use.variables.derive.land module
    • utd_erutland()
  • nudb_use.variables.derive.nus_variants module
  • nudb_use.variables.derive.person module
    • pers_bokommune_16aar()
    • pers_bokommune_nr()
    • pers_bydel_nr()
    • pers_foedeland()
    • pers_foedselsdato()
    • pers_foerste_bosattdato()
    • pers_fra_land()
    • pers_gkrets_nr()
    • pers_innflyttingsdato()
    • pers_inngrunn1()
    • pers_invkat()
    • pers_kjoenn()
    • pers_landbak3gen()
    • pers_statsborgerskap()
  • nudb_use.variables.derive.person_idents module
    • snr_mrk()
  • nudb_use.variables.derive.registrert module
    • gr_ergrunnskole_registrering()
    • uh_erbachelor_registrering()
    • uh_erhoeyereutd_registrering()
    • uh_ermaster_registrering()
    • vg_erstudiespess_registrering()
    • vg_ervgo_erutdprogram_registrering()
    • vg_ervgo_registrering()
    • vg_eryrkesfag_registrering()
  • nudb_use.variables.derive.registrert_foerste module
    • gr_foerste_registrert_dato()
    • uh_bachelor_foerste_registrert_dato()
    • uh_foerste_nus2000()
    • uh_foerste_registrert_dato()
    • uh_master_foerste_registrert_dato()
    • vg_foerste_registrert_dato()
    • vg_foerste_registrert_erutdprogram_dato()
  • nudb_use.variables.derive.uh_univ_eller_hoegskole module
  • nudb_use.variables.derive.utd_foreldres_utdnivaa module
    • utd_foreldres_utdnivaa_16aar()
    • utd_foreldres_utdnivaa_16aar_nus2000()
    • utd_hoeyeste_far_nus2000()
    • utd_hoeyeste_mor_nus2000()
  • nudb_use.variables.derive.utd_hoeyeste module
    • utd_hoeyeste_nus2000()
    • utd_hoeyeste_rangering()
  • nudb_use.variables.derive.utd_skoleaar module
• nudb_use.variables.specific_vars package
  • nudb_use.variables.specific_vars.kommuner module
    • correct_kommune_single_values()
    • keep_only_valid_kommune_codes()
  • nudb_use.variables.specific_vars.snr module
    • generate_uuid_for_snr_with_fnr_catalog()
    • generate_uuid_for_snr_with_fnr_col()
    • update_snr_with_snrkat()

nudb_use.variables.checks module

Validation utilities for ensuring variable schemas match expectations.

check_cols_against_klass_codelists(df, col_codelist=None)

Validate DataFrame values against KLASS codelists.

Return type:

None

Parameters:

• df (DataFrame)

• col_codelist (dict[str, list[str] | dict[str, str]] | None)

check_column_presence(df, dataset_name=None, check_for=None, raise_errors=True)

Validate columns against config or a supplied list.

Return type:

list[Exception]

Parameters:

• df (DataFrame)

• dataset_name (str | None)

• check_for (None | list[str])

• raise_errors (bool)

identify_cols_not_in_keep_drop_in_paths(paths, cols_keep, cols_drop, raise_error_found=False)

Identify columns present in data files that are missing from keep/drop lists.

Return type:

set[str]

Parameters:

• paths (list[Path])

• cols_keep (list[str])

• cols_drop (list[str])

• raise_error_found (bool)

pyarrow_columns_from_metadata(path)

Read column names from a Parquet file via metadata only.

Return type:

list[str]

Parameters:

path (str | Path)

nudb_use.variables.cleanup module

Utilities for reorganizing and trimming NUDB datasets.

move_col_after_col(df, col_anchor, col_move_after)

Move a specified column in a DataFrame to immediately follow another specified column.

Parameters:

• df (DataFrame) – Input pandas DataFrame.

• col_anchor (str) – Name of the column after which the specified column will be moved.

• col_move_after (str) – Name of the column to move.

Returns:

New DataFrame with the specified column moved to follow the anchor column.

Return type:

DataFrame

move_content_from_col_to(df, from_col, to_col)

Fill empty values (NA) in one column with values from another column.

Parameters:

• df (DataFrame) – DataFrame

• from_col (str) – Column where information is taken.

• to_col (str) – Column where information is moved to.

Returns:

DataFrame with values filled out.

Return type:

DataFrame

sort_all_values(df, priority_cols=None)

Sort the dataset semi-deterministically, by using all columns, but prioritizing some.

Notes

• If the people analyzing our data do not sort on all the columns…

• And they use a semi-random “keep first of duplicates strategy”…

• They will be dependant on the order of values we are sending them to reproduce the same aggregations across versions…

• So providing a semi-deterministic sort order of values might be a good thing?

• Still, people removing duplicates dependent on a semi-random strategy, should strive to make it less random…

Parameters:

• df (DataFrame) – The dataframe to be sorted.

• priority_cols (list[str] | None) – A list of columns to weigh first in sorting. Default is (if None): [“utd_skoleaar_start”, “nus2000”, “utd_skolekom”,]

Returns:

A pandas dataframe sorted.

Return type:

DataFrame

Raises:

KeyError – If some of the priority_cols are not in the dataframe sent in.