ssb_timeseries.logging

Logging for time series library.

Logging every dataset read and write operation is useful for process monitoring and provides set level data lineage. Writes are natural drivers for an automated event based workflow.

It depends on the system context which role the time series library should play. Workflow control and monitoring should generally be managed by other components, but the time series system should play nicely with whatever these might be. In practical terms, that boils down to structured logging. That in turn allow generating events from the log information. In some system setups a backend database or an API could take care of logging and event generation. However, the timeseries library could also be used with only a file system or object storage as backend. In that case, the library is possibly the most central point close to the reads and writes.

Since the library requires proper configuration anyway, the natural solution is to let the configuration control the logging behaviour as well. This also leave it up to the application to make decisions on logging; in line with the general recommendation for code libraries, https://docs.python.org/3.10/howto/logging.html#library-config.

Legacy: If the configuration contains a non empty log_file field, logs are written there (and to console).

If logging field is specified, it takes presedence, and is used with logging.config.dictConfig. If both are empty, or dictConfig does not specify any log handlers, logging should be disabled.

TO DO: Add support for a ‘workflow’ log handler that puts all entries with log level INFO onto a queue. (A default/option for logging configuration may be sufficient?)

TODO: Logs should provide lineage at the dataset level, identifying which named processes reads and writes data. That requires log messages to include process names, UUIDs or other identifiers. The time series library may be able to get this information (from the scope of the calling code) by stack inspection. Otherwise, it must be passed in as parameters to read/write functions.

class EnterExitLog(name, logger)

Bases: object

Class supporting decorator to log on enter and exit.

Parameters:

  • name (str)

  • logger (Logger)

__enter__()

Before each workflow process step, do this.

Return type:

Self

__exit__(exc_type, exc_value, exc_traceback)

After each workflow process step, do this.

Return type:

None

__init__(name, logger)

Enter/exit template for workflow process.

Parameters:

  • name (str)

  • logger (Logger)

Return type:

None

exception LoggerNameNotDefined

Bases: Exception

The provided logger name is not defined in the accompanying configuration.

console_handler(format_string='%(name)s | %(levelname)s | %(asctime)s | %(message)s \\n')

Create handler for logging to console.

Return type:

StreamHandler

Parameters:

format_string (str)

file_handler(file, format_string='{"name": "%(name)s"; "level": %(levelname)s; "timestamp": %(asctime)s; "message": "%(message)s" }')

Create handler for logging to file.

Return type:

FileHandler

Parameters:

  • file (str)

  • format_string (str)

log_start_stop(func)

Log start and stop of decorated function.

Return type:

Callable

Parameters:

func (Callable)

set_up_logging_according_to_config(name, log_config)

Set up logging according to configuration.

Normally, this happens only in ssb_timeseries.__init__.py, which is run when the package is first imported. (For later imports, a cached instance will be retrieved.)

Return type:

Logger

Parameters:

  • name (str)

  • log_config (dict[str, Any])