Skip to content

vdk-structlog: support parse log level module and logger config granularity #2952

@DeltaMichael

Description

@DeltaMichael

Overview

As part of the run logs initiative, we should transfer functionality from the builtin logging config plugin to structlog.

Transfer this configuration to vdk-structlog and make it programmatic https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-core/src/vdk/internal/builtin_plugins/config/log_config.py#L127

Support adding loggers to the above configuration via _parse_log_level_module

https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-core/src/vdk/internal/builtin_plugins/config/log_config.py#L133

https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-core/src/vdk/internal/builtin_plugins/config/log_config.py#L28

Example for programmatic configuration.

Instead of

_LOGGERS = {
    "requests_kerberos": {"level": "INFO"},
    "requests_oauthlib": {"level": "INFO"},
    "urllib3": {"level": "INFO"},
    "vdk": {"level": vdk_logging_level},
}

we should do something like

logging.getLogger("requests_kerberos").setLevel("INFO")
# other stuff
logging.getLogger("vdk").setLevel(vdk_logging_level)

Acceptance Criteria

  1. Above functionality is transferred
  2. Support related config options https://github.com/vmware/versatile-data-kit/blob/main/projects/vdk-core/src/vdk/internal/builtin_plugins/config/log_config.py#L270
  3. Above functionality is sufficiently tested

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions