Skip to content

Commit f30a1ae

Browse files
author
Dilyan Marinov
committed
vdk-structlog: add default logging format values
Why? For users who do not wish to configure each structlog option separately, introduce presets for different cases. What? Introduce CLOUD and LOCAL presets. The CLOUD preset hardcodes the following options ```sh export VDK_STRUCTLOG_FORMAT=console export VDK_STRUCTLOG_CONSOLE_LOG_PATTERN="%(asctime)s [VDK] %(vdk_job_name)s [%(levelname)-5.5s] %(name)-30.30s %(filename)20.20s:%(lineno)-4.4s %(funcName)-16.16s[id:%(attempt_id)s]- %(message)s" ``` LOCAL just uses the defaults set in the configuration code Refactor structlog plugin to eliminate repeating code Remove syslog config tests Fix bug with vdk fields crashing logging for custom format Add tests for vdk fields with custom format How was this tested? Functional tests CI What kind of change is this? Feature/non-breaking Signed-off-by: Dilyan Marinov <mdilyan@vmware.com>
1 parent f1c24b5 commit f30a1ae

File tree

6 files changed

+219
-287
lines changed

6 files changed

+219
-287
lines changed

‎projects/vdk-plugins/vdk-structlog/README.md‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,16 +21,18 @@ pip install vdk-structlog
2121

2222
(`vdk config-help` is a useful command to browse all config options of your installation of vdk)
2323

24-
| Name | Description | Example Value | Possible Values |
25-
|----------------------------|-------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
26-
| logging_metadata | Configure the metadata that will be output along with the log message | "timestamp, level, logger_name, file_name, vdk_job_name | Any combination of the following: "timestamp, level, logger_name, file_name, line_number, function_name, vdk_job_name, vdk_step_name, vdk_step_type". Can be expanded by extra params and bound key-value pairs. See the bound logger examples for more information |
27-
| logging_format | Configure the logging output format. Available formats: json, console | "console" | "console", "json", "ltsv" |
28-
| custom_console_log_pattern | Custom format string for console logging, applied only when`logging_format` is 'console'. Overrides `logging_metadata`. | "%(asctime)s %(name)-12s %(levelname)-8s %(message)s" | Any valid Python logging format string |
29-
| log_level_module | Configure the log level of different Python modules separately | "a.b.c=INFO;foo.bar=ERROR" | Semicolon-separated list of pairs of Python module paths and log level labels |
30-
| syslog_host | Syslog host to which logs are emitted | "syslog.vmware.com" | Any valid host name |
31-
| syslog_port | Syslog port used to emit logs | 514 | Any valid port number |
32-
| syslog_protocol | Protocol used to emit logs | "UDP" | "TCP", "UDP" |
33-
| syslog_enabled | Enable/disable syslog | "True" | "True", "False" |
24+
| Name | Description | Example Value | Possible Values |
25+
|-------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
26+
| use_structlog | Use the structlog logging config instead of using the one in vdk-core | "True" | "True", "False" |
27+
| structlog_metadata | Configure the metadata that will be output along with the log message | "timestamp, level, logger_name, file_name, vdk_job_name | Any combination of the following: "timestamp, level, logger_name, file_name, line_number, function_name, vdk_job_name, vdk_step_name, vdk_step_type". Can be expanded by extra params and bound key-value pairs. See the bound logger examples for more information |
28+
| structlog_format | Configure the logging output format. Available formats: json, console, ltsv | "console" | "console", "json", "ltsv" |
29+
| structlog_console_log_pattern | Custom format string for console logging, applied only when`logging_format` is 'console'. Overrides `logging_metadata`. Note: For config.ini, %-signs should be escaped by doubling, e.g. %(asctime)s should become %%(asctime)s | "%(asctime)s %(name)-12s %(levelname)-8s %(message)s" | Any valid Python logging format string |
30+
| structlog_config_preset | Choose a configuration preset. Available presets: LOCAL, CLOUD | "CLOUD" | "console", "json", "ltsv" |
31+
| log_level_module | Configure the log level of different Python modules separately | "a.b.c=INFO;foo.bar=ERROR" | Semicolon-separated list of pairs of Python module paths and log level labels |
32+
| syslog_host | Syslog host to which logs are emitted | "syslog.vmware.com" | Any valid host name |
33+
| syslog_port | Syslog port used to emit logs | 514 | Any valid port number |
34+
| syslog_protocol | Protocol used to emit logs | "UDP" | "TCP", "UDP" |
35+
| syslog_enabled | Enable/disable syslog | "True" | "True", "False" |
3436

3537
### Example: Configure Custom Console Format
3638

‎projects/vdk-plugins/vdk-structlog/src/vdk/plugin/structlog/constants.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,14 +3,17 @@
33
import logging
44
import socket
55

6+
STRUCTLOG_USE_STRUCTLOG = "use_structlog"
67
STRUCTLOG_LOGGING_METADATA_KEY = "structlog_metadata"
78
STRUCTLOG_LOGGING_FORMAT_KEY = "structlog_format"
89
STRUCTLOG_CONSOLE_LOG_PATTERN = "structlog_console_custom_format"
10+
STRUCTLOG_CONFIG_PRESET = "structlog_config_preset"
911

1012
STRUCTLOG_LOGGING_FORMAT_POSSIBLE_VALUES = ["console", "json", "ltsv"]
1113
STRUCTLOG_LOGGING_FORMAT_DEFAULT = "console"
1214

1315
STRUCTLOG_LOGGING_METADATA_JOB = {
16+
"attempt_id": "%(attempt_id)s",
1417
"vdk_job_name": "%(vdk_job_name)s",
1518
"vdk_step_name": "%(vdk_step_name)s",
1619
"vdk_step_type": "%(vdk_step_type)s",
@@ -44,6 +47,7 @@
4447
}
4548

4649
CONSOLE_STRUCTLOG_LOGGING_METADATA_JOB = {
50+
"attempt_id": "[id:%(attempt_id)s]",
4751
"vdk_job_name": "%(vdk_job_name)s",
4852
"vdk_step_name": "%(vdk_step_name)s",
4953
"vdk_step_type": "%(vdk_step_type)s",
@@ -78,3 +82,8 @@
7882
DEFAULT_SYSLOG_ENABLED = False
7983

8084
SYSLOG_PROTOCOLS = {"UDP": socket.SOCK_DGRAM, "TCP": socket.SOCK_STREAM}
85+
86+
DETAILED_LOGGING_FORMAT = (
87+
"%(asctime)s [VDK] %(vdk_job_name)s [%(levelname)-5.5s] %(name)-30.30s %(filename)20.20s:%("
88+
"lineno)-4.4s %(funcName)-16.16s[id:%(attempt_id)s]- %(message)s"
89+
)

0 commit comments

Comments
 (0)