vdk-structlog: add default logging format values

Why?

There should be separate default configs for local
and cloud runs

What?

Refactor structlog plugin to eliminate repeating code
Remove syslog config tests
Fix bug with vdk fields crashing logging for custom format
Add tests for vdk fields with custom format
Add default logging config to the data job base image
Note: The default config will not have effect unless vdk-structlog is installed

How was this tested?

Functional tests
CI

What kind of change is this?

Feature/non-breaking

Signed-off-by: Dilyan Marinov <mdilyan@vmware.com>

Author: Dilyan Marinov

projects/control-service/projects/job-base-image/Dockerfile-data-job-base

@@ -11,9 +11,9 @@ WORKDIR /job
 # Install native dependencies so that requirements in requirements.txt can be installed
 # some (like openssl) should be pre-installed in the base image but let's be explicit
 RUN set -ex \
-       && apt-get update \
-       && apt-get -y install --no-install-recommends \
-              build-essential openssl g++
+      && apt-get update \
+      && apt-get -y install --no-install-recommends \
+      build-essential openssl g++
 
 # Install the native dependencies necessary for cx_Oracle python library
 # See https://oracle.github.io/odpi/doc/installation.html#linux
@@ -27,6 +27,11 @@ RUN set -ex \
       && ldconfig \
       && apt-get purge -y --auto-remove curl unzip
 
+# Setup default logging with vdk-structlog
+ENV VDK_STRUCTLOG_FORMAT="console"
+ENV VDK_STRUCTLOG_CONSOLE_CUSTOM_FORMAT="%(asctime)s [VDK] %(vdk_job_name)s [%(levelname)-5.5s] %(name)-30.30s %(filename)20.20s:%(lineno)-4.4s %(funcName)-16.16s[id:%(attempt_id)s] - %(message)s"
+
+
 # libaio1 - LGPL-2.1+ https://developer.puri.sm/licenses/Librem5/Birch/libaio1/copyright
 # curl - MIT/X* modified https://curl.se/docs/copyright.html
 # unzip - MIT https://github.com/vipsoft/Unzip/blob/master/LICENSE

projects/vdk-plugins/vdk-structlog/src/vdk/plugin/structlog/constants.py

@@ -11,6 +11,7 @@
 STRUCTLOG_LOGGING_FORMAT_DEFAULT = "console"
 
 STRUCTLOG_LOGGING_METADATA_JOB = {
+    "attempt_id": "%(attempt_id)s",
     "vdk_job_name": "%(vdk_job_name)s",
     "vdk_step_name": "%(vdk_step_name)s",
     "vdk_step_type": "%(vdk_step_type)s",
@@ -44,6 +45,7 @@
 }
 
 CONSOLE_STRUCTLOG_LOGGING_METADATA_JOB = {
+    "attempt_id": "[id:%(attempt_id)s]",
     "vdk_job_name": "%(vdk_job_name)s",
     "vdk_step_name": "%(vdk_step_name)s",
     "vdk_step_type": "%(vdk_step_type)s",