Containerize Uyuni #3451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

moio wants to merge 82 commits into master from containers

Contributor

moio commented Mar 19, 2021

What does this PR change?

This is an experiment to deliver Uyuni as containers.

See the official HackWeek page or the Development Wiki page for more information.

GUI diff

No difference.

DONE

Documentation

No documentation needed: add explanation. This can't be used if there is a GUI diff
No documentation needed: only internal and user invisible changes
Documentation issue was created: Link for SUSE Manager contributors, Link for community contributors.
(OPTIONAL) Documentation PR
DONE

Test coverage

No tests: add explanation
No tests: already covered
Unit tests were added
Cucumber tests were added
DONE

Links

official HackWeek page

DONE

Changelogs

If you don't need a changelog check, please mark this checkbox:

No changelog needed

If you uncheck the checkbox after the PR is created, you will need to re-run changelog_test (see below)

Re-run a test

If you need to re-run a test, please mark the related checkbox, it will be unchecked automatically once it has re-run:

Re-run test "changelog_test"
Re-run test "backend_unittests_pgsql"
Re-run test "java_pgsql_tests"
Re-run test "schema_migration_test_oracle"
Re-run test "schema_migration_test_pgsql"
Re-run test "susemanager_unittests"
Re-run test "javascript_lint"
Re-run test "spacecmd_unittests"

github-actions bot added the xmlrpc label

agraul reviewed

View reviewed changes

containers/create-proxy-config.py Outdated

+              organization = args.organization
+              organization_unit = args.organization_unit
+              email = args.email
+              cname = '.'.join(fqdn.split('.')[-2:])

Member

agraul Mar 25, 2021

I think this should be the following. FQDN can be specified with a trailing dot for the dns root, would give you canonical names such as com. instead of example.com for this.example.com.

Suggested change

      
            cname = '.'.join(fqdn.split('.')[-2:])
          
            cname = '.'.join(fqdn.rstrip(".").split('.')[-2:])

Shirocco88 suggested changes

View reviewed changes

Contributor

Shirocco88 left a comment

Please update from "opensuse/leap:15.2" to "opensuse/leap:15.3"

containers/proxy-main/Dockerfile Outdated

		@@ -0,0 +1,27 @@
		# rebuild with: docker build -t proxy .

		FROM registry.opensuse.org/opensuse/leap:15.2

Contributor

Shirocco88 Sep 11, 2021

FROM registry.opensuse.org/opensuse/leap:15.3

containers/proxy-salt-broker/Dockerfile Outdated

		@@ -0,0 +1,19 @@
		# rebuild with: docker build -t proxy .

		FROM registry.opensuse.org/opensuse/leap:15.2

Contributor

Shirocco88 Sep 11, 2021

FROM registry.opensuse.org/opensuse/leap:15.3

containers/proxy-squid/Dockerfile Outdated

		@@ -0,0 +1,18 @@
		# rebuild with: docker build -t proxy .

		FROM registry.opensuse.org/opensuse/leap:15.2

Contributor

Shirocco88 Sep 11, 2021

FROM registry.opensuse.org/opensuse/leap:15.3

containers/proxy/Dockerfile Outdated

		@@ -0,0 +1,22 @@
		# rebuild with: docker build -t proxy .

		FROM registry.opensuse.org/opensuse/leap:15.2

Contributor

Shirocco88 Sep 11, 2021

FROM registry.opensuse.org/opensuse/leap:15.3

containers/srv/uyuni-container/Dockerfile

		@@ -0,0 +1,44 @@
		FROM opensuse/leap:15.2

Contributor

Shirocco88 Sep 11, 2021

FROM opensuse/leap:15.3

containers/srv/uyuni-container/todo.md


		podman run --pod hub-opensuse152 -ti --name hub-opensuse152-server localhost/uyuni

		podman generate kube hub-opensuse152 > hub-opensuse152.yml

Contributor

Shirocco88 Sep 11, 2021

podman generate kube hub-opensuse153 > hub-opensuse153.yml

containers/srv/uyuni-container/todo.md

+              open file and remove the last empty lines
+              > change image to be load from localhost:5000/uyuni
+              podman pod rm hub-opensuse152

Contributor

Shirocco88 Sep 11, 2021

podman pod rm hub-opensuse153

containers/srv/uyuni-container/todo.md



		## deployment and management
		minikube kubectl -- delete po hub-opensuse152

Contributor

Shirocco88 Sep 11, 2021

minikube kubectl -- delete po hub-opensuse153

containers/srv/uyuni-container/todo.md

+              ## deployment and management
+              minikube kubectl -- delete po hub-opensuse152
+              minikube kubectl -- create -f hub-opensuse152.yml

Contributor

Shirocco88 Sep 11, 2021

minikube kubectl -- create -f hub-opensuse153.yml

containers/srv/uyuni-container/todo.md


		minikube kubectl -- get po -A

		minikube kubectl -- describe pod hub-opensuse152

Contributor

Shirocco88 Sep 11, 2021

minikube kubectl -- describe pod hub-opensuse153

Contributor Author

moio commented Sep 15, 2021

Thanks for your comments @Shirocco88, we hope to be back on this topic soon.

moio force-pushed the containers branch from 566119d to 286fb55 Compare

November 17, 2021 16:15

moio requested a review from a team as a code owner

November 17, 2021 16:15

github-actions bot added API java java_lint_checkstyle java_pgsql_tests old-ui labels

vzhestkov reviewed

View reviewed changes

Contributor

vzhestkov left a comment

Suggestions appended

containers/create-proxy-config.py

Comment on lines +73 to +75

+              os.mkdir(config_path + "/salt")
+              os.mkdir(config_path + "/salt/pki")
+              os.mkdir(config_path + "/salt/pki/minion")

Contributor

vzhestkov Nov 18, 2021

why not to use os.makedirs instead?

Suggested change

      
            os.mkdir(config_path + "/salt")
          
            os.mkdir(config_path + "/salt/pki")
          
            os.mkdir(config_path + "/salt/pki/minion")
          
            os.makedirs("/salt/pki/minion")

containers/create-proxy-config.py

Comment on lines +92 to +93

		os.mkdir(config_path + "/sysconfig")
		os.mkdir(config_path + "/sysconfig/rhn")

Contributor

vzhestkov Nov 18, 2021

the same as before:

Suggested change

      
            os.mkdir(config_path + "/sysconfig")
          
            os.mkdir(config_path + "/sysconfig/rhn")
          
            os.makedirs(config_path + "/sysconfig/rhn")

moio and others added 18 commits

November 23, 2021 10:59


          Initial Proxy Dockerfile added

6fb3c0d


          add salt-minion to container

da16d89


          add runner script for easier starting of the conatainer

2fa0a6f


          initial startup script

22ff708


          container start script

07dd6bb


          Fix typo (UYUNI_ACTIVATION_KEY)

7d5a72b


          update

cc446b6


          use salt to configure the container

bc2f838


          change variable name

2ee0fc0


          Collect proxy config files on the server

58dce02


          Remove redundant requires

c1c110b

SLS ordering is respected within a single file


          Refactoring: move config file content to same file

dcd3832


          use salt keys in config dir and store self generated for re-using

e3b7cc9


          make runner script podman ready

b9a5918


          Revert "Remove redundant requires"

42f0464

It is not redundant as we do not want to run that state
when others failed before. Using this it fail faster
and do not try to start things which are not ready or not configured

This reverts commit 71916ce.


          Add activation of Proxy

13766a4


          for development install vim and less

fc2e4fd


          handle certs

f54388f

mcalmer and others added 22 commits

November 23, 2021 10:59


          Split into 3 containers and start them as POD

43bf547


          force same uid/gid in all containers

92872a3


          more info

04572c7

Signed-off-by: Ricardo Mateus <[email protected]>


          server integration with k3s

f127b7a

Signed-off-by: Ricardo Mateus <[email protected]>


          fix for python3

16b8274


          reduce memory usage

b605809

Signed-off-by: Ricardo Mateus <[email protected]>


          rmateus todo

53559a6

Signed-off-by: Ricardo Mateus <[email protected]>


          Containers and Helm chart to run Postgres in k3s. First version

f3d4036


          Do not use an init container


          Add secret to securely store credentials

798ee82


          Update proxy base images

Signed-off-by: Silvio Moioli <[email protected]>


          Update gids after base image change

c959006

Signed-off-by: Silvio Moioli <[email protected]>


          Update building instructions

08548f8

Signed-off-by: Silvio Moioli <[email protected]>


          Dropped single-container Proxy

24cbfab

Signed-off-by: Silvio Moioli <[email protected]>


          Drop unused activation key param

6cfdcb6

Signed-off-by: Silvio Moioli <[email protected]>


          Drop unused machine id param

feeb92e

Signed-off-by: Silvio Moioli <[email protected]>


          Change uid/gid enforcement for squid user

c0c0502

Signed-off-by: Silvio Moioli <[email protected]>


          drop salt user setting, we dropped salt-master from the image before

cfedbea

Signed-off-by: Silvio Moioli <[email protected]>


          Pass environment variables in bulk

84c2bae

Signed-off-by: Silvio Moioli <[email protected]>


          Introduce registry in workflow

ddb0411

Signed-off-by: Silvio Moioli <[email protected]>


          HACK: work around avahi

1ca5325

Signed-off-by: Silvio Moioli <[email protected]>


          drop duplicated files

d4374f9

Signed-off-by: Silvio Moioli <[email protected]>

moio force-pushed the containers branch from 27f76d6 to d4374f9 Compare

November 23, 2021 10:52

moio added 3 commits

November 23, 2021 11:53


          add hacking guide

1fe8508

Signed-off-by: Silvio Moioli <[email protected]>


          add utility script to kill

2803c80

Signed-off-by: Silvio Moioli <[email protected]>


          add build script

1b85274

Signed-off-by: Silvio Moioli <[email protected]>

SchoolGuy reviewed

View reviewed changes

Contributor

SchoolGuy left a comment

I did my best looking over the PR and found a few things I am unsure about. In general I like the shape this is already!

containers/killProxyPod.sh

		@@ -0,0 +1,3 @@
		#!/bin/bash

		podman ps \| grep proxy- \| awk '{print $1}' \| xargs -l1 podman kill

Contributor

SchoolGuy Nov 23, 2021

I find this grep a bit too wide. Do we have any way to specify a more precise filter? Maybe proxy-uyuni? Or we use the Label we add to the container?

For Podman the filter can be added like this: podman ps -f KEY=VALUE

Docs: https://docs.podman.io/en/latest/markdown/podman-ps.1.html#filter-f

Member

agraul Nov 23, 2021 •

edited

Loading

The grep could also be removed in favor of awk '/proxy-/ {print $1}'. See number 49 on https://catonmat.net/awk-one-liners-explained-part-three

Contributor

SchoolGuy Nov 23, 2021

Yeah or we move it in the filter expression of podman. Both is possible.

containers/postgres/helm/templates/statefulset.yaml

+                      {{- end }}
+                      resources:
+                        requests:
+                          storage: {{ .Values.storageSize }}

Contributor

SchoolGuy Nov 23, 2021

Not all storageclasses support that you resize them. I would suggest we add an educated guess to the docs about the size this will have after what time/configuration.

java/code/src/com/redhat/rhn/frontend/xmlrpc/proxy/ProxyHandler.java

@@ @@ -32,7 +32,6 @@ @@
               import com.redhat.rhn.frontend.xmlrpc.ProxyNotActivatedException;
               import com.redhat.rhn.frontend.xmlrpc.ProxySystemIsSatelliteException;
               import com.redhat.rhn.frontend.xmlrpc.system.XmlRpcSystemHelper;
-              import com.redhat.rhn.manager.entitlement.EntitlementManager;

Contributor

SchoolGuy Nov 23, 2021

I believe that this Java change is correct but I would like to know the reason for it. The commit message is a bit vague about this change.


          Squid dep is taken care by #4480

e9e6478

Signed-off-by: Silvio Moioli <[email protected]>

Contributor Author

moio commented Dec 16, 2021

While code and approaches tried out in this PR are good and will continue to be useful, I am closing this now in favor of more specialized PRs in-progress and upcoming, including in particular #4526, as the bulk of the work will progress there.

Many thanks to all authors and commenters, see you in the other PRs!

moio closed this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API java_lint_checkstyle java_pgsql_tests java old-ui xmlrpc