Skip to content

TLS 1.3 / Modern profile tests #29611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open

Conversation

jacobsee
Copy link
Member

@jacobsee jacobsee commented Mar 19, 2025

This PR:

  • Removes a skip on an existing TLS test - it appears to have been a setup issue
  • Adds a test for ensuring that components are properly listening for only the expected TLS version or higher.

cc: @dusk125

…sue. Add TLS modern profile test to APIServer. Add (WIP) TLS 1.3 test to etcd.
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 19, 2025
@openshift-ci openshift-ci bot requested review from deads2k and Elbehery March 19, 2025 23:54
… test as it is just one additional check with otherwise entirely common test needs.
@jacobsee jacobsee changed the title [WIP] TLS 1.3 / Modern profile tests TLS 1.3 / Modern profile tests Mar 20, 2025
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 20, 2025
@jacobsee
Copy link
Member Author

jacobsee commented Mar 20, 2025

Supports OCPBUGS-37706, OCPSTRAT-1364

Copy link

openshift-trt bot commented Mar 20, 2025

Job Failure Risk Analysis for sha: 6c00d75

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-gcp-ovn-etcd-scaling High
[sig-architecture] platform pods in ns/openshift-etcd should not exit an excessive amount of times
This test has passed 100.00% of 1 runs on jobs [periodic-ci-openshift-release-master-nightly-4.19-e2e-gcp-ovn-etcd-scaling periodic-ci-openshift-release-master-nightly-4.18-e2e-gcp-ovn-etcd-scaling] in the last 14 days.

@dusk125
Copy link
Contributor

dusk125 commented Mar 20, 2025

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 20, 2025
@jacobsee
Copy link
Member Author

/retest-required

Copy link

openshift-trt bot commented Mar 21, 2025

Job Failure Risk Analysis for sha: ff62120

Job Name Failure Risk
pull-ci-openshift-origin-main-4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback MissingData
pull-ci-openshift-origin-main-e2e-aws-disruptive Medium
[sig-node] static pods should start after being created
Potential external regression detected for High Risk Test analysis
pull-ci-openshift-origin-main-e2e-azure-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 0.00% of 1 runs on release 4.19 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:azure SecurityMode:default Topology:ha Upgrade:none] in the last week.
---
[bz-kube-storage-version-migrator] clusteroperator/kube-storage-version-migrator should not change condition/Available
This test has passed 0.00% of 1 runs on release 4.19 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:azure SecurityMode:default Topology:ha Upgrade:none] in the last week.
---
[bz-openshift-apiserver] clusteroperator/openshift-apiserver should not change condition/Available
This test has passed 0.00% of 1 runs on release 4.19 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:azure SecurityMode:default Topology:ha Upgrade:none] in the last week.
pull-ci-openshift-origin-main-e2e-gcp-disruptive Medium
[sig-node] static pods should start after being created
Potential external regression detected for High Risk Test analysis
---
[sig-arch] events should not repeat pathologically for ns/openshift-kube-apiserver-operator
Potential external regression detected for High Risk Test analysis
pull-ci-openshift-origin-main-e2e-gcp-ovn-etcd-scaling Medium
[bz-etcd][invariant] alert/etcdMembersDown should not be at or above info
Potential external regression detected for High Risk Test analysis
---
[sig-architecture] platform pods in ns/openshift-etcd should not exit an excessive amount of times
Potential external regression detected for High Risk Test analysis

@p0lyn0mial
Copy link
Contributor

which jobs run the new test ? i have checked metal-ipi-serial, aws-ovn-microshift-serial and aws-ovn-serial but didn't find the new test on the list.

…s causing TLS 1.2 connections to be rejected.
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Mar 21, 2025
Copy link

openshift-trt bot commented Mar 22, 2025

Job Failure Risk Analysis for sha: b6d72fd

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-gcp-disruptive Medium
[sig-node] static pods should start after being created
Potential external regression detected for High Risk Test analysis

@p0lyn0mial
Copy link
Contributor

which jobs run the new test ? i have checked metal-ipi-serial, aws-ovn-microshift-serial and aws-ovn-serial but didn't find the new test on the list.

@jacobsee I think you need to run make update to actually "add" the test to the list of tests to be executed. For example: https://github.com/openshift/origin/pull/29191/commits

@jacobsee
Copy link
Member Author

@p0lyn0mial Thanks! I hadn't seen that. It's in there now, and I've realized that it should probably be marked [slow] as well. To your earlier comment, do we now still need to pick jobs to run this? This is my first brush with the origin tests, so I'm not sure what all is needed to get a new one plugged in correctly.

Copy link

openshift-trt bot commented Mar 25, 2025

Job Failure Risk Analysis for sha: 39e5f22

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 50.00% of 2 runs on release 4.19 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:none] in the last week.
pull-ci-openshift-origin-main-e2e-gcp-disruptive High
[bz-Monitoring] clusteroperator/monitoring should not change condition/Degraded
This test has passed 99.30% of 5292 runs on release 4.19 [Overall] in the last week.
---
[bz-Monitoring] clusteroperator/monitoring should not change condition/Available
This test has passed 99.45% of 5292 runs on release 4.19 [Overall] in the last week.

@p0lyn0mial
Copy link
Contributor

To your earlier comment, do we now still need to pick jobs to run this?

it looks like you could use /payload-job-with-prs to test your new test with the o/k PR, for example:

/payload-job-with-prs periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn openshift/kubernetes#2135

xref: https://docs.ci.openshift.org/docs/release-oversight/pull-request-testing/#payload-job

we just need to find a periodic job that will run your test. maybe some serial test from ?

so, overall:

  1. we add a new test to o/o. (this PR)
  2. we use the/payload-job-with-prs command to run the test with the o/k PR
  3. we make sure the test is green
  4. we merge the o/k PR
  5. we merge this PR

Copy link
Contributor

openshift-ci bot commented Mar 25, 2025

@p0lyn0mial: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.19-e2e-aws-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/745c1ee0-0985-11f0-8a93-335e1efa7bdb-0

@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label May 29, 2025
Copy link

openshift-trt bot commented May 30, 2025

Job Failure Risk Analysis for sha: f132b67

Job Name Failure Risk
pull-ci-openshift-origin-main-4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback High
operator conditions network
This test has passed 99.00% of 3807 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-aws-disruptive Medium
[sig-node] node-lifecycle detects unexpected not ready node
Potential external regression detected for High Risk Test analysis

Open Bugs
node-lifecycle detects unexpected not ready node failing on azure serial and upgrade jobs
---
[sig-node] static pods should start after being created
Potential external regression detected for High Risk Test analysis
---
[bz-Etcd] clusteroperator/etcd should not change condition/Available
Potential external regression detected for High Risk Test analysis
pull-ci-openshift-origin-main-e2e-azure-ovn-upgrade IncompleteTests
Tests for this run (2125) are below the historical average (2876): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)
pull-ci-openshift-origin-main-e2e-gcp-disruptive Medium
[sig-node] static pods should start after being created
Potential external regression detected for High Risk Test analysis
---
[bz-Etcd] clusteroperator/etcd should not change condition/Available
Potential external regression detected for High Risk Test analysis

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: f132b67

Job Name New Test Risk
pull-ci-openshift-origin-main-e2e-aws-ovn-fips High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-dualstack High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-dualstack-local-gateway High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-virtualmedia High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit

New tests seen in this PR at sha: f132b67

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 18, Pass: 12, Fail: 6, Flake: 0]

skip the test on IPv6 or DualStack clusters instead.
@jacobsee
Copy link
Member Author

jacobsee commented Jun 3, 2025

/retest-required

Copy link

openshift-trt bot commented Jun 3, 2025

Job Failure Risk Analysis for sha: 28f893b

Job Name Failure Risk
pull-ci-openshift-origin-main-4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback MissingData
pull-ci-openshift-origin-main-e2e-aws-disruptive High
[sig-arch] events should not repeat pathologically for ns/openshift-kube-apiserver-operator
This test has passed 99.86% of 3666 runs on release 4.20 [Overall] in the last week.
---
[sig-node] static pods should start after being created
This test has passed 99.13% of 3666 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-azure-ovn-upgrade IncompleteTests
Tests for this run (107) are below the historical average (3087): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: 28f893b

Job Name New Test Risk
pull-ci-openshift-origin-main-e2e-aws High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-fips High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node-upgrade High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-proxy High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-azure High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-dualstack High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-dualstack-local-gateway High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that was not present in all runs against the current commit, and also failed 1 time(s).
pull-ci-openshift-origin-main-e2e-metal-ipi-virtualmedia High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-openstack-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit

New tests seen in this PR at sha: 28f893b

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 23, Pass: 1, Fail: 22, Flake: 0]

Copy link

openshift-trt bot commented Jun 3, 2025

Job Failure Risk Analysis for sha: 28f893b

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-disruptive High
[sig-arch] events should not repeat pathologically for ns/openshift-kube-apiserver-operator
This test has passed 99.86% of 3664 runs on release 4.20 [Overall] in the last week.
---
[sig-node] static pods should start after being created
This test has passed 99.13% of 3664 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-azure-ovn-upgrade IncompleteTests
Tests for this run (107) are below the historical average (3087): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New Test Risks for sha: 28f893b

Job Name New Test Risk
pull-ci-openshift-origin-main-e2e-aws High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-cgroupsv2 High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-edge-zones High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-fips High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node-upgrade High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-aws-proxy High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-azure High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-gcp-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-dualstack High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-dualstack-local-gateway High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-ovn-ipv6 High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-metal-ipi-virtualmedia High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-openstack-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-e2e-vsphere-ovn-upi High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 2 time(s) against the current commit
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn High - "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" is a new test that failed 1 time(s) against the current commit

New tests seen in this PR at sha: 28f893b

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 23, Pass: 0, Fail: 23, Flake: 0]

@dusk125
Copy link
Contributor

dusk125 commented Jun 3, 2025

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jun 3, 2025
going through the route. Add retry with backoff to port-forward. Fix up
IP family detection.
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Jun 3, 2025
return &config
}

func getIPFamilyForCluster(client exutil.CLI, namespace string) IPFamily {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the code related to getting the current IP family exactly the same as the one used in the existing network tests?

If so, maybe we could move it to a common place and reuse it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ping ^ (we could open a new PR to refactor)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's mostly the same, but it adds a security context to let the check pod run in restricted contexts

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That and the port forwarder are probably both reusable. Where should they live?

Copy link

openshift-trt bot commented Jun 4, 2025

Job Failure Risk Analysis for sha: caeab10

Job Name Failure Risk
pull-ci-openshift-origin-main-4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback MissingData
pull-ci-openshift-origin-main-e2e-gcp-disruptive High
[sig-etcd] etcd should not log excessive took too long messages
This test has passed 98.71% of 3561 runs on release 4.20 [Overall] in the last week.

Open Bugs
Component Readiness: [Etcd] [Other] test regressed (excessive took too long messages)
---
[bz-Monitoring] clusteroperator/monitoring should not change condition/Available
This test has passed 98.52% of 3643 runs on release 4.20 [Overall] in the last week.
---
[sig-arch][Late] operators should not create watch channels very often
This test has passed 99.49% of 3345 runs on release 4.20 [Overall] in the last week.

Open Bugs
Component Readiness Shows Old Test Name For Renamed Tests
ResilientWatchCacheInitialization (Re)enablement - operator watch counts from component readiness
---
[bz-Monitoring] clusteroperator/monitoring should not change condition/Degraded
This test has passed 98.41% of 3643 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-gcp-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 0.00% of 1 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:gcp SecurityMode:default Topology:ha Upgrade:none] in the last week.
---
[bz-kube-storage-version-migrator] clusteroperator/kube-storage-version-migrator should not change condition/Available
This test has passed 69.70% of 3643 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (98) are below the historical average (3234): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: caeab10

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 15, Pass: 15, Fail: 0, Flake: 0]

Copy link

openshift-trt bot commented Jun 4, 2025

Job Failure Risk Analysis for sha: caeab10

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-gcp-disruptive High
[sig-etcd] etcd should not log excessive took too long messages
This test has passed 98.71% of 3561 runs on release 4.20 [Overall] in the last week.

Open Bugs
Component Readiness: [Etcd] [Other] test regressed (excessive took too long messages)
---
[bz-Monitoring] clusteroperator/monitoring should not change condition/Available
This test has passed 98.52% of 3643 runs on release 4.20 [Overall] in the last week.
---
[sig-arch][Late] operators should not create watch channels very often
This test has passed 99.49% of 3345 runs on release 4.20 [Overall] in the last week.

Open Bugs
Component Readiness Shows Old Test Name For Renamed Tests
ResilientWatchCacheInitialization (Re)enablement - operator watch counts from component readiness
---
[bz-Monitoring] clusteroperator/monitoring should not change condition/Degraded
This test has passed 98.41% of 3643 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-e2e-gcp-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 0.00% of 1 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:gcp SecurityMode:default Topology:ha Upgrade:none] in the last week.
---
[bz-kube-storage-version-migrator] clusteroperator/kube-storage-version-migrator should not change condition/Available
This test has passed 69.70% of 3643 runs on release 4.20 [Overall] in the last week.
pull-ci-openshift-origin-main-okd-scos-e2e-aws-ovn IncompleteTests
Tests for this run (98) are below the historical average (3243): IncompleteTests (not enough tests ran to make a reasonable risk analysis; this could be due to infra, installation, or upgrade problems)

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: caeab10

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 15, Pass: 15, Fail: 0, Flake: 0]

Copy link

openshift-trt bot commented Jun 16, 2025

Job Failure Risk Analysis for sha: 59f3798

Job Name Failure Risk
pull-ci-openshift-origin-main-4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback MissingData
pull-ci-openshift-origin-main-e2e-azure-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 0.00% of 1 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:azure SecurityMode:default Topology:ha Upgrade:none] in the last week.

Open Bugs
etcd-scaling jobs failing ~60% of the time

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 59f3798

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 16, Pass: 16, Fail: 0, Flake: 0]

Copy link

openshift-trt bot commented Jun 16, 2025

Job Failure Risk Analysis for sha: 59f3798

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-azure-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 0.00% of 1 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:azure SecurityMode:default Topology:ha Upgrade:none] in the last week.

Open Bugs
etcd-scaling jobs failing ~60% of the time

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 59f3798

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 16, Pass: 16, Fail: 0, Flake: 0]

g.Skip("tls configuration is only tested on IPv4 clusters, skipping")
}

insecure := true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused

}

insecure := true
configFlags := &genericclioptions.ConfigFlags{}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused, please remove.

})
})

func ForwardPortsAndExecute(serviceName string, namespace string, ports []string, maxConnectRetries int, initialBackoff time.Duration, toExecute func(int)) error {
Copy link
Contributor

@p0lyn0mial p0lyn0mial Jun 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this function could be simplified a bit:

func forwardPortAndExecute(serviceName string, namespace string, remotePort string, toExecute func(localPort int)) error {
	var err error
	for i := 0; i < 3; i++ {
		if err = func() error {
			ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
			defer cancel()
			localPort := rand.Intn(65534-1025) + 1025
			args := []string{
				"port-forward",
				fmt.Sprintf("svc/%s", serviceName),
				fmt.Sprintf("%d:%s", localPort, remotePort),
				"-n", namespace,
			}

			cmd := exec.CommandContext(ctx, "oc", args...)

			stdout, stderr, err := e2e.StartCmdAndStreamOutput(cmd)
			if err != nil {
				return err
			}
			defer stdout.Close()
			defer stderr.Close()
			defer e2e.TryKill(cmd)

			e2e.Logf("oc port-forward output: %s", readPartialFrom(stdout, 1024))

			toExecute(localPort)

			return nil
		}(); err != nil {
			err = fmt.Errorf("failed to start oc port-forward command: %w", err)
			e2e.Logf(err.Error())
			time.Sleep(2 * time.Second)
		}
	}

	return err
}

func readPartialFrom(r io.Reader, maxBytes int) string {
	buf := make([]byte, maxBytes)
	n, err := r.Read(buf)
	if err != nil && err != io.EOF {
		return fmt.Sprintf("error reading: %v", err)
	}
	return string(buf[:n])
}

WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added, but modified to let the check return early on success instead of needing to do the port forward & function call 3 times on every invocation

200*time.Millisecond,
func(port int) {
conn, err := tls.Dial("tcp", fmt.Sprintf("localhost:%d", port), tlsShouldWork)
o.Expect(err).NotTo(o.HaveOccurred())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could change the signature of the toExecute function to return an err so that we could retry on errors.

err = ForwardPortsAndExecute(
"apiserver",
"openshift-kube-apiserver",
[]string{"443"},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why [] ?

}
}

//////
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please remove

[]string{"443"},
3,
200*time.Millisecond,
func(port int) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

most of the time this function is repeated, extract to a common function and reuse.

})
return podIPs, err
}

var _ = g.Describe("[sig-api-machinery][Feature:APIServer]", func() {
defer g.GinkgoRecover()

oc := exutil.NewCLI("apiserver")

g.It("TestTLSDefaults", func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this test differs from the one that has been added ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The older test goes deeper and tests acceptable cipher suites, but it only checks the API server, and only when it's running under the default configuration.

conn.Close()

_, err = tls.Dial("tcp", fmt.Sprintf("localhost:%d", port), tlsShouldNotWork)
o.Expect(err).To(o.HaveOccurred())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what type/kind of err do we expect here? how are we going to distinguish it from a network err ?

…ortAndExecute a single remote port, make the test callback return an error to trigger retry logic on failure, refactor repeated TLS connection logic into CheckTLSConnection, which now checks that the failure is due to TLS
Copy link
Contributor

openshift-ci bot commented Jun 26, 2025

@jacobsee: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-serial-shard-1 043c2a8 link false /test e2e-aws-ovn-serial-shard-1
ci/prow/e2e-aws-ovn-serial-shard-2 043c2a8 link false /test e2e-aws-ovn-serial-shard-2
ci/prow/e2e-aws-ovn-serial b1847f6 link true /test e2e-aws-ovn-serial
ci/prow/e2e-metal-ipi-serial 52f1f26 link false /test e2e-metal-ipi-serial
ci/prow/e2e-gcp-fips-serial 52f1f26 link false /test e2e-gcp-fips-serial
ci/prow/e2e-metal-ipi-serial-ovn-ipv6 52f1f26 link false /test e2e-metal-ipi-serial-ovn-ipv6
ci/prow/e2e-aws-ovn-serial-publicnet 52f1f26 link true /test e2e-aws-ovn-serial-publicnet
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview 52f1f26 link false /test e2e-metal-ipi-ovn-dualstack-bgp-local-gw-techpreview
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp-techpreview 52f1f26 link false /test e2e-metal-ipi-ovn-dualstack-bgp-techpreview
ci/prow/4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback 59f3798 link false /test 4.12-upgrade-from-stable-4.11-e2e-aws-ovn-upgrade-rollback
ci/prow/e2e-gcp-ovn-etcd-scaling 64f5a15 link false /test e2e-gcp-ovn-etcd-scaling
ci/prow/e2e-aws-ovn-single-node-serial 64f5a15 link false /test e2e-aws-ovn-single-node-serial
ci/prow/e2e-aws-ovn-single-node-upgrade 64f5a15 link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-azure-ovn-upgrade 64f5a15 link false /test e2e-azure-ovn-upgrade
ci/prow/e2e-metal-ipi-virtualmedia 64f5a15 link false /test e2e-metal-ipi-virtualmedia
ci/prow/e2e-openstack-serial 64f5a15 link false /test e2e-openstack-serial
ci/prow/e2e-aws-disruptive 64f5a15 link false /test e2e-aws-disruptive
ci/prow/e2e-metal-ipi-ovn-ipv6 64f5a15 link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-vsphere-ovn-upi 64f5a15 link true /test e2e-vsphere-ovn-upi
ci/prow/e2e-aws-ovn-serial-publicnet-1of2 64f5a15 link false /test e2e-aws-ovn-serial-publicnet-1of2
ci/prow/e2e-aws-ovn-etcd-scaling 64f5a15 link false /test e2e-aws-ovn-etcd-scaling
ci/prow/e2e-metal-ipi-ovn 64f5a15 link false /test e2e-metal-ipi-ovn
ci/prow/e2e-vsphere-ovn-dualstack-primaryv6 64f5a15 link false /test e2e-vsphere-ovn-dualstack-primaryv6
ci/prow/e2e-gcp-fips-serial-1of2 64f5a15 link false /test e2e-gcp-fips-serial-1of2
ci/prow/e2e-azure-ovn-etcd-scaling 64f5a15 link false /test e2e-azure-ovn-etcd-scaling
ci/prow/e2e-aws-ovn-single-node 64f5a15 link false /test e2e-aws-ovn-single-node
ci/prow/e2e-vsphere-ovn-etcd-scaling 64f5a15 link false /test e2e-vsphere-ovn-etcd-scaling
ci/prow/e2e-gcp-disruptive 64f5a15 link false /test e2e-gcp-disruptive
ci/prow/e2e-azure 64f5a15 link false /test e2e-azure
ci/prow/e2e-gcp-fips-serial-2of2 64f5a15 link false /test e2e-gcp-fips-serial-2of2
ci/prow/okd-e2e-gcp 64f5a15 link false /test okd-e2e-gcp
ci/prow/e2e-metal-ipi-serial-2of2 64f5a15 link false /test e2e-metal-ipi-serial-2of2
ci/prow/e2e-openstack-ovn 64f5a15 link false /test e2e-openstack-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Copy link

openshift-trt bot commented Jun 26, 2025

Job Failure Risk Analysis for sha: 64f5a15

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-disruptive High
[sig-architecture] platform pods in ns/openshift-etcd should not exit an excessive amount of times
This test has passed 100.00% of 7 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:hidden Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:micro-downgrade] in the last week.

Open Bugs
etcd platform pod exist test failing on etcd-scaling jobs
pull-ci-openshift-origin-main-e2e-aws-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 50.00% of 2 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:none] in the last week.

Open Bugs
etcd-scaling jobs failing ~60% of the time
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node-upgrade IncompleteTests
pull-ci-openshift-origin-main-e2e-azure-ovn-upgrade IncompleteTests
pull-ci-openshift-origin-main-e2e-vsphere-ovn-etcd-scaling Medium
[sig-network] pods should successfully create sandboxes by adding pod to network
This test has passed 95.08% of 3538 runs on release 4.20 [Overall] in the last week.

Open Bugs
"[sig-network] pods should successfully create sandboxes by adding pod to network" fails often on compact CI jobs
Ignore pod sandbox creation failures due to networking when the node is NetworkUnavailable=true
Component Readiness: pods should successfully create sandboxes by adding pod to network: expected pod UID "aa853924-c6c6-45b7-be56-e059960bc3c6" but got "ab26e0dc-d736-4945-aa02-91fa3f066cdc" from Kube API

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 64f5a15

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 15, Pass: 15, Fail: 0, Flake: 0]

1 similar comment
Copy link

openshift-trt bot commented Jun 26, 2025

Job Failure Risk Analysis for sha: 64f5a15

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-disruptive High
[sig-architecture] platform pods in ns/openshift-etcd should not exit an excessive amount of times
This test has passed 100.00% of 7 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:hidden Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:micro-downgrade] in the last week.

Open Bugs
etcd platform pod exist test failing on etcd-scaling jobs
pull-ci-openshift-origin-main-e2e-aws-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 50.00% of 2 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:none] in the last week.

Open Bugs
etcd-scaling jobs failing ~60% of the time
pull-ci-openshift-origin-main-e2e-aws-ovn-single-node-upgrade IncompleteTests
pull-ci-openshift-origin-main-e2e-azure-ovn-upgrade IncompleteTests
pull-ci-openshift-origin-main-e2e-vsphere-ovn-etcd-scaling Medium
[sig-network] pods should successfully create sandboxes by adding pod to network
This test has passed 95.08% of 3538 runs on release 4.20 [Overall] in the last week.

Open Bugs
"[sig-network] pods should successfully create sandboxes by adding pod to network" fails often on compact CI jobs
Ignore pod sandbox creation failures due to networking when the node is NetworkUnavailable=true
Component Readiness: pods should successfully create sandboxes by adding pod to network: expected pod UID "aa853924-c6c6-45b7-be56-e059960bc3c6" but got "ab26e0dc-d736-4945-aa02-91fa3f066cdc" from Kube API

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 64f5a15

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 15, Pass: 15, Fail: 0, Flake: 0]

Copy link

openshift-trt bot commented Jun 26, 2025

Job Failure Risk Analysis for sha: 64f5a15

Job Name Failure Risk
pull-ci-openshift-origin-main-e2e-aws-disruptive High
[sig-architecture] platform pods in ns/openshift-etcd should not exit an excessive amount of times
This test has passed 100.00% of 7 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:hidden Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:micro-downgrade] in the last week.

Open Bugs
etcd platform pod exist test failing on etcd-scaling jobs
pull-ci-openshift-origin-main-e2e-aws-ovn-etcd-scaling Low
[bz-Cloud Compute] clusteroperator/control-plane-machine-set should not change condition/Degraded
This test has passed 50.00% of 2 runs on release 4.20 [Architecture:amd64 FeatureSet:default Installer:ipi JobTier:rare Network:ovn NetworkStack:ipv4 Owner:eng Platform:aws SecurityMode:default Topology:ha Upgrade:none] in the last week.

Open Bugs
etcd-scaling jobs failing ~60% of the time
pull-ci-openshift-origin-main-e2e-vsphere-ovn-etcd-scaling Medium
[sig-network] pods should successfully create sandboxes by adding pod to network
This test has passed 95.08% of 3538 runs on release 4.20 [Overall] in the last week.

Open Bugs
"[sig-network] pods should successfully create sandboxes by adding pod to network" fails often on compact CI jobs
Ignore pod sandbox creation failures due to networking when the node is NetworkUnavailable=true
Component Readiness: pods should successfully create sandboxes by adding pod to network: expected pod UID "aa853924-c6c6-45b7-be56-e059960bc3c6" but got "ab26e0dc-d736-4945-aa02-91fa3f066cdc" from Kube API

Risk analysis has seen new tests most likely introduced by this PR.
Please ensure that new tests meet guidelines for naming and stability.

New tests seen in this PR at sha: 64f5a15

  • "[sig-api-machinery][Feature:APIServer] TestTLSMinimumVersions [Suite:openshift/conformance/parallel]" [Total: 15, Pass: 15, Fail: 0, Flake: 0]

@dusk125
Copy link
Contributor

dusk125 commented Jul 8, 2025

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 8, 2025
Copy link
Contributor

openshift-ci bot commented Jul 8, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dusk125, jacobsee
Once this PR has been reviewed and has the lgtm label, please assign dennisperiquet for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm Indicates that a PR is ready to be merged. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants