feat: HBase Listener integration

[adwk67]

Description

fixes: #618

How to test

• build the image locally with bake -p hbase from the branch for this PR
• upload image(s) to local kind cluster with e.g. kind load docker-image oci.stackable.tech/sdp/hbase:2.6.1-stackable0.0.0-dev --name stackable-data-platform
• deploy the operator with stackable op in hbase=0.0.0-pr639
• run tests with e.g. ./scripts/run-tests --skip-operator hbase --test ...

Implementation notes

• Persistent volumes are used for all endpoints as they are all user-facing
• the listener endpoints are written to hbase-site.xml
• no group listeners are used across a group of replicas, but each listener is specific to a particular pod

Output from the HBase demo:

17:59 $ stackablectl stacklet list

┌───────────┬───────────┬───────────┬──────────────────────────────────────────────────────────────┬─────────────────────────────────┐
│ PRODUCT   ┆ NAME      ┆ NAMESPACE ┆ ENDPOINTS                                                    ┆ CONDITIONS                      │
╞═══════════╪═══════════╪═══════════╪══════════════════════════════════════════════════════════════╪═════════════════════════════════╡
│ hbase     ┆ hbase     ┆ default   ┆ master-default-0-master              172.19.0.3:32673        ┆ Available, Reconciling, Running │
│           ┆           ┆           ┆ master-default-0-ui-http             172.19.0.3:30430        ┆                                 │
│           ┆           ┆           ┆ regionserver-default-0-regionserver  172.19.0.4:31204        ┆                                 │
│           ┆           ┆           ┆ regionserver-default-0-ui-http       172.19.0.4:31767        ┆                                 │
│           ┆           ┆           ┆ regionserver-default-1-regionserver  172.19.0.3:30602        ┆                                 │
│           ┆           ┆           ┆ regionserver-default-1-ui-http       172.19.0.3:31627        ┆                                 │
│           ┆           ┆           ┆ restserver-default-0-rest-http       172.19.0.3:30155        ┆                                 │
│           ┆           ┆           ┆ restserver-default-0-ui-http         172.19.0.3:30586        ┆                                 │
├╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤

The Localities indicate that the serving region is the same node as the HDFS data:

CRD change

Note

There does not need to be a decision on this (discussed briefly in planning 09.04.2024) as the implementation is in line with existing implementations for HDFs and Kafka.

old:

clusterConfig:
   listenerClass: {external-unstable|external-stable|cluster-internal}

new:
(this is inline with the implementations for HDFS and Kafka)

spec:
  masters:
    config:
      listenerClass: {external-unstable|external-stable|cluster-internal}
  regionServers:
    config:
      listenerClass: {external-unstable|external-stable|cluster-internal}
  restServers:
    config:
      listenerClass: {external-unstable|external-stable|cluster-internal}

Tests (OpenShift)

--- PASS: kuttl (1736.72s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/cluster-operation_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (251.16s)
        --- PASS: kuttl/harness/omid_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true_omid-1.1.2 (287.90s)
        --- PASS: kuttl/harness/orphaned_resources_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (186.60s)
        --- PASS: kuttl/harness/smoke_hbase-2.6.1_hdfs-3.4.1_zookeeper-3.9.3_listener-class-external-unstable_openshift-true (320.82s)
        --- PASS: kuttl/harness/overrides_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (182.74s)
        --- PASS: kuttl/harness/kerberos_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_listener-class-external-unstable_kerberos-realm-PROD.MYCORP_kerberos-backend-mit_openshift-true (510.35s)
        --- PASS: kuttl/harness/snapshot-export_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (259.22s)
        --- PASS: kuttl/harness/profiling_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (196.50s)
        --- PASS: kuttl/harness/opa_hbase-opa-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_krb5-1.21.1_opa-1.0.1_openshift-true (280.21s)
        --- PASS: kuttl/harness/logging_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (209.32s)
        --- PASS: kuttl/harness/shutdown_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (331.83s)
        --- PASS: kuttl/harness/external-access_hbase-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_listener-class-external-unstable_openshift-true (180.55s)
        --- PASS: kuttl/harness/resources_hbase-latest-2.6.1_hdfs-latest-3.4.1_zookeeper-latest-3.9.3_openshift-true (213.95s)
PASS

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

• Changes are OpenShift compatible
• CRD changes approved
• CRD documentation for all fields, following the style guide.
• Helm chart can be installed and deployed operator works
• Integration tests passed (for non trivial changes)
• Changes need to be "offline" compatible

Reviewer

• Code contains useful comments
• Code contains useful logging statements
• (Integration-)Test cases added
• Documentation added or updated. Follows the style guide.
• Changelog updated
• Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

• Feature Tracker has been updated
• Proper release label has been added
• Roadmap has been updated

[sbernauer]

Can you please run cargo update -p tokio to fix the cargo deny? :)

[nightkr]

The corresponding product patch branch is (AIUI) https://github.com/stackabletech/docker-images/tree/feat/hbase-resolvable-endpoints.

[nightkr]

I haven't tested accessing hbase with an actual client and there's a lot (lot) of cleanup to do, but https://github.com/stackabletech/hbase-operator/tree/feat/integrate-listener-operator-/nat (61c79a5) and https://github.com/stackabletech/docker-images/tree/feat/hbase-resolvable-endpoints-/nat (stackabletech/docker-images@2a2a5ac) seem to consistently make it all look correct in the UI at least.