feat: dependents use traits to specify which features they support

[metacosm]

This is meant to be a cleaner alternative to both #956 and #958.

[scrocquesel]

We do have sort of the same idea (#958 (comment))

And I see that you were force to slip some logic in the manager. But if you do for delete why not for the reconcile ? The dependent is not really standalone anymore.
BTW, the StandaloneDependentTestReconciler assume delete should not be called while it should because KubernetesDependentResource implements Deleter. If has used the appropriate DependentManager, it shouldn't have been an issue.

[csviri]

In general like this implementation. Made some comments that might be worth to revisit and discuss but in general it is good IMHO.

Just one thing I don't really get, does this solve the Read Only case for KubernetesDependetResource? since it is already implements all the trait interfaces. Eventually those will be feature flags? Or how that is meant to be solved?

Because probably something like that would hold if we would provide some generic external resource implementations.

Isn't that at the end breaks down to decision if we have nice trait methods to specify how to create and update (or delete), but then we are not able to do default implementations, or we need feature flags. In a very similar way we have in #958 for editable.

[csviri]

I would not remove this, understand that would not nice with traits, but this is our higher level abstraction should not be changed because of the undelying implementation. We can still have delete trait, that is called only if the AbstractDependentResource implements Deleter within this function.

Also it might look nicer in AbstractDependentResource that there is explicit mention of Deleter

[sclorng]

reconcile and delete are two sides of the same coin. Basically, delete is just reconcile with primary being null. So maybe

void reconcile(Optional<P> primary, Context context) can embrace both. And then, in the default implementation, call delete if primary is not present and it implements Deleter.

[csviri]

This would not work, basically the delete is only needed if there is non kubernetes objects managed or no owner reference set. But then we don't know what to delete if there is no primary as input.

[sclorng]

This would not work, basically the delete is only needed if there is non kubernetes objects managed or no owner reference set. But then we don't know what to delete if there is no primary as input.

That's true. Still, delete and reconcile are tight together in some way.

[csviri]

Currently the delete is called if there is finalizer in place, and the custom resource is marked for deletion.

[sclorng]

In terms of workflows there might be an actual relationship (eventually planned to manage workflows automatically), the delete is a reverse workflow maybe just calling delete instead of reconcile? I mean there is such a relationship for sure, but cannot think of other.

Yes, delete should be understood as remove desired on actual. For creatable, it means to delete the actual. For editonly, it means to undo the patch on actual
Regarding finalizers, in the future, maybe the default can be guess automatically from dependent and set if one dependent is actually implementing traits Deleter.

[csviri]

What is undo patch? Not sure that is possible in general. How it would know the previous state?

[sclorng]

some operators just need to patch a dependent resource. If you don't handle spec change, the dependent knows the content of the patch from the spec. Otherwise, yes, applied spec should be persist somewhere to be able to undo it.

Currently, in my operator, I have something like below and I'm pretty sure it can fit in the dependent model

    @Override
    public UpdateControl<Primary> reconcile(Primary primary,
            Context context) {

        var secondaryResource = context.getSecondaryResource(Dependent.class);
        return secondaryResource.map((Dependent dependent) -> {
            // ensure the secondary resource has still the desired patch applied
            // and update the status
            return ensureIsPatched(dependent, primary);
        }).orElseGet(() -> {
            // give a chance to update the primary resource status
            // for eg. adding a False Ready Condition with reason "missing secondary"
            return missingSecondary(primary);
        });
    }

    @Override
    public DeleteControl cleanup(Primary primary, Context context) {

        var target = context.getSecondaryResource(Dependent.class);
        target.ifPresent((Dependent dependent) -> {
            ensureIsNotPatched(dependent, primary);
        });

        return DeleteControl.defaultDelete();
    }

ensureIsPatched actually do something like

k8sclient.resources(actual).edit( (a) -> applyPatch(a));
primary.getStatus().setObservedSpec(primary.getSpec());
return UpdateControl.updateStatus(primary);

and ensureIsNotPatched

Optional.ofNullable(primary.getStatus().getObservedSpec()).map(spec -> {
            this.client.resources(actual).edit(o -> undoPatch(o, spec));    
        });

The main issue is that while it's a dependent, the reconcile logic is a bit different and having this logic implemented in the dependent base clase itself do not allow much of reuse. I need to take time to look at it with traits aspect.

[metacosm]

Interesting use case! 😄

[csviri]

@sclorng I see, but this is quite an exotic case, I mean probably not makes sense to generalize the framework to support this case at this point. But certainly would be doable just overriding the appropriate methods.

[csviri]

Do I understand correctly that in a read only resource the initializer would be overriden with NOOP operations?
So it would be something Like this in constructor?

ReadOnlyDeployment() {
   init(Creator.NOOP, Updater.NOOP, Deleter.NOOP);
}

One more thing that I might consider, is replacing trait methods with feature flags (so boolean values for editor, updator, deleter), and for NOOP that would mean the method is empty - so in AbstractDependentResource would have already an empty implementation. Wouldn't be more straigforward?

In more detail:

public abstract class AbstractDependentResource<R, P extends HasMetadata>
    implements DependentResource<R, P> {

  protected boolean creator;
  protected boolean updater;
  protected boolean deleter;

  public AbstractDependentResource(boolean creator, boolean updater, boolean deleter) {
      this.creator = creator;
      this.updater = updater;
      this.updater = updater;
  }

  @Override
  public void reconcile(P primary, Context context) {

    if (creator || updater) {
      var maybeActual = getResource(primary);
      var desired = desired(primary, context);
      if (maybeActual.isEmpty()) {
        if (creator) {
          create(desired, primary, context);
        }
      } else {
        final var actual = maybeActual.get();
        if (updater  && match(actual, desired, context)) {
          update(actual, desired, primary, context);
        }
      }
    }
  }

   boolean match(R actualResource, R desiredResource, Context context) {
      Objects.equals(actualResource, desiredResource);
   }
 
 // these should be probably abstract, just in case of KubernetedDependentResource 
  public void update(R actual, R target, P primary, Context context) {
  }
  public void create(R target, P primary, Context context) {
  }
  public void delete(P primary, Context context) {
  }

The positive side would be that it's really just flags, for me it feels that there is quite a lot ceremony to allow these methods for CUP (create,update,delete), while these are anyways implemented in KubernetesDependentResource. Just in little compex way with private base classes in the background.

And in the read only version, or just edit or just create version (not sure we want something like just create actually, but lets say some might need it), these flags needs to be configured anyways just in constructor just in form of methods - what is a little strange.

public class OnlyEditKubernetesClass extends KubernetesDependentResource{
 
public JustEditClass() {
  init(NOOP, new UpdateDependentOperation(), NOOP)
}

// ...
}

What do you think?

[metacosm]

The idea is that DependentResource implementations need to implement the traits to support the feature. That should be the only thing that is required if you somehow extends AbstractDependentResource. By default, if your implementation doesn't implement any traits then it will be read-only. Each trait you implement adds a feature: if you want to be edit-only, you only need to implement Updater, if you want to be create-only, you only need to implement Creator, if you want to be delete-only (not sure that's useful but this makes things coherent), you only need to implement Deleter.

The init method is only there to provide sub-classes of AbstractDependentResource with a way to specify their own default behavior if one of their sub-classes wants to support one of the trait without actually providing an implementation because they are satisfied with the behavior provided by their super-class. This is indeed a little tricky but it mostly makes sense for KubernetesDependentResource where we want to provide default, generic behavior that could be used with minimal effort without actually implementing the traits directly. This allows for a sub-class of KubernetesDependentResource to implement Creator for example without having to actually implement the method since it will inherit the behavior from KubernetesDepedentResource automatically (because KubernetesDependentResource implements all the traits without actually declaring it does, thus allowing sub-classes to inherit default implementations for free).

[metacosm]

The problem with flags is that you could have an implementation that sets the flags without actually providing the implementation which makes things incoherent.

The SDK should try to enforce the semantics: if a user says that their dependent resource needs to be created, then it should enforce that there is indeed logic to create the dependent, though, granted, that logic could be empty.

The act of requiring to implement an interface makes things more explicit than a boolean in a constructor (which could also be passed wrong, i.e. in the wrong order, it's always tricky to have several parameters of the same type on methods).

Another aspect of flags is that these should probably be made available to the annotation layer making it more complex as well. The first dependent resource implementations actually had these flags but I now think that trait interfaces are a more elegant solution because it extracts the information from the code (i.e. what you actually implement) instead of relying on an explicit configuration step, which creates the problem of having to deal with the configuration (and you've been removing the configuration layers that the non-standalone version had) and with the fact that there could be a mismatch between what the configuration says and what is actually implemented. Using traits solves the problem elegantly I think by making the code the configuration 😄.

[csviri]

I understand the goal here, and the structure. I really like it, just wanted to discuss if it could be simplified by the flags.

Maybe we could add a sample Read Only and Update Only implementation, how it would look like, just we can see the code together, but all I want say that at the end this is how an edit only kubernetes would look like at the end:

public class OnlyUpdateKubernetesResource extends KubernetesDependentResource<Deployment, Primary> {
 
public JustEditClass() {
  init(NOOP, new UpdateDependentOperation(), NOOP)
}

 @Override
  public Deployment desired(Primary primary, Context context) {
   //...
  }
}

Where this part:

init(NOOP, new UpdateDependentOperation(), NOOP) is basically same as init(false, true, false); Just more unnatural/complex to understand. So in that regard it's already feature flags used.

That is true that there can be a mismatch for flags and overriden methods. We could provide adapters, which overrides the only the wanna be empty methods and sets the flags. In that regard the current implementation is really good.

I think we should create issues to provide abstract dependent resource for external ones, since there will be the same issue as with kubernetes ones for the getResource(). Will do that.

But I'm otherwise fine to merge this, it's really good.

[csviri]

Also we should add unit tests :)

[csviri]

Maybe we should have a setter for this still on KDR but that can be an another PR when needed

[metacosm]

There actually was one but I removed it with the idea that now everything is defined in a similar fashion using traits to override the default behavior.

[csviri]

The thing is that for standalone DRs, it wouldn't be necessary to make subclasses, but it is how it is. So we will se how it evolves, maybe if someone for standalone ones would not want to do subclass we can provide an "adaptor".

[csviri]

Cannot this be done by the init?

[metacosm]

No, the whole point is to declare what your dependent is doing explicitly. init is only there to set the defaults for subclasses of AbstractDependentResource.

[csviri]

Hmm, won't in most cases actually a full CRUD resource? Should we have an adaptor for that?

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
25 Code Smells

31.0% Coverage
0.5% Duplication