Provide out of the box EventSources for Kubernetes Resources

[adam-sandor]

EventSources for basic Kubernetes resources like Deployment, Pod, etc shouldn't have to be part of an Operator but should be provided by the SDK.
Case in point: DeploymentEventSource in the tomcat-sample.

This is not as easy however as moving that class into a new package in the operator-framework. The DeploymentEventSource makes assumptions about how the Deployment is connected to the Tomcat object. Those assumptions are partly encoded in the TomcatController as it is responsible for adding the right labels and ownerReferences to the newly created Deployment. So currently the DeploymentEventSource is tighly coupled to the TomcatController.

This all still feels like there should be a default implementation coming from the framework as it's pretty complicated and even though there are different ways one could implement the functionality it's

[metacosm]

In https://github.com/halkyonio/operator-framework, we defined the notion of dependent resources, which I've discussed already with @csviri. The idea is that, very often, a controller, when it reconciles the state of the main custom resource it deals with, needs, in fact, to check the status of several other resources that influence the status of the main resource. The framework we developed took care of simplifying the experience by making that dependency explicit and making it possible to compute an aggregated state for the main resource based on the status of each dependent resource. The framework also took care of creating/updating/deleting all the dependent resources associated with a main resource. It also provided support for often-declared dependent resources such as Secret / ServiceAccount…

I think we should look into implementing something similar here, particularly, because, based on discussions with the operator-sdk team, it seems to be something that kubebuilder takes into account as well.

[csviri]

@metacosm yes, I agree that we need also a higher abstraction. I would wait with this until we meet with operator SDK team.
Also would be probably quite easy to migrate event sources to this notion. I prepared some prototype how a higher level abstraction could look like. We can take a look also on that together.

[jmesnil]

That is something that I would need in my operator and that I'm able to achieve in Golang by watching secondary resources owned by my custom resources:

https://github.com/wildfly/wildfly-operator/blob/1989b113ec70cc550f60428590de61171e8eb8dd/pkg/controller/wildflyserver/wildflyserver_controller.go#L81-L88

This will trigger a call to my custom resource code handling so that I can reconcile its status based on these secondary resources.

This relies on making sure that any secondary resources that are created by my operator is properly flagged with a ownerReference (something like https://github.com/jmesnil/wildfly-operator/blob/c24f0d1b37889c9184a400821de12ff09a92f8f8/src/main/java/org/wildfly/operator/resources/Resources.java#L29-L39).

Related to this, I'm more interested by "watching" these resources and be notified periodically (in addition to events sources) so that my custom reference can periodically update its status based on these secondary resources's statues (that may not trigger any event when their status changes).

[adam-sandor]

@jmesnil I'm not sure if you're aware but this is already possible (see tomcat sample). The missing thing now is that the framework doesn't do as much as it could to help. You have to create your own EventSource even for standard Kubernetes objects (see DeploymentEventSource) and you have to manage owner references yourself.

[jmesnil]

@adam-sandor sorry, if I was not clear.

I saw the tomcat sample but this requires lots of boilerplate code that should be provided (or helped) by the SDK.
The key word in this RFE is "out-of-the-box" :)

[csviri]

this is done, see InformerEventSource,
see usage: #659

[metacosm]

This isn't addressed by the InformerEventSource. This issue is about providing generic event sources for native kubernetes resources such as Deployment or Secret. The InformerEventSource makes it easier but this isn't really what this issue is about… at least, if I understood properly. 😄

[csviri]

Sorry, you are right, I just read the title probably closed it and 10 minutes later I actually created an issue for this topic because I was not able to find this issue :D