Skip to content

feat: add a DependentResource implementation that's also an EventSource #1094

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 28, 2022

Conversation

metacosm
Copy link
Collaborator

The benefit of this change is multi-fold:

  • simplifies AbstractDependentResource which doesn't have to handle
    cases it should not be concerned about
  • allows to share more code with AbstractSimpleDependentResource
  • enforces that the same name is used for the DependentResource and
    EventSource even in the standalone mode when using one of the provided
    implementations

@metacosm metacosm self-assigned this Mar 27, 2022
@metacosm metacosm requested a review from csviri March 27, 2022 19:21
@csviri
Copy link
Collaborator

csviri commented Mar 28, 2022

Generally speaking, I think AbstractDependetResource should not be aware of caches.
Consider a case when external resources are managed: but triggers comes from an event queue (like SQS) where the message is not resource oriented. This is typically what happens (and desired also from operators view) for example with AWS resources, see for example subscription on RDS events message:

https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-cloudwatch-events.sample.html

In this case the event would trigger the reconciliation, but the dependent resource would not have an event source, especially not a caching event source.
The benefit of this that the AWS RDS api is not polled, the operator is just reacting on actual changes. This is meant to be supported by https://github.com/java-operator-sdk/java-operator-sdk/blob/9d36fb3566b4df6a47208ab689a87e0b75f631d8/operator-framework-core/src/main/java/io/javaoperatorsdk/operator/processing/event/source/inbound/SimpleInboundEventSource.java
to some extent.

(See also: https://aws.amazon.com/premiumsupport/knowledge-center/create-rds-event-subscription/ )

see also the event notification here: https://martinfowler.com/articles/201701-event-driven.html
AWS and other cloud providers implement this pattern regarding eventing of service changes.

private final Class<R> resourceType;

protected AbstractCachingDependentResource(Class<R> resourceType) {
this.resourceType = resourceType;
}

public Optional<R> fetchResource(P primaryResource) {
return eventSource.getAssociated(primaryResource);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know that you did not changed this, but probably the featch resource should be abstract. The idea is that it does not read from the cache.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we do this in a separate PR?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure

Copy link
Collaborator

@csviri csviri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See the comments above.
It would be great if we could have this, just the AbstractEventSource, would not be related to caching at all.

The benefit of this change is multi-fold:
- simplifies AbstractDependentResource which doesn't have to handle
  cases it should not be concerned about
- allows to share more code with AbstractSimpleDependentResource
- enforces that the same name is used for the DependentResource and
  EventSource even in the standalone mode when using one of the provided
  implementations
@metacosm metacosm force-pushed the dependent-event-source-2 branch from 2fe5d60 to fcace92 Compare March 28, 2022 15:46
@metacosm metacosm requested a review from csviri March 28, 2022 15:47
* newly created resource
* @param created the newly created resource
*/
protected abstract void processPostCreate(ResourceID primaryResourceId, R created);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would probably rename to postProcessCreate.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see it. This method, and the other similar one, is to process the given resource post-create, so I think it makes more sense the way it currently is. However, since it seems confusing, maybe something more event-oriented would work better like onCreated?

@@ -37,56 +37,50 @@
private static final Logger log =
LoggerFactory.getLogger(WebPageStandaloneDependentsReconciler.class);

private KubernetesDependentResource<ConfigMap, WebPage> configMapDR;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an unnecessary simplification I actually plan to extend this sample with some if or other workflow enahncements, that is not possible with managed dependent resources.

Basically that is the point that we can hold a simple reference to the dependent resource, that can be easily accessed. Putting it into a map and accessing it via String, is just unnecessary complicaiton.

See also other PR pl with defaul naming.

Copy link
Collaborator Author

@metacosm metacosm Mar 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure but the current example only shows code duplication which isn't really something we want people to replicate so I'd rather leave it as it is for this PR and you can make the modifications you want on it in a subsequent PR.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean code duplicaiton here:

 private void createDependentResources(KubernetesClient client) {
    this.configMapDR = new ConfigMapDependentResource();
    this.configMapDR.setKubernetesClient(client);
    configMapDR.configureWith(new KubernetesDependentResourceConfig()
        .setLabelSelector(DEPENDENT_RESOURCE_LABEL_SELECTOR));

    this.deploymentDR = new DeploymentDependentResource();
    deploymentDR.setKubernetesClient(client);
    deploymentDR.configureWith(new KubernetesDependentResourceConfig()
        .setLabelSelector(DEPENDENT_RESOURCE_LABEL_SELECTOR));

    this.serviceDR = new ServiceDependentResource();
    serviceDR.setKubernetesClient(client);
    serviceDR.configureWith(new KubernetesDependentResourceConfig()
        .setLabelSelector(DEPENDENT_RESOURCE_LABEL_SELECTOR));
  }

?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And calling reconcile on each dependent and create the dependents each exactly the same way… Basically, as it stands, this sample isn't really interesting (apart from showing that people should probably use managed dependents for this kind of use case 😄) so I'd just leave it as it is here and then we can make more changes to make it more interesting in other PRs.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, right, will come up with something to improve it.

Btw otherwise look good to me. Happy to merge! Pls do. ( Just don't know how to approve from mobile app :) )

@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

19.4% 19.4% Coverage
0.0% 0.0% Duplication

Copy link
Collaborator

@csviri csviri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (found it :) )

Copy link
Collaborator

@csviri csviri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@metacosm metacosm merged commit ab23a2c into main Mar 28, 2022
@metacosm metacosm deleted the dependent-event-source-2 branch March 28, 2022 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants