Skip to content

Event Sources, Dependent Resource, Caching and Related Architectural Dilemmas #1134

Closed
@csviri

Description

@csviri
Collaborator

A selection of related architectural questions to discuss about for the future.

1. Generic Cache Layer

Currently caching is done in Event Sources, this is very convenient since Informers provide an out of the box cache. Also it is quite natural since usually the resources we want to caches is the same that we want to get events about.

In addition to that Dependent Resources (DRs), can provide these event sources. However the relationship here is not that obvious. DRs however actually also comes naturally with event sources, since when we managed a resources, naturally we are interested in it.

An alternative would be (that comes up sometimes in discussion) is if we should separate caching from event sources, so if triggering the reconciliation and underling caches should not be tightly coupled as it is now.

One of the goals would be to have a nice structured foundation for limiting cache sizes:
#886

Although the absolute generic implementation would have to go to fabric8 client, in most of the cases in reconciler implementation just a secondary resource is read by a simple getSecondaryResource(). So if the resource of a type not found in the cache, this method might check it on the server.

The getSecondaryResource needs to define the event source, if more event sources are related to the same type. This ideally should not be the case, the most optimal solution would be just to have one event source for one type. But even if there are more, we should not cache the same resource twice, so a shared cache per type would kinda solve this problem / make it nicer.

2. Dependent Resources and Caching

An other question is if all the dependent resources should be always an event source. This is something to discuss, but if not: the getSecondaryResource might rather check the managed dependent resources first, or if the generic caching layer is implemented, the dependent should play well together with the caching layer. Either the dependent should propagate it's value on read to the cache, or getting the resource on a cache miss might use DependentResource.getSecondaryResource(...).

In case every DependentResource would be an event source, the event source could directly work with the cache, so in that case this might be simpler. Especially for standalone mode since, there the dependent resource are not known.

3. Sharing Resources Between Controllers

If there are multiple controllers within one operator, it could be reasonable to assume that those are related to each other (at least some cases), if those are related to each other probably there are event sources informers which could be shared between them. Same applies in case for a generic caching. So we can also take a look in the future on the architecture and see how feasible is to share EventSource and/or caches between controllers.

Activity

changed the title [-]Separation of Event Sources and Caching and Related Architectural Issues[/-] [+]Discussion Separation of Event Sources and Caching and Related Architectural Issues[/+] on Apr 5, 2022
changed the title [-]Discussion Separation of Event Sources and Caching and Related Architectural Issues[/-] [+]Separation of Event Sources and Caching and Related Architectural Issues (?)[/+] on Apr 5, 2022
changed the title [-]Separation of Event Sources and Caching and Related Architectural Issues (?)[/-] [+]Separation of Event Sources and Caching and Related Architectural Issues[/+] on Apr 5, 2022
changed the title [-]Separation of Event Sources and Caching and Related Architectural Issues[/-] [+]Event Sources, Caching and Related Architectural Issues[/+] on Apr 5, 2022
changed the title [-]Event Sources, Caching and Related Architectural Issues[/-] [+]Event Sources, Caching and Related Architecture[/+] on Apr 5, 2022
changed the title [-]Event Sources, Caching and Related Architecture[/-] [+]Event Sources, Caching and Related Architectural Dilemmas[/+] on Apr 5, 2022
changed the title [-]Event Sources, Caching and Related Architectural Dilemmas[/-] [+]Event Sources, Dependent Resource, Caching and Related Architectural Dilemmas[/+] on Apr 6, 2022
github-actions

github-actions commented on Jun 7, 2022

@github-actions

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

added
lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
on Jun 7, 2022
github-actions

github-actions commented on Jun 21, 2022

@github-actions

This issue was closed because it has been stalled for 14 days with no activity.

1 remaining item

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    architecturecaching-epickind/featureCategorizes issue or PR as related to a new feature.lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @csviri

        Issue actions

          Event Sources, Dependent Resource, Caching and Related Architectural Dilemmas · Issue #1134 · operator-framework/java-operator-sdk