HTTP connection sharing across managed channels

[shivamgupta1]

Is your feature request related to a problem?

gRPC usage within our organization requires creation of several managed channels, each with a unique authority, for various endpoints within the same server app. This leads to creation of multiple redundant HTTP/2 connections that could've just been re-used. The current NettyChannelBuilder impl doesn't feature any option to enable transport level connection re-use.

Describe the solution you'd like

NettyChannelBuilder currently defines NettyTransportFactory as a nested class and the implementation spawns a new NettyClientTransport each time it's called.

I'd like to:

• Add a new method to NettyChannelBuilder that allows injecting a custom ClientTransportFactoryBuilder. If this is too much flexibility, then we can just add a new method to enable connection re-use.
• Create a new implementation of ClientTransportFactory that returns a preexisting (if available) NettyClientTransport wrapped by a new decorator. The decorator object will ensure that methods like start will still provide appropriate lifecycle notifications to the supplied listeners.

Describe alternatives you've considered

We tried re-using subchannels within the loadbalancer but it isn't clean. For e.g., the synchronization context isn't shared across different loadbalancer objects.

Additional context

If the gRPC maintainers see value in this addition and agree with the proposed approach, I can send a PR for this feature soon.

[ejona86]

If you only need to change the authority, then there is CallOptions.withAuthority(), so you can change it on a per-RPC basis. That requires the client to know that two hostnames use the same backend/proxy, so it might not be what you need.

The documentation mentions that we don't validate the certificate is allowed to host the authority. That has been implemented now, but it only warns at the moment IIRC. CC @kannanjgithub

[shivamgupta1]

@ejona86 thanks for looking into this. Let me clarify, we don't need to change the authority. The motivation here is to reduce the memory and network overhead of the HTTP connections themselves. We need to use different managed channels since these endpoints do have different configurations (e.g. timeouts, retry config, loadbalancer strategy etc.), although they're all served by the same backend server hosts.

[ejona86]

We need to use different managed channels since these endpoints do have different configurations (e.g. timeouts, retry config, loadbalancer strategy etc.)

Timeouts and retry should be able to be to share a channel. Or are you wanting different configuration for the same gRPC methods in different circumstances? LB strategy would have trouble in general, as LB policies should be allowed to create multiple of the same subchannel to get additional connections and that won't be possible at a ClientTransportFactory level.

The different LB strategies doesn't necessarily require using separate channels. An LB could look at the authority/method being accessed and choose to change its behavior. Generally today it would have different child policies each with their own subchannels; obviously here you want those subchannels to be shared. That isn't possible today, but we're working on removing Subchannel.updateAddresses() which would make it possible.

This sort of conversation works best if we are more concrete about what you actually need to change across the channels. We're much more likely to solve it from that angle.

[shivamgupta1]

In our design, clients create channels using only the service name, for e.g. managedChannelProvider.forServiceName(<name>). This allows us more flexibility since services can move across apps seamlessly without requiring any migrations on the clients. Many of these services are within the same server app but each requires a new managed channel on the client side. Our custom service discovery system also manages configuration at the service level, which is then supplied to these channels – some of this is a result of retrofitting our existing solutions to gRPC. As you can see, this prevents us from reusing managed channels as you're suggesting, if I'm understanding it correctly. Even if we wanted to change this behavior to lose the flexibility and encapsulation described above, it won't fix the thousands of apps that already use this approach unless we embarked on a large migration.

[shivamgupta1]

@ejona86 @kannanjgithub any further thoughts?

[ejona86]

clients create channels using only the service name, for e.g. managedChannelProvider.forServiceName()

That is compatible with deadline and retries. You could have a single service config for all services.

I think you can do most of what you need by sharing ManagedChannels across services. Given your API, you'd need to implement the ManagedChannel API to manage reference counts on the "real" ManagedChannels. Your managedChannelProvider would hold all the "real" ManagedChannels in a map and select the proper one for the service, and increment its refcount while returning your ManagedChannel shim implementation. The only problem I see is you wouldn't bother to have awaitTermination() delay until all pending RPCs are complete just for that one channel. But that's probably not a problem for you; you'll just say the channel shuts down instantly.

There's still a problem of supporting different load balancing policies. But how much of that is a real requirement? I imagine it doesn't vary that much, so maybe you could just create a different "real" managed channel for each LB configuration.