Skip to content

Configure http proxy uploaders #351

Open
@ulrfa

Description

@ulrfa

Allow configuring the constants in httpproxy.go, from configuration file:

const numUploaders = 100
const maxQueuedUploads = 1000000

And specifically allow numUploaders to be set to 0. That would mean that bazel-remote will try fetching from parent cache, but never uploading anything to parent cache.

Optionally also allow disabling forwarding of HEAD requests via proxy.

Use cases:

  • Set to zero if the local bazel-remote instance mostly contains artifacts irrelevant for the parent cache. But there is chance of cache hits in other direction.

  • Set to zero if the executor using the local bazel-remote instance is not trusted enough to populate a central shared cache.

  • Set to zero if the remote execution input files uploaded to a specific remote execution cache instance, should not be distributed futher to also central cache instance. But if the remote executor should still get build result cache hits from central cache. (For this use case it could also make sense to disable forwarding of HEAD requests, and only forward GET requests).

  • Reduce number of uploaders based on what a parent central cache could handle, to avoid overload conditions. (Not sure if it would work i practice or if a single uploader would still consume almost as much resources)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions