Skip to content

Cache main OCI packages when running with kcl run oci://... #1992

@kptranum

Description

@kptranum

Feature Request

Is your feature request related to a problem? Please describe:
When executing remote OCI-hosted KCL packages directly using kcl run oci://<registry>/<repo>:<tag>, KCL downloads the package on every invocation. Unlike dependencies declared in a kcl.mod file (which are cached by KPM), the main package itself is treated as ephemeral. This causes:

  • Repeated network downloads (performance penalty)
  • Higher load on private registries
  • Inability to run previously used packages offline
  • Inefficiencies in controller/operator contexts (e.g. Crossplane reconciliation loops)

Concrete pain point (Crossplane):
In a Crossplane Composition using crossplane-contrib/function-kcl, the function is invoked during every reconciliation cycle. A configuration like:

input:
  apiVersion: template.fn.crossplane.io/v1beta1
  kind: KCLInput
  metadata:
    name: test
  spec:
    source: oci://<registry>/<repo>:<tag>

leads to the OCI package being re-downloaded on each reconciliation, effectively hammering the registry and adding latency.


Describe the feature you'd like:
Add persistent caching for the primary OCI package referenced by kcl run oci://..., similar to how:

  • Docker caches images after the first pull
  • KPM caches declared dependencies in kcl.mod

Desired behavior:

  1. On first execution, download and store the referenced <registry>/<repo>:<tag> (or digest) in the existing KPM cache (e.g. ~/.kcl/kpm/...).
  2. On subsequent identical references (same tag or digest), reuse the cached artifact with zero network fetch.
  3. Provide optional flags:
    • --no-cache or --refresh to force a re-download
    • --cache-info (or similar) to show whether a cached artifact was used

Describe alternatives you've considered:
???


Teachability, Documentation, Adoption, Migration Strategy:

Scenarios improved:

  • Crossplane compositions: Reconciliation loops no longer trigger repeated downloads; faster convergence and reduced registry load.
  • CI pipelines: Multiple jobs referencing the same OCI package avoid redundant network I/O.
  • Developer workflows: Faster iterative runs when testing remote packages.
  • Offline / air-gapped: A package can be “primed” once, then reused offline.
  • Reproducibility: Digest-based caching encourages immutable references.

Documentation updates:

  • Add a “Main Package Caching” section under “Running Remote Packages”.
  • Update kcl run --help with new flags (--no-cache, --refresh).
  • Provide troubleshooting guidance: how to inspect/clear the cache.
  • Best practice note: prefer digest references for guaranteed immutability.

Backward compatibility:

  • Default behavior silently improves performance (no flag required).
  • Users needing the current always-fetch semantics can opt out with --no-cache or an env var.

No migration blockers—this is an additive optimization.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions