Skip to content

[Feature Request] Improve dependency management experience #18285

Open
@andrross

Description

@andrross

Is your feature request related to a problem? Please describe

Let's say I want to create a new plugin or library that depends on jackson-databind. In theory, I should be able to define a gradle file like this:

dependencies {
  api "com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}"
}

However, with OpenSearch, this will fail with a message like:

> Task :libs:opensearch-example-lib:thirdPartyAudit FAILED
Missing classes:
  * com.fasterxml.jackson.annotation.JacksonAnnotationsInside
  * com.fasterxml.jackson.annotation.JacksonInject
  * com.fasterxml.jackson.annotation.JacksonInject$Value
  ...long list of missing classes...

That's because something in the build framework strips transitive dependencies and forces the library to manually find all transitive dependencies and include them. For this case it requires that I explicitly list two additional dependencies:

dependencies {
  api "com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}"
  api "com.fasterxml.jackson.core:jackson-core:${versions.jackson_databind}"
  api "com.fasterxml.jackson.core:jackson-annotation:${versions.jackson_databind}"
}

But for cases like the AWS SDK it becomes a bit absurd:

// aws sdk v2 stack
api "software.amazon.awssdk:sdk-core:${versions.aws}"
api "software.amazon.awssdk:annotations:${versions.aws}"
api "software.amazon.awssdk:aws-core:${versions.aws}"
api "software.amazon.awssdk:auth:${versions.aws}"
api "software.amazon.awssdk:identity-spi:${versions.aws}"
api "software.amazon.awssdk:checksums:${versions.aws}"
api "software.amazon.awssdk:checksums-spi:${versions.aws}"
api "software.amazon.awssdk.crt:aws-crt:${versions.awscrt}"
api "software.amazon.awssdk:http-auth:${versions.aws}"
api "software.amazon.awssdk:http-auth-aws:${versions.aws}"
api "software.amazon.awssdk:http-auth-spi:${versions.aws}"
api "software.amazon.awssdk:retries:${versions.aws}"
api "software.amazon.awssdk:retries-spi:${versions.aws}"
api "software.amazon.awssdk:endpoints-spi:${versions.aws}"
api "software.amazon.awssdk:http-client-spi:${versions.aws}"
api "software.amazon.awssdk:apache-client:${versions.aws}"
api "software.amazon.awssdk:metrics-spi:${versions.aws}"
api "software.amazon.awssdk:profiles:${versions.aws}"
api "software.amazon.awssdk:regions:${versions.aws}"
api "software.amazon.awssdk:utils:${versions.aws}"
api "software.amazon.awssdk:aws-json-protocol:${versions.aws}"
api "software.amazon.awssdk:protocol-core:${versions.aws}"
api "software.amazon.awssdk:json-utils:${versions.aws}"
api "software.amazon.awssdk:third-party-jackson-core:${versions.aws}"
api "software.amazon.awssdk:s3:${versions.aws}"
api "software.amazon.awssdk:signer:${versions.aws}"
api "software.amazon.awssdk:aws-xml-protocol:${versions.aws}"
api "software.amazon.awssdk:aws-json-protocol:${versions.aws}"
api "software.amazon.awssdk:aws-query-protocol:${versions.aws}"
api "software.amazon.awssdk:sts:${versions.aws}"
api "software.amazon.awssdk:netty-nio-client:${versions.aws}"
api "org.apache.httpcomponents:httpclient:${versions.httpclient}"
api "org.apache.httpcomponents:httpcore:${versions.httpcore}"
api "commons-logging:commons-logging:${versions.commonslogging}"
api "org.apache.logging.log4j:log4j-1.2-api:${versions.log4j}"
api "commons-codec:commons-codec:${versions.commonscodec}"
api "com.fasterxml.jackson.core:jackson-core:${versions.jackson}"
api "com.fasterxml.jackson.core:jackson-databind:${versions.jackson_databind}"
api "com.fasterxml.jackson.core:jackson-annotations:${versions.jackson}"
api "com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:${versions.jackson}"
api "joda-time:joda-time:${versions.joda}"
api "org.slf4j:slf4j-api:${versions.slf4j}"

This pain also compounds over time, as sometimes version updates result in a change in transitive dependencies, which cause the thirdPartyAudit check to fail.

Describe the solution you'd like

It would be great if we could leverage the build system to resolve transitive dependencies.

Why does OpenSearch do it this way? I assume it may have something to do with licensing, as this approach also requires that we include the license and notice file for all dependencies. Surely there's a better way to solve this problem?

Related component

Build

Metadata

Metadata

Assignees

No one assigned

    Labels

    BuildBuild Tasks/Gradle Plugin, groovy scripts, build tools, Javadoc enforcement.enhancementEnhancement or improvement to existing feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions