NoClassDefFoundError regressions since 1.0.0-rc7

[dsyer]

This happens with rc8 but not rc7.

...
[target/demo:36]     analysis:  59,408.78 ms
fatal error: java.lang.NoClassDefFoundError
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:598)
	at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
	at com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:419)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:278)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:375)
	at com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:104)
Caused by: java.lang.NoClassDefFoundError
	at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.lookupType(HotSpotJVMCIRuntime.java:464)
	at jdk.vm.ci.hotspot.HotSpotResolvedObjectTypeImpl.lookupType(HotSpotResolvedObjectTypeImpl.java:957)
	at jdk.vm.ci.meta.UnresolvedJavaType.resolve(UnresolvedJavaType.java:92)
	at jdk.vm.ci.hotspot.HotSpotResolvedJavaFieldImpl.getType(HotSpotResolvedJavaFieldImpl.java:126)
	at com.oracle.graal.pointsto.meta.AnalysisType.<init>(AnalysisType.java:185)
	at com.oracle.graal.pointsto.meta.AnalysisUniverse.createType(AnalysisUniverse.java:251)
	at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookupAllowUnresolved(AnalysisUniverse.java:193)
	at com.oracle.graal.pointsto.meta.AnalysisUniverse.lookup(AnalysisUniverse.java:170)
	at com.oracle.graal.pointsto.meta.AnalysisType.getEnclosingType(AnalysisType.java:947)
	at com.oracle.svm.hosted.analysis.Inflation.checkType(Inflation.java:153)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080)
	at com.oracle.svm.hosted.analysis.Inflation.checkObjectGraph(Inflation.java:130)
	at com.oracle.graal.pointsto.BigBang.checkObjectGraph(BigBang.java:590)
	at com.oracle.graal.pointsto.BigBang.finish(BigBang.java:562)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:691)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:402)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1386)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.ClassNotFoundException: com.oracle.mxtool.junit.MxJUnitRequest
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at jdk.vm.ci.hotspot.CompilerToVM.lookupType(Native Method)
	at jdk.vm.ci.hotspot.HotSpotJVMCIRuntime.lookupType(HotSpotJVMCIRuntime.java:456)

Any ideas for a workaround?

[sdeleuze]

Same kind of regression when compiling https://github.com/spring-projects/spring-fu/tree/master/samples/kofu-reactive-graal, it works with rc7, but not with rc8 (Error: Processing image build request failed message) nor with rc9 where it fails with following message:

com.sample.applicationkt:22405]     analysis:   9,021.73 ms
fatal error: java.lang.NoClassDefFoundError: Lorg/aspectj/weaver/World;
	at java.lang.Class.getDeclaredFields0(Native Method)
	at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
	at java.lang.Class.getDeclaredField(Class.java:2068)
	at jdk.vm.ci.hotspot.HotSpotResolvedJavaFieldImpl.toJava(HotSpotResolvedJavaFieldImpl.java:191)
	at jdk.vm.ci.hotspot.HotSpotResolvedJavaFieldImpl.getAnnotation(HotSpotResolvedJavaFieldImpl.java:179)
	at com.oracle.svm.hosted.SVMHost.platformSupported(SVMHost.java:150)

[sdeleuze]

See also #494 (comment) that mentions related regressions.

[cstancu]

Can you please try with latest master? In d598be2 we introduced a new option: --allow-incomplete-classpath. The type resolution errors should be reported at run time when the types are accessed for the first time, instead of during image building. This will be available in rc10.

[sdeleuze]

I tried to build my usual https://github.com/spring-projects/spring-fu/tree/master/samples/kofu-reactive-graal project with latest GraalVM master and --allow-incomplete-classpath, but it fails with following error message:

[com.sample.applicationkt:20264]     analysis:  23,238.68 ms
error: Error loading a referenced type: java.lang.reflect.InvocationTargetException
Detailed message:
Error: Error loading a referenced type: java.lang.reflect.InvocationTargetException
Trace: 
	at parsing org.springframework.boot.autoconfigure.validation.ValidatorAdapter.create(ValidatorAdapter.java:137)
Call path from entry point to org.springframework.boot.autoconfigure.validation.ValidatorAdapter.create(): 
	at org.springframework.boot.autoconfigure.validation.ValidatorAdapter.create(ValidatorAdapter.java:137)
	at org.springframework.boot.autoconfigure.validation.ValidatorAdapter.getExistingOrCreate(ValidatorAdapter.java:119)
	at org.springframework.boot.autoconfigure.validation.ValidatorAdapter.get(ValidatorAdapter.java:111)
	at org.springframework.boot.autoconfigure.web.reactive.WebFluxAutoConfiguration$EnableWebFluxConfiguration.webFluxValidator(WebFluxAutoConfiguration.java:243)
	at org.springframework.web.reactive.config.WebFluxConfigurationSupport.getConfigurableWebBindingInitializer(WebFluxConfigurationSupport.java:331)
	at org.springframework.web.reactive.config.WebFluxConfigurationSupport.requestMappingHandlerAdapter(WebFluxConfigurationSupport.java:271)
	at org.springframework.boot.autoconfigure.web.reactive.ReactiveWebServerInitializer.lambda$initialize$14(ReactiveWebServerInitializer.java:98)
	at org.springframework.boot.autoconfigure.web.reactive.ReactiveWebServerInitializer$$Lambda$505/1430693128.get(Unknown Source)
	at com.oracle.svm.core.jdk.SystemPropertiesSupport.initializeLazyValue(SystemPropertiesSupport.java:130)
	at com.oracle.svm.core.jdk.SystemPropertiesSupport.setProperty(SystemPropertiesSupport.java:115)
	at com.oracle.svm.core.jdk.Target_java_lang_System.setProperty(JavaLangSubstitutions.java:359)
	at com.oracle.svm.core.properties.RuntimePropertyParser.parseProperty(RuntimePropertyParser.java:68)
	at com.oracle.svm.core.properties.RuntimePropertyParser.parse(RuntimePropertyParser.java:43)
	at com.oracle.svm.jni.functions.JNIInvocationInterface$Exports.JNI_CreateJavaVM(JNIInvocationInterface.java:144)
	at com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ejni_002efunctions_002eJNIInvocationInterface_0024Exports_002eJNI_005fCreateJavaVM_0028com_002eoracle_002esvm_002ejni_002enativeapi_002eJNIJavaVMPointer_002ccom_002eoracle_002esvm_002ejni_002enativeapi_002eJNIEnvironmentPointer_002ccom_002eoracle_002esvm_002ejni_002enativeapi_002eJNIJavaVMInitArgs_0029(generated:0)
Original exception that caused the problem: java.lang.reflect.InvocationTargetException
	at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:87)
	at com.oracle.graal.pointsto.infrastructure.WrappedConstantPool.loadReferencedType(WrappedConstantPool.java:109)
	at com.oracle.svm.hosted.phases.SubstrateClassInitializationPlugin.loadReferencedType(SubstrateClassInitializationPlugin.java:70)
	at org.graalvm.compiler.java.BytecodeParser.maybeEagerlyResolve(BytecodeParser.java:4020)
	at org.graalvm.compiler.java.BytecodeParser.lookupType(BytecodeParser.java:3929)
	at org.graalvm.compiler.java.BytecodeParser.genNewInstance(BytecodeParser.java:4167)
	at org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:4948)
	at org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3150)
	at org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:2959)
	at org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:889)
	at org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:783)
	at org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:95)
	at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
	at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:204)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:323)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:310)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:300)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:107)
	at com.oracle.graal.pointsto.flow.StaticInvokeTypeFlow.update(InvokeTypeFlow.java:346)
	at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:508)
	at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:174)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.lang.IncompatibleClassChangeError: class org.springframework.validation.beanvalidation.OptionalValidatorFactoryBean has interface org.springframework.validation.beanvalidation.LocalValidatorFactoryBean as super class
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at com.oracle.svm.hosted.NativeImageClassLoader.loadClass(NativeImageClassLoader.java:67)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at jdk.vm.ci.hotspot.CompilerToVM.resolveTypeInPool(Native Method)
	at jdk.vm.ci.hotspot.HotSpotConstantPool.loadReferencedType(HotSpotConstantPool.java:700)
	... 32 more

Error: Image building with exit status 1

You can reproduce it as following:

• git clone https://github.com/spring-projects/spring-fu.git
• cd spring-fu/samples/kofu-reactive-graal
• vi build.sh
• Modify "native-image" to "mx -p /path_to_substratevm native-image --allow-incomplete-classpath"
• ./build.sh

Getting that fixed before my upcoming GraalVM demo at Devoxx next week would be much appreciated ;-)

[cstancu]

@sdeleuze thanks for the steps to reproduce. It looks like you hit a corner case in our support for incomplete classpaths. Essentially what is happening is that javax.validation.Validator, which is implemented by one of the super classes of OptionalValidatorFactoryBean is missing from the classpath. Here is the complete hierarchy: org.springframework.validation.beanvalidation.OptionalValidatorFactoryBean extends org.springframework.validation.beanvalidation.LocalValidatorFactoryBean extends org.springframework.validation.beanvalidation.SpringValidatorAdapter implements javax.validation.Validator.

[sdeleuze]

Ok thanks for the explanation, please ping me when a fix is available on master to allow me to test it again.

[cstancu]

@sdeleuze please build from master. With this fix you should be able to build an image for your kofu-reactive-graal example. Additionally to --allow-incomplete-classpath you also need --delay-class-initialization-to-runtime=org.springframework.format.support.DefaultFormattingConversionService.

[sdeleuze]

Awesome, this indeed now works with --allow-incomplete-classpath, this issue can be closed.

@dsyer It should solves a bunch of issues you opened like #655 and potentially others.

[dsyer]

I just tried to build a native image with GraalVM from master:

mx -p ~/dev/scratch/graal-vm/substratevm/ native-image -Dio.netty.noUnsafe=true -H:Name=target/bunc -Dio.netty.noUnsafe=true -Dio.netty.noJdkZlibDecoder=true -Dio.netty.noJdkZlibEncoder=true -H:ReflectionConfigurationFiles=func.json --report-unsupported-elements-at-runtime --delay-class-initialization-to-runtime=io.netty.handler.codec.http.HttpObjectEncoder,org.springframework.core.io.VfsUtils,io.netty.handler.ssl.JdkNpnApplicationProtocolNegotiator,io.netty.handler.ssl.ReferenceCountedOpenSslEngine,org.springframework.format.support.DefaultFormattingConversionService --allow-incomplete-classpath -cp $CP com.example.func.BuncApplication
...
[target/bunc:14602]    (compile):  20,885.41 ms
[target/bunc:14602]      compile:  28,733.14 ms
[target/bunc:14602]        image:   2,208.52 ms
error: Static field or an object referenced from a static field changed during native image generation?
  object:[Ljava.lang.reflect.Parameter;@2e54d0d5  of class: java.lang.reflect.Parameter[]
  reachable through:
    object: public org.springframework.http.HttpHeaders org.springframework.web.reactive.result.method.RequestMappingInfoHandlerMapping$HttpOptionsHandler.handle()  of class: java.lang.reflect.Method
    root: null

Error: Image building with exit status 1

This is a new one, so technically a regression. I could build this native image with rc5. Source code and README: https://github.com/dsyer/spring-boot-micro-apps.

[dsyer]

The MicroApplication (which was relatively stable befoew) also fails in a new way:

mx -p ~/dev/scratch/graal-vm/substratevm/ native-image -Dio.netty.noUnsafe=true -H:Name=target/micro -H:ReflectionConfigurationFiles=micro.json --report-unsupported-elements-at-runtime --delay-class-initialization-to-runtime=io.netty.handler.codec.http.HttpObjectEncoder,org.springframework.core.io.VfsUtils,io.netty.handler.ssl.JdkNpnApplicationProtocolNegotiator,io.netty.handler.ssl.ReferenceCountedOpenSslEngine --allow-incomplete-classpath -cp $CP com.example.micro.MicroApplication
$ ./target/micro 
Exception in thread "main" java.lang.NoClassDefFoundError: org.apache.logging.log4j.LogManager
	at java.lang.Throwable.<init>(Throwable.java:265)
	at java.lang.Error.<init>(Error.java:70)
	at java.lang.LinkageError.<init>(LinkageError.java:55)
	at java.lang.NoClassDefFoundError.<init>(NoClassDefFoundError.java:59)
	at com.oracle.svm.core.Exceptions.throwNoClassDefFoundError(Exceptions.java:43)
	at org.apache.commons.logging.LogAdapter$Log4jLog.<clinit>(LogAdapter.java:135)
	at com.oracle.svm.core.hub.ClassInitializationInfo.invokeClassInitializer(ClassInitializationInfo.java:341)
	at com.oracle.svm.core.hub.ClassInitializationInfo.initialize(ClassInitializationInfo.java:261)
	at java.lang.Class.ensureInitialized(DynamicHub.java:381)
	at org.apache.commons.logging.LogAdapter$Log4jAdapter.createLog(LogAdapter.java:102)
	at org.apache.commons.logging.LogAdapter.createLog(LogAdapter.java:79)
	at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:67)
	at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:59)
	at org.springframework.context.support.AbstractApplicationContext.<init>(AbstractApplicationContext.java:162)
	at org.springframework.context.support.GenericApplicationContext.<init>(GenericApplicationContext.java:111)
	at com.example.micro.MicroApplication.run(MicroApplication.java:55)
	at com.example.micro.MicroApplication.main(MicroApplication.java:47)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:164)

[cstancu]

@dsyer I am trying to replicate the issues you pointed above using https://github.com/dsyer/spring-boot-micro-apps. I am probably missing something but it looks like my CP is incomplete:

~/projects/spring-boot-micro-apps$ CP=`java -jar target/micro-0.0.1-SNAPSHOT.jar --thin.classpath --thin.profile=graal`
~/projects/spring-boot-micro-apps$ echo $CP
/home/cdrtz/projects/spring-boot-micro-apps/target/micro-0.0.1-SNAPSHOT.jar