metals tests failing in community build

[lrytz]

log: https://scala-ci.typesafe.com/job/scala-2.13.x-jdk21-integrate-community-build/2307/artifact/logs/metals-build.log

Metals commit: https://github.com/scalameta/metals/commits/565323c2f5dc8f2d4f43aeded7799958fddfab96

Example log

[metals] [info] tests.pc.CompletionSuite.fuzzy1_2.13.17-bin-f4ca74b started
[metals] [error] ==> X tests.pc.CompletionSuite.fuzzy1_2.13.17-bin-f4ca74b  0.219s munit.ComparisonFailException: tests/cross/src/test/scala/tests/pc/CompletionSuite.scala:357
[metals] [error] 356:
[metals] [error] 357:  check(
[metals] [error] 358:    "fuzzy1",
[metals] [error] diff assertion failed
[metals] [error] => Obtained
[metals] [error] ProcessBuilder java.lang
[metals] [error] ProcessBuilder$Redirect java.lang
[metals] [error] ProcessBuilder$Redirect$Type java.lang
[metals] [error] ProcessBuilder - scala.sys.process
[metals] [error] CertPathBuilder - java.security.cert
[metals] [error] CertPathBuilderSpi - java.security.cert
[metals] [error] ProcessBuilderImpl - scala.sys.process
[metals] [error] CertPathBuilderResult - java.security.cert
[metals] [error] PKIXBuilderParameters - java.security.cert
[metals] [error] PooledConnectionBuilder - javax.sql
[metals] [error] CertPathBuilderException - java.security.cert
[metals] [error] PKIXCertPathBuilderResult - java.security.cert
[metals] [error] => Diff (- expected, + obtained)
[metals] [error]  ProcessBuilder java.lang
[metals] [error] +ProcessBuilder$Redirect java.lang
[metals] [error] +ProcessBuilder$Redirect$Type java.lang
[metals] [error]  ProcessBuilder - scala.sys.process

Looks like scala/scala#11024.

The other failures in the log (search for ComparisonFailException) all seem to be about patterns somehow. Not immediately obvious what could have caused these

[metals] [error] ==> X tests.pc.CompletionCaseSuite.trailing-block_2.13.17-bin-f4ca74b  0.105s munit.ComparisonFailException: tests/cross/src/test/scala/tests/pc/CompletionCaseSuite.scala:103
[metals] [error] 102:
[metals] [error] 103:  check(
[metals] [error] 104:    "trailing-block",
[metals] [error] Obtained empty output!
[metals] [error] => Expected:
[metals] [error] case None => scala
[metals] [error] case Some(value) => scala

[som-snytt]

Adjacently scala/scala#10991 (comment)

[lrytz]

Since scala/scala#11024 a lot more symbols make it to metals, including one (java.lang.WeakPairMap$Pair$Weak$1) that happens to crash the classfile parser. scala/bug#9152 (comment).

The exception is triggered in getting some info about the symbol, metals has a catch _ somewhere higher up and we end up with empty scope completions. That leads to test the test failures / missing completions.

[lrytz]

https://gist.github.com/lrytz/6846d2b0b5038c461b429380e4597b6b/revisions shows the diff of symbols that the compiler returns for scope completion (after scala/scala#11024).

Before it was 790 symbols for a practically empty scope (default imports), now it's 1640 symbols.

• Java inner classes (named and anonymous), e.g., java.lang.ClassValue$Entry, java.lang.Module$1
• Symbols for Scala module classes, e.g., scala.None$
• Scala inner classes, e.g., scala.PartialFunction$Lifted, scala.anon$1
• Scala specialized classes, e.g., scala.Product2$mcID$sp

For all of these cases, the symbols that are found are created by SymbolLoaders; when scanning the classpath, for each .class file a pair of ClassSymbol and ModuleSymbol are created. This needs to be done for Java classes, one symbol will hold instance members, the other static members.

For the cases listed above, the two symbols are unlinked and marked non-existing when their type is forced (new symbols for inner classes are created in the unpickler, or from reading the java inner class table). But scope completion returns those symbols without forcing them.

[lrytz]

WIP: https://github.com/scala/scala/compare/2.13.x...lrytz:scala:pr11024fu?expand=1. Not sure yet how to test it. cc @harpocrates.

[harpocrates]

Ooof, sorry I caused this. Next time, feel free to ping me much sooner - looks like you had an inkling this was my fault a month ago!

WIP seems reasonable. I'll try to test out locally by following https://github.com/scala/community-build/blob/6e1e2c2452b0f5437099abc1d2ea3b4f55210896/proj/metals.conf.

[harpocrates]

@lrytz I was able to reproduce the failure from the community build, then also able to confirm that cross/test from Metals passes after your two changes.

---

Mostly for myself for next time, steps to repro community build:

• Publish scala itself (just publishLocal from whatever Scala branch - 2.13.x in this case)
• Publish some stuff from scalameta (basically just this) after also updating AllScala2Versions cross versions to include the snapshot version (2.13.17-bin-SNAPSHOT in this case)
• Run ++2.13.17-bin-SNAPSHOT cross/test in metals after updating cross versions in V.scala to include the snapshot version (2.13.17-bin-SNAPSHOT in this case)