Overhaul tropical varieties, add various new options

[YueRen]

New feature: Interface to tropicalVariety in Singular for tropicalization of ideals.

[fingolfin]

Unfortunately now has a merged conflict in src/TropicalGeometry/variety.jl

[YueRen]

A book test (from my chapter) is failing due to a syntax change in tropical_variety (it used to return a list of TropicalVariety, not it simply returns one TropicalVariety), is it okay to ignore book tests or do we still want to adhere to them strictly?

[joschmitt]

it used to return a list of TropicalVariety, not it simply returns one TropicalVariety

This sounds like a breaking change to me which we cannot do in 1.*?

[benlorenz]

A book test (from my chapter) is failing due to a syntax change in tropical_variety (it used to return a list of TropicalVariety, not it simply returns one TropicalVariety), is it okay to ignore book tests or do we still want to adhere to them strictly?

Unless it is a clear bugfix, the input in the booktests should keep working for all 1.x versions. Only the printing is allowed to change.

Looking at the log there is also an error and not an output change:

  julia> TropV = tropical_variety(I,nu)
- 2-element Vector{TropicalVariety}:
-  Min tropical variety
-  Min tropical variety
+ ERROR: MethodError: no method matching tropical_variety_prime_singular(::MPolyIdeal{…}, ::TropicalSemiringMap{…}; weighted_polyhedral_complex_only::Bool)
+ 
+ Closest candidates are:
+   tropical_variety_prime_singular(::MPolyIdeal, !Matched::TropicalSemiringMap{QQField, ZZRingElem, <:Union{typeof(max), typeof(min)}}; weighted_polyhedral_complex_only)
+    @ Oscar ~/work/Oscar.jl/Oscar.jl/src/TropicalGeometry/variety_prime.jl:42
+   tropical_variety_prime_singular(::MPolyIdeal, !Matched::TropicalSemiringMap{QQField, Nothing, <:Union{typeof(max), typeof(min)}}; weighted_polyhedral_complex_only)
+    @ Oscar ~/work/Oscar.jl/Oscar.jl/src/TropicalGeometry/variety_prime.jl:23
+ 
+ Stacktrace:
+  [1] tropical_variety_prime(I::MPolyIdeal{…}, nu::TropicalSemiringMap{…}; weighted_polyhedral_complex_only::Bool)
+    @ Oscar ~/work/Oscar.jl/Oscar.jl/src/TropicalGeometry/variety_prime.jl:18
+  [2] tropical_variety(I::MPolyIdeal{…}, nu::TropicalSemiringMap{…}; weighted_polyhedral_complex_only::Bool)
+    @ Oscar ~/work/Oscar.jl/Oscar.jl/src/TropicalGeometry/variety.jl:209
+  [3] tropical_variety(I::MPolyIdeal{…}, nu::TropicalSemiringMap{…})
+    @ Oscar ~/work/Oscar.jl/Oscar.jl/src/TropicalGeometry/variety.jl:189
+  [4] top-level scope
+    @ REPL[51]:1
+ Some type information was truncated. Use `show(err)` to see complete types.

[YueRen]

it used to return a list of TropicalVariety, not it simply returns one TropicalVariety

This sounds like a breaking change to me which we cannot do in 1.*?

Yes, I agree that it is a breaking change. So it will have to wait until 2.x then?

[joschmitt]

it used to return a list of TropicalVariety, not it simply returns one TropicalVariety

This sounds like a breaking change to me which we cannot do in 1.*?

Yes, I agree that it is a breaking change. So it will have to wait until 2.x then?

Sounds like it, but in the end this is not for me to decide. Could you provide a reasonable version of this pull request without the breaking changes? By "reasonable", I mean both in terms of functionality but also the work that needs to be done.

[fingolfin]

The conflict has been resolved. Is this ready for a review then, @YueRen ?

[YueRen]

@fingolfin I've double checked everything and I'm happy to merge this PR.

Just to re-iterate: The command tropical_variety has two new optional parameters skip_saturation::Bool=false and skip_decomposition::Bool=false. It works as before by default, returning a list of TropicalVariety, hence all book tests are passing. However, setting skip_decomposition=true will skip a primary decomposition and return a single TropicalVariety, which is closer to the behaviour of that of other software.

[YueRen]

There were some failing test were that are unrelated to my code. I've rebased the PR to the latest master, hopefully that fixes the issue.

[benlorenz]

Everything related to julia nightly is currently expected to fail.

[codecov]

Codecov Report

Attention: Patch coverage is 86.97917% with 25 lines in your changes missing coverage. Please review.

Project coverage is 84.93%. Comparing base (628b77f) to head (f0d0162).
Report is 12 commits behind head on master.

Files with missing lines	Patch %	Lines
src/TropicalGeometry/variety_prime.jl	68.18%	21 Missing ⚠️
src/TropicalGeometry/variety.jl	89.65%	3 Missing ⚠️
src/TropicalGeometry/variety_affine_linear.jl	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4061      +/-   ##
==========================================
- Coverage   84.94%   84.93%   -0.01%     
==========================================
  Files         686      691       +5     
  Lines       92258    92309      +51     
==========================================
+ Hits        78364    78402      +38     
- Misses      13894    13907      +13     

Files with missing lines	Coverage Δ
src/TropicalGeometry/homogenization.jl	100.00% <100.00%> (+1.69%)	⬆️
src/TropicalGeometry/variety_binomial.jl	100.00% <100.00%> (ø)
src/TropicalGeometry/variety_principal.jl	100.00% <100.00%> (ø)
src/TropicalGeometry/variety_zerodimensional.jl	100.00% <100.00%> (ø)
src/TropicalGeometry/variety_affine_linear.jl	85.71% <85.71%> (ø)
src/TropicalGeometry/variety.jl	86.95% <89.65%> (-3.84%)	⬇️
src/TropicalGeometry/variety_prime.jl	68.18% <68.18%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lgoettgens]

This now has a conflict due to #4838. @YueRen could you please give resolving this a try?

[YueRen]

I've rebased the changes now and the only book tests that are failing are pertaining to areas that this PR is not touching. Is there anything else left to do before this PR can be merged?

[joschmitt]

I've rebased the changes now and the only book tests that are failing are pertaining to areas that this PR is not touching. Is there anything else left to do before this PR can be merged?

Someone needs to review it :)

[YueRen]

@micjoswig Would you be able to review the PR? Here are the major changes:

• Dispatching to various specialized tropicalization algorithms was moved from the function tropical_variety to a new function tropical_variety_dispatch.

• The function tropical_variety is now only responsible for some pre-processing and calling tropical_variety_dispatch. Depending on the optional parameters, it may call tropical_variety_dispatch once or multiple times (e.g., once per prime factor). It does the latter by default, as in the OSCAR book.

• All specialized tropicalization algorithms were moved to their own variety_*.jl file. This is because variety.jl became too difficult to navigate and it will only get worse.

• There is a new specialized tropicalization algorithm tropical_variety_prime_singular in variety_prime.jl (because it pertains prime ideals). It essentially calls the tropicalVariety function in Singular via Singular.call_interpreter and uses string manipulation to reconstruct the resulting polyhedral complex in Oscar. This is of course far from ideal, but wrapping tropicalVariety properly in Singular.jl would require some work as it returns a pointer to a Gfan fan. I consider this a temporary bandaid until Oscar gets its own native tropicalization routines.

[micjoswig]

I've rebased the changes now and the only book tests that are failing are pertaining to areas that this PR is not touching. Is there anything else left to do before this PR can be merged?

Someone needs to review it :)

@ooinaruhugh : you had a closer look. Happy?

[ooinaruhugh]

I've rebased the changes now and the only book tests that are failing are pertaining to areas that this PR is not touching. Is there anything else left to do before this PR can be merged?

Someone needs to review it :)

@ooinaruhugh : you had a closer look. Happy?

@micjoswig @YueRen I tried some examples and looks good.

[lgoettgens]

This PR had a release notes: to be added label. I just replaced this be a release notes: use title and updated the title.
If somewants wants to have some more detailed release notes for this PR, then please change the label back and write approx two sentences in a comment below, and then I'll make sure they end up in the changelog.

[fingolfin]

Seems fine to me, perhaps @YueRen has another wish?

[YueRen]

Sounds good, thanks @lgoettgens @fingolfin