Unify type encoding for similar types

[antonydellavecchia]

These changes allow for a many to one serialization.

The use case is to have a single "name" for types with the same encoding.
The Oscar type is now stored using the "_instance" keyword, when there isn't a bijection between the type
and the type "name"

[antonydellavecchia]

Default type storing FqField only the type name has changed.

{
  "_ns": {
    "Oscar": [
      "https://github.com/oscar-system/Oscar.jl",
      "1.3.0-DEV-2ebe51fbb819d32ee59f8f43cd3fb506b796b249"
    ]
  },
  "_type": "FiniteField",
  "data": "2",
  "id": "aad30397-63b7-4082-a55f-4cc425217b83"
}

Non default types have _instance keyword which is a string of the Oscar type that it was before serialization.

{
  "_instance": "fpField",
  "_ns": {
    "Oscar": [
      "https://github.com/oscar-system/Oscar.jl",
      "1.3.0-DEV-2ebe51fbb819d32ee59f8f43cd3fb506b796b249"
    ]
  },
  "_type": "FiniteField",
  "data": "2"
}

[micjoswig]

Should I be irritated by these comments in the code?

if haskey(s.obj, :_instance)
   # to be safe we need this check but I can't figure out how to get sending
   # types to strings to behave properly
   #s.obj["_instance"] in keys(reverse_type_map[s.obj[type_key]])
   T = eval(Meta.parse(s.obj["_instance"]))
 else
   T = decode_type(s)
 end

What does it mean that "I can't figure out how to get sending types to strings to behave properly"?

[micjoswig]

See above.

[antonydellavecchia]

Should I be irritated by these comments in the code?

if haskey(s.obj, :_instance)
   # to be safe we need this check but I can't figure out how to get sending
   # types to strings to behave properly
   #s.obj["_instance"] in keys(reverse_type_map[s.obj[type_key]])
   T = eval(Meta.parse(s.obj["_instance"]))
 else
   T = decode_type(s)
 end

What does it mean that "I can't figure out how to get sending types to strings to behave properly"?

This is mainly a todo for me, I describe this issue in the description for the pull request. It appears that string(fpField) = "Nemo.fpField" and sometimes it is string(fpField) = "fpField". I do believe this is fixable, I just haven't had the time to figure out how yet.

I've commented in the code the places where the unexpected results are happening (see following commit)

[antonydellavecchia]

The changes would have the following consequences on the example in the paper.
With a bit of work it should also be possible to not have the "_instance" key appear in the types such as PolyRing

{
  "_ns": {
    "Oscar": [
      "https://github.com/oscar-system/Oscar.jl",
      "1.3.0-DEV-4d1a3ffa24b8e340470afa6b40c02150e3558b4e"
    ]
  },
  "_refs": {
    "0377c284-0b31-4d83-b009-0776e883e80b": {
      "_instance": "FqMPolyRing",
      "_type": "MPolyRing",
      "data": {
        "base_ring": "8c4b9843-a0e1-4306-aff3-1a8dca4954a0",
        "symbols": [
          "y",
          "z"
        ]
      }
    },
    "8c4b9843-a0e1-4306-aff3-1a8dca4954a0": {
      "_type": "FiniteField",
      "data": {
        "def_pol": {
          "_type": {
            "name": "PolyRingElem",
            "params": "c22521f2-ddaf-4125-9c34-c5294f59602d"
          },
          "data": [
            [
              "0",
              "3"
            ],
            [
              "1",
              "6"
            ],
            [
              "2",
              "1"
            ]
          ]
        }
      }
    },
    "c22521f2-ddaf-4125-9c34-c5294f59602d": {
      "_instance": "FqPolyRing",
      "_type": "PolyRing",
      "data": {
        "base_ring": "fd0fa33d-250c-46e6-8d11-0ff6969625bd",
        "symbols": [
          "x"
        ]
      }
    },
    "fd0fa33d-250c-46e6-8d11-0ff6969625bd": {
      "_type": "FiniteField",
      "data": "7"
    }
  },
  "_type": {
    "name": "MPolyRingElem",
    "params": "0377c284-0b31-4d83-b009-0776e883e80b"
  },
  "data": [
    [
      [
        "3",
        "4"
      ],
      [
        [
          "0",
          "2"
        ]
      ]
    ],
    [
      [
        "1",
        "0"
      ],
      [
        [
          "1",
          "5"
        ]
      ]
    ],
    [
      [
        "0",
        "2"
      ],
      [
        [
          "0",
          "3"
        ],
        [
          "1",
          "1"
        ]
      ]
    ],
    [
      [
        "0",
        "0"
      ],
      [
        [
          "0",
          "1"
        ]
      ]
    ]
  ]
}

[antonydellavecchia]

with the latest commit example file looks like this

{
  "_ns": {
    "Oscar": [
      "https://github.com/oscar-system/Oscar.jl",
      "1.3.0-DEV-4d1a3ffa24b8e340470afa6b40c02150e3558b4e"
    ]
  },
  "_refs": {
    "2f5028cc-0fd6-4381-8f7e-0ff1520da04d": {
      "_instance": "FqPolyRing",
      "_type": "PolyRing",
      "data": {
        "base_ring": "d58dcad8-17fa-48aa-aaf9-3a3cd3f86fd4",
        "symbols": [
          "x"
        ]
      }
    },
    "5c637058-63cd-4773-84f1-965164c98ab5": {
      "_instance": "FqField",
      "_type": "FiniteField",
      "data": {
        "def_pol": {
          "_type": {
            "name": "PolyRingElem",
            "params": "2f5028cc-0fd6-4381-8f7e-0ff1520da04d"
          },
          "data": [
            [
              "0",
              "3"
            ],
            [
              "1",
              "6"
            ],
            [
              "2",
              "1"
            ]
          ]
        }
      }
    },
    "d58dcad8-17fa-48aa-aaf9-3a3cd3f86fd4": {
      "_instance": "FqField",
      "_type": "FiniteField",
      "data": "7"
    },
    "dc9554a8-0ed5-4504-9c21-7760d6c4d02c": {
      "_instance": "FqMPolyRing",
      "_type": "MPolyRing",
      "data": {
        "base_ring": "5c637058-63cd-4773-84f1-965164c98ab5",
        "symbols": [
          "y",
          "z"
        ]
      }
    }
  },
  "_type": {
    "name": "MPolyRingElem",
    "params": "dc9554a8-0ed5-4504-9c21-7760d6c4d02c"
  },
  "data": [
    [
      [
        "3",
        "4"
      ],
      [
        [
          "0",
          "2"
        ]
      ]
    ],
    [
      [
        "1",
        "0"
      ],
      [
        [
          "1",
          "5"
        ]
      ]
    ],
    [
      [
        "0",
        "2"
      ],
      [
        [
          "0",
          "3"
        ],
        [
          "1",
          "1"
        ]
      ]
    ],
    [
      [
        "0",
        "0"
      ],
      [
        [
          "0",
          "1"
        ]
      ]
    ]
  ]
}

[micjoswig]

We agreed that there should be an _instance entry every time where the many to one map is not bijective, even if suitable OSCAR defaults are in place.

With this addition the PR is good for merge IMHO.

[fingolfin]

What the status of this PR? (Besides the merge issues, the last comment was in November 2024)

[antonydellavecchia]

The example from paper will now look like this. (not all changes are from this upgrade, only the type info changes)

{
  "_ns": {
    "Oscar": [
      "https://github.com/oscar-system/Oscar.jl",
      "1.4.0-DEV-0fd6fd83395a5818eba53daf541e9d9ecd6a9b58"
    ]
  },
  "_type": {
    "name": "MPolyRingElem",
    "params": "869a359a-43d3-43f4-9821-0af9346be019"
  },
  "data": [[["3", "4"], [["0", "2"]]],
           [["0", "2"], [["0", "3"], ["1", "1"]]],
           [["1", "0"], [["1", "5"]]],
           [["0", "0"], [["0", "1"]]]],
  "_refs": {
    "869a359a-43d3-43f4-9821-0af9346be019": {
      "_type": {
        "name": "MPolyRing",
        "params": "a8309b96-caec-443c-bedb-e23bb0634c14"
      },
      "data": {
        "symbols": [
          "y",
          "z"
        ]
      }
    },
    "a8309b96-caec-443c-bedb-e23bb0634c14": {
      "_type": {
        "_instance": "FqField",
        "name": "FiniteField",
        "params": "152ac7bd-e85a-4b36-acc2-743ade2cad4f"
      },
      "data": [
        [
          "0",
          "1"
        ],
        [
          "2",
          "1"
        ]
      ]
    },
    "152ac7bd-e85a-4b36-acc2-743ade2cad4f": {
      "_type": {
        "name": "PolyRing",
        "params": "9ec94393-c512-4a49-8fd6-81cf06a9a7e5"
      },
      "data": {
        "symbols": [
          "x"
        ]
      }
    },
    "9ec94393-c512-4a49-8fd6-81cf06a9a7e5": {
      "_type": {
        "name": "FiniteField",
        "_instance": "FqField"
      },
      "data": "7"
    }
  }
}

[codecov]

Codecov Report

Attention: Patch coverage is 93.44262% with 4 lines in your changes missing coverage. Please review.

Project coverage is 84.87%. Comparing base (8641dfc) to head (8fd390c).
Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
src/Serialization/Upgrades/0.11.3.jl	90.00%	2 Missing ⚠️
src/Serialization/main.jl	92.59%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4331      +/-   ##
==========================================
- Coverage   84.88%   84.87%   -0.01%     
==========================================
  Files         683      683              
  Lines       91859    91883      +24     
==========================================
+ Hits        77976    77990      +14     
- Misses      13883    13893      +10     

Files with missing lines	Coverage Δ
src/Serialization/Fields.jl	91.53% <ø> (ø)
src/Serialization/PolyhedralGeometry.jl	89.28% <ø> (ø)
src/Serialization/Rings.jl	96.53% <ø> (ø)
src/Serialization/Upgrades/0.12.0.jl	92.85% <100.00%> (-0.25%)	⬇️
src/Serialization/Upgrades/1.4.0.jl	84.04% <100.00%> (+0.57%)	⬆️
src/Serialization/serializers.jl	98.43% <100.00%> (+0.01%)	⬆️
src/Serialization/Upgrades/0.11.3.jl	84.21% <90.00%> (-1.51%)	⬇️
src/Serialization/main.jl	87.27% <92.59%> (+0.54%)	⬆️

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[benlorenz]

A few small remarks, otherwise this should be good to merge.