Skip to content

MlFlow gives error when loging in additional mitric as dice per label  #697

Closed
Project-MONAI/MONAI
#8169
@AHarouni

Description

@AHarouni

Describe the bug

Following instructions to have bundle work with mlflow, I managed to get it to work and it logs the metrics. however, when I add additional metric specifically dice per label I get error below

To Reproduce
Steps to reproduce the behavior:

  1. Follow setup for mlflow with bundles
  2. make sure it works correctly for a segmentation task
  3. add section in the validatation as
        "additional_metrics": {
            "val_dice_lb": {
                "_target_": "MeanDice",
                "include_background": true,
                "output_transform": "$monai.handlers.from_engine(['pred', 'label'])",
                "num_classes": "@output_classes",
                "return_with_label": true,
                "reduction": "mean_batch"
            }
        },
  1. run training bundle again to get error below
File "/usr/local/lib/python3.10/dist-packages/mlflow/utils/validation.py", line 150, in _validate_metric
    raise MlflowException(
mlflow.exceptions.MlflowException: Got invalid value {'label_0': 0.9401, 'label_1': 0.0, 'label_2': 0.0, 'label_3': 0.0} for metric 'val_dice_lb' (timestamp=1727801874854). Please specify value as a valid double (64-bit floating point)

Expected behavior
bundle should run as mlflow should have multiple metrics for each dice label

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions