Closed
Description
Describe the bug
Following instructions to have bundle work with mlflow, I managed to get it to work and it logs the metrics. however, when I add additional metric specifically dice per label I get error below
To Reproduce
Steps to reproduce the behavior:
- Follow setup for mlflow with bundles
- make sure it works correctly for a segmentation task
- add section in the validatation as
"additional_metrics": {
"val_dice_lb": {
"_target_": "MeanDice",
"include_background": true,
"output_transform": "$monai.handlers.from_engine(['pred', 'label'])",
"num_classes": "@output_classes",
"return_with_label": true,
"reduction": "mean_batch"
}
},
- run training bundle again to get error below
File "/usr/local/lib/python3.10/dist-packages/mlflow/utils/validation.py", line 150, in _validate_metric
raise MlflowException(
mlflow.exceptions.MlflowException: Got invalid value {'label_0': 0.9401, 'label_1': 0.0, 'label_2': 0.0, 'label_3': 0.0} for metric 'val_dice_lb' (timestamp=1727801874854). Please specify value as a valid double (64-bit floating point)
Expected behavior
bundle should run as mlflow should have multiple metrics for each dice label
Metadata
Metadata
Assignees
Labels
No labels