Generalize GridPatchDataset and PatchDataset. Potentially introduce Persistent and/or Cache IterableDatasets.

[ibro45]

Is your feature request related to a problem? Please describe.

• GridPatchDataset's data argument must be a sequence of already loaded data. However, it would be consistent with other datasets to allow loading and manipulating other types of data through transform. For example, you might want to provide a dict of image and label paths and load them through transforms using LoadImaged().
• PatchDataset should follow the same. Moreover, it has a transform argument that is actually the same as patch_transform in GridPatchDataset, so it should follow that naming.
• Finally, it would be useful to have Persistent or Cache versions of these (or just any IterableDataset) - is that something that you would want to have?

Describe the solution you'd like

• GridPatchDataset currently has patch_transform, but should have two transforms: transform and patch_transform. The transform is to be used the same as in the IterableDataset and is applied prior to patching, while patch_transform is applied to the patch.
• PatchDataset should follow the same.

Describe alternatives you've considered
None

Additional context
I started looking into these patch-based datasets because I want to perform validation on large images during my training. Specifically, images are sometimes too big to perform SlidingWindowInferer on GPU, and using cpu_thresh just results in a very long computation. I hope to use this as a replacement for it, and it should be way faster if Persistent/Cache mode is available too. It basically emulates sliding window inference - still allows you to split into patches with some overlap and pass them through the network to get the prediction, but with a caveat that the predicted patches are never merged and instead evaluated against the corresponding label patches individually.

I am happy to make a PR for the first two points. I am also interested in implementing the third point if you agree with it and if we discuss how we should approach it.

[KumoLiu]

Hi, @ibro45, thanks for your proposal!

• Is it possible to use GridPatchDataset right after a Dataset? Something similar in this tutorial.
• It seems to be transform in both GridPatchDataset and PatchDataset
  

  MONAI/monai/data/grid_dataset.py

  Line 185 in 2daabf9

  transform: a callable data transform operates on the patches.

  
  

  MONAI/monai/data/grid_dataset.py

  Line 270 in 2daabf9

  transform: transform applied to each patch.

• For the third one, is it similar to the first one, we can also have GridPatchDataset right after PersistentantDataset or CacheDataset? A similar tutorial is here.
  https://github.com/Project-MONAI/tutorials/blob/87792475f4476dcc63f92e47e109dde18cdfb30d/modules/2d_slices_from_3d_training.ipynb#L22

Thanks!

[wyli]

there are multiple feature requests for caching:

• Use GridPatchDataset with caching features #5936
• Make GridPatchDataset cacheable by adding shapes parameter #5662

I think we should add this feature @ericspod @KumoLiu @Nic-Ma

see also related report #6585

[ericspod]

I think the patch dataset classes can definitely be improved though I wonder how much this is now overlapping with the WSI dataset and reader concepts. Unlike WSI the patch dataset classes load whole files and sample from them, but otherwise there's some similarity. I do see some inefficiencies such as with caching and continually indexing datasets, looking at the WSI implementation could give us some better ideas of what to do improve here. CC @drbeh

I'm adding this ticket to our backlog and I'd suggest we look at this for 1.4 as we're trying for a faster 1.3 release.