subset_data does not function for regional grids that span across Greenwich longitude zero

[ekluzek]

Brief summary of bug

When you tell subset_data to give you a region that goes across zero longitude.

General bug information

CTSM version you are using: ctsm5.1.dev125-192-gec92580a3

Does this bug cause significantly incorrect results in the model's science? No

Configurations affected: Regions where lon1 > lon2 on a 0 to 360 longitude grid

Details of bug

It dies first in check_region_lons in regional_case.py, but dies in xarray for an indexing operation
if the first abort is removed.

Important details of your setup / configuration so we can reproduce the bug

[Specify anything relevant: the compset, resolution, machine, compiler, any xml or namelist changes, etc. You don't have to repeat anything that you have already noted above.]

Important output or errors that show the problem

TSMscript_tools.sh: running /glade/work/erik/ctsm_worktrees/neon/tools/site_and_regional/subset_data with region --lat1 0 --lat2 40 --lon1 -20 --lon2 40 --create-mesh --create-surface --create-user-mods --create-domain --create-landuse --verbose --overwrite --reg f09_NorthAfrica --inputdata-dir /glade/p/cesmdata/cseg/inputdata; output in /glade/scratch/erik/clmTests/test-driver.35924/TSMscript_tools.site_and_regional.subset_data.subset_data_f09_NorthAfrica/test.log
TSMscript_tools.sh: error running subset_data, error= 1
TSMscript_tools.sh: see /glade/scratch/erik/clmTests/test-driver.35924/TSMscript_tools.site_and_regional.subset_data.subset_data_f09_NorthAfrica/test.log for details
end of input
(ctsm_pylib) test/tools> cat /glade/scratch/erik/clmTests/test-driver.35924/TSMscript_tools.site_and_regional.subset_data.subset_data_f09_NorthAfrica/test.log 
INFO: User = erik
INFO: Current directory = /glade/scratch/erik/clmTests/test-driver.35924/TSMscript_tools.site_and_regional.subset_data.subset_data_f09_NorthAfrica
INFO: ----------------------------------------------------------------------------
INFO: This script extracts a region from the global CTSM datasets.
INFO: fdomain_in:  /glade/p/cesmdata/cseg/inputdata/share/domains/domain.lnd.fv0.9x1.25_gx1v7.151020.nc
INFO: fdomain_out: /glade/scratch/erik/clmTests/test-driver.35924/TSMscript_tools.site_and_regional.subset_data.subset_data_f09_NorthAfrica/subset_data_regional/domain.lnd.fv0.9x1.25_gx1v7_f09_NorthAfrica_c230606.nc
INFO: Creating domain file at region: f09_NorthAfrica
Traceback (most recent call last):
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 85, in _getitem
    array = getitem(original_array, key)
  File "src/netCDF4/_netCDF4.pyx", line 4383, in netCDF4._netCDF4.Variable.__getitem__
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/netCDF4/utils.py", line 458, in _out_array_shape
    c = count[..., i].ravel()[0] # All elements should be identical.
IndexError: index 0 is out of bounds for axis 0 with size 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/glade/work/erik/ctsm_worktrees/neon/tools/site_and_regional/subset_data", line 37, in <module>
    main()
  File "/glade/work/erik/ctsm_worktrees/neon/tools/site_and_regional/../../python/ctsm/subset_data.py", line 733, in main
    subset_region(args, file_dict)
  File "/glade/work/erik/ctsm_worktrees/neon/tools/site_and_regional/../../python/ctsm/subset_data.py", line 662, in subset_region
    region.create_domain_at_reg(file_dict["main_dir"], file_dict["fdomain_in"])
  File "/glade/work/erik/ctsm_worktrees/neon/tools/site_and_regional/../../python/ctsm/site_and_regional/regional_case.py", line 214, in create_domain_at_reg
    self.write_to_netcdf(f_out, wfile)
  File "/glade/work/erik/ctsm_worktrees/neon/tools/site_and_regional/../../python/ctsm/site_and_regional/base_case.py", line 232, in write_to_netcdf
    xr_ds.to_netcdf(path=nc_fname, mode="w", format="NETCDF3_64BIT")
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/dataset.py", line 1654, in to_netcdf
    invalid_netcdf=invalid_netcdf,
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/api.py", line 1112, in to_netcdf
    dataset, store, writer, encoding=encoding, unlimited_dims=unlimited_dims
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/api.py", line 1158, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/common.py", line 250, in store
    variables, attributes = self.encode(variables, attributes)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/common.py", line 339, in encode
    variables, attributes = cf_encoder(variables, attributes)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/conventions.py", line 771, in cf_encoder
    _update_bounds_encoding(variables)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/conventions.py", line 413, in _update_bounds_encoding
    ) or contains_cftime_datetimes(v)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/common.py", line 1760, in contains_cftime_datetimes
    return _contains_cftime_datetimes(var.data)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/variable.py", line 361, in data
    return self.values
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/variable.py", line 512, in values
    return _as_array_or_item(self._data)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/variable.py", line 274, in _as_array_or_item
    data = np.asarray(data)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/indexing.py", line 693, in __array__
    self._ensure_cached()
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/indexing.py", line 690, in _ensure_cached
    self.array = NumpyIndexingAdapter(np.asarray(self.array))
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/indexing.py", line 663, in __array__
    return np.asarray(self.array, dtype=dtype)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/indexing.py", line 568, in __array__
    return np.asarray(array[self.key], dtype=None)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 73, in __getitem__
    key, self.shape, indexing.IndexingSupport.OUTER, self._getitem
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/core/indexing.py", line 853, in explicit_indexing_adapter
    result = raw_indexing_method(raw_key.tuple)
  File "/glade/work/erik/conda-envs/ctsm_pylib/lib/python3.7/site-packages/xarray/backends/netCDF4_.py", line 95, in _getitem
    raise IndexError(msg)
IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load().

[ekluzek]

I think this might work if the longitude grid is converted for these cases from 0 to 360 to -180 to 180. From other work I think having negative longitudes might be OK, but the caveat is that answers will be at least roundoff different from 0 to 360 cases.

Allowing subset_data to convert to a -180 to 180 grid will most likely take a bit of work, but may not be that hard to do.

[ekluzek]

This is an important capability, but we won't be able to work on this immediately.

[ekluzek]

The current work around is to create two grids one on each side of the zero longitude line.

[wwieder]

Unless there's a quick fix for this, I suggest we don't try and address this issue for the CLM6 release, but note the limitation of regional capabilities in our documentation