API: __getitem__[int] vs __setitem__[int] with Float64Index

[jbrockmendel]

ser = pd.Series([1, 2, 3, 4], index=[1.1, 2.1, 3.0, 4.1])

>>> ser[5]                    # <--we are treating 5 as a label
KeyError: 5

>>> ser[5] = 5                # < --we are treating 5 as position
IndexError: index 5 is out of bounds for axis 0 with size 4

>>> ser[3] = 5                # <-- we are treating 3 as a label
>>> ser
1.1    1
2.1    2
3.0    5
4.1    4
dtype: int64

The ser[5] = 5 case is an outlier because instead of having its label-vs-positional behavior determined by ser.index._should_fallback_to_positional, it is defermined by if is_integer(key) and not self.index.inferred_type == "integer":

[jorisvandenbossche]

This API is certainly a mess .. ;)

The proposal is to change the setitem case where it fallbacks to positional to also be strictly label-based?

But so the integer would also be interpreted as label, and thus do enlargement in case it is not yet present?

Using your example series:

In [40]: ser
Out[40]: 
1.1    1
2.1    2
3.0    3
4.1    4
dtype: int64

In [41]: ser[2] = 100

In [42]: ser
Out[42]: 
1.1      1
2.1      2
3.0    100
4.1      4
dtype: int64

In [43]: ser[2.0] = -100

In [44]: ser
Out[44]: 
1.1      1
2.1      2
3.0    100
4.1      4
2.0   -100
dtype: int64

I was first thinking this would be a relatively safe change, since it right now already tries label-based for setitem first (so if the integer key is present as a label, it would already do label-based now). So the potential difference in behaviour is only when the label is not present. Right now that can update a value, but in the future add another value like the above? Which is kind of silently resulting in different data. We could also deprecate integer keys for setitem first or raise an error for it?

[jbrockmendel]

We could also deprecate integer keys for setitem first or raise an error for it?

Do you mean "deprecate ever treating integer as a label for Float64Index"? Or "deprecate allowing ser[integer] with Float64Index"? The first seems reasonable.

I'd be OK with "integers are always considered labels" or "integers are always considered positional", so there is no question of fall-back behavior.

[jbrockmendel]

Options that come to mind for fixing this:

1. Deprecate casting ints to floats in Float64Index.get_loc, so that ser[4] is always positional when we have a Float64Index.

2. Change Float64Index._should_fallback_to_positional to return not (self == self.astype(int)).any() (ATM it always returns False).

   • Upsides: Fixes the OP issue without breaking any tests, fixes the inconsistency of how we check for fallback in Series.__setitem__ vs Series.__getitem__
   • Downsides: perf (caching would help some), value-dependent behavior

3. Change Float64Index._should_fallback_to_positional to always return True.

   • Breaks 5 tests, 4 of which are checking that we raise, the last is trivial to fix.