Skip to content

Cannot add TimedeltaIndex to tz-aware DatetimeIndex (but can to non-tz-aware DatetimeIndex) #14725

Closed
@Liam3851

Description

@Liam3851
Contributor

Code Sample, a copy-pastable example if possible

Under 0.19.1 and latest master:

In [19]: dr = pd.date_range("20161123 00:00", "20161123 02:00", freq="H")                           
                                                                                                    
In [20]: tdr = pd.timedelta_range("1 minute", "3 minutes", freq="T")                                
                                                                                                    
In [21]: dr + tdr                                                                                   
Out[21]:                                                                                            
DatetimeIndex(['2016-11-23 00:01:00', '2016-11-23 01:02:00',                                        
               '2016-11-23 02:03:00'],                                                              
              dtype='datetime64[ns]', freq='61T')                                                   
                                                                                                    
In [22]: dr_tz = pd.date_range("20161123 00:00", "20161123 02:00", freq="H", tz="America/New_York") 
    ...:                                                                                            
                                                                                                    
In [23]: dr_tz + tdr                                                                                
---------------------------------------------------------------------------                         
TypeError                                 Traceback (most recent call last)                         
<ipython-input-23-c10d4a537816> in <module>()                                                       
----> 1 dr_tz + tdr                                                                                 
                                                                                                    
C:\Anaconda\envs\bleeding\lib\site-packages\pandas\tseries\base.pyc in __add__(self, other)         
    617             from pandas.tseries.offsets import DateOffset                                   
    618             if isinstance(other, TimedeltaIndex):                                           
--> 619                 return self._add_delta(other)                                               
    620             elif isinstance(self, TimedeltaIndex) and isinstance(other, Index):             
    621                 if hasattr(other, '_add_delta'):                                            
                                                                                                    
C:\Anaconda\envs\bleeding\lib\site-packages\pandas\tseries\index.pyc in _add_delta(self, delta)     
    787             new_values = self._add_delta_td(delta)                                          
    788         elif isinstance(delta, TimedeltaIndex):                                             
--> 789             new_values = self._add_delta_tdi(delta)                                         
    790             # update name when delta is Index                                               
    791             name = com._maybe_match_name(self, delta)                                       
                                                                                                    
C:\Anaconda\envs\bleeding\lib\site-packages\pandas\tseries\base.pyc in _add_delta_tdi(self, other)  
    704             mask = (self._isnan) | (other._isnan)                                           
    705             new_values[mask] = tslib.iNaT                                                   
--> 706         return new_values.view(self.dtype)                                                  
    707                                                                                             
    708     def isin(self, values):                                                                 
                                                                                                    
TypeError: data type not understood                             

Problem description

Given a DatetimeIndex that is not tz-aware, adding a TimedeltaIndex of the same length adds the values elementwise, as one would expect. Given a tz-aware DatetimeIndex, however, adding a TimedeltaIndex results in TypeError. Performing the same operation by iterating over the tz-aware index and adding the values directly works in the expected manner. This behavior did not occur under pandas 0.16.2; however, the above behavior occurs under at least 0.19.1 and the latest master.

Expected Output

Equivalent to the elementwise operation below, which works on latest master:

In [25]: pd.Index([dr_tz[i] + tdr[i] for i in xrange(len(dr_tz))])                                  
Out[25]:                                                                                            
DatetimeIndex(['2016-11-23 00:01:00-05:00', '2016-11-23 01:02:00-05:00',                            
               '2016-11-23 02:03:00-05:00'],                                                        
              dtype='datetime64[ns, America/New_York]', freq=None)                                  

This also worked under 0.16.2:

In [14]: dr_tz = pd.date_range("20161123 00:00", "20161123 02:00", freq="H", tz="America/New_York") 
                                                                                                    
In [15]: tdr = pd.timedelta_range("1 minute", "3 minutes", freq="T")                                
                                                                                                    
In [16]: dr_tz + tdr                                                                                
Out[16]:                                                                                            
DatetimeIndex(['2016-11-23 00:01:00-05:00', '2016-11-23 01:02:00-05:00',                            
               '2016-11-23 02:03:00-05:00'],                                                        
              dtype='datetime64[ns]', freq='61T', tz='America/New_York')     

Output of pd.show_versions()

In [26]: pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.19.0+98.gc045e1d
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.24.1
numpy: 1.11.2
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 5.1.0
sphinx: 1.4.8
patsy: None
dateutil: 2.5.3
pytz: 2016.7
blosc: None
bottleneck: None
tables: 3.2.2
numexpr: 2.6.1
matplotlib: 1.5.3
openpyxl: 2.4.0
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: None
lxml: 3.6.4
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None

Activity

sinhrks

sinhrks commented on Nov 23, 2016

@sinhrks
Member

xref: #13905

jreback

jreback commented on Nov 24, 2016

@jreback
Contributor

closing as duplicate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @Liam3851@jreback@sinhrks

        Issue actions

          Cannot add TimedeltaIndex to tz-aware DatetimeIndex (but can to non-tz-aware DatetimeIndex) · Issue #14725 · pandas-dev/pandas