Skip to content

UTC Timestamps constructed from datetime.datetime (with datetime.timezone.utc) cannot be compared to pandas Timestamps #22133

Closed
@skaae

Description

@skaae

Code Sample, a copy-pastable example if possible

import pandas as pd
import datetime
ts_to_datetime = pd.to_datetime(datetime.datetime(2018, 3, 8, 16, 58, 41, 182221, tzinfo=datetime.timezone.utc))

ts_utcnow = pd.Timestamp.utcnow()

ts_timestamp = pd.Timestamp(datetime.datetime(2018, 3, 8, 16, 58, 41, 182221, tzinfo=datetime.timezone.utc))

ts_to_datetime - ts_utcnow  # Timestamp subtraction must have the same timezones or no timezones
ts_timestamp - ts_utcnow # same error
ts_to_datetime - ts_timestamp # ok 

Problem description

Timestamps constructed from datatime.datetime objects with UTC timezone have tz that is type datetime.timezone.utc but pd.Timestamp.utwnow() have tz that is pytz.UTC.

Expected Output

In [2]: ts_to_datetime - ts_utcnow
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-5564b4c82b42> in <module>()
----> 1 ts_to_datetime - ts_utcnow

pandas/_libs/tslibs/timestamps.pyx in pandas._libs.tslibs.timestamps._Timestamp.__sub__()

TypeError: Timestamp subtraction must have the same timezones or no timezones

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.0.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.3
pytest: 3.0.5
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.13.3
scipy: 0.19.1
pyarrow: 0.7.1
xarray: 0.9.6
IPython: 5.1.0
sphinx: 1.5.1
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: 0.4.0
matplotlib: 2.0.2
openpyxl: 2.4.1
xlrd: 1.1.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.2
bs4: 4.5.3
html5lib: 0.9999999
sqlalchemy: 1.1.5
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Activity

mroeschke

mroeschke commented on Jul 30, 2018

@mroeschke
Member

xref #17173

The main issue is that pandas do not yet support datetime.timezone objects. Currently pandas only supports pytz or dateutil timezone objects.

I anticipate pandas supporting datetime.timezone objects soon enough since it's part of the standard library and since pandas will drop Python 2 support, but it will require quite some work. Pull requests are always welcome!

changed the title [-]UTC Timestamps constructed from datetime.datetime cannot be compared to pandas Timestamps[/-] [+]UTC Timestamps constructed from datetime.datetime (with datetime.timezone.utc) cannot be compared to pandas Timestamps[/+] on Jul 30, 2018
added this to the No action milestone on Jul 31, 2018
jreback

jreback commented on Jul 31, 2018

@jreback
Contributor

as indicated this is need of implementation ; closing as a duplicate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @jreback@skaae@mroeschke

        Issue actions

          UTC Timestamps constructed from datetime.datetime (with datetime.timezone.utc) cannot be compared to pandas Timestamps · Issue #22133 · pandas-dev/pandas