Skip to content

ENH: Series Creation Does Not Intelligently Handle datetime.date #4338

@cancan101

Description

@cancan101
Contributor

For example:

In [77]: pd.Series([datetime.datetime(2012, 1, 1)]).dtype
Out[77]: dtype('<M8[ns]')

works as expected.

However:

In [78]: pd.Series([datetime.date(2012, 1, 1)]).dtype
Out[78]: dtype('O')

create a Series of type object rather than datetime64.

You can get the desired effect:

In [89]: pd.Series(np.array([np.datetime64(datetime.date(2012, 1, 1))])).dtype
Out[89]: dtype('<M8[D]')

Activity

jreback

jreback commented on Jul 24, 2013

@jreback
Contributor

there is nothing wrong with datetime.date, but it REALLY should be passed as a datetime.datetime. Its very hard to do this 'automatically' because there is nothing technically wrong with allowing it in (its just dtyped == object).

OTOH maybe should just do this conversion .....

and your dtype in 89 will just not work; its not a valid dtype (well in 0.11 it causes subtle issues), in 0.12 I think this will raise (but'll I'll add this as an enhancement issue)

cancan101

cancan101 commented on Jul 24, 2013

@cancan101
ContributorAuthor

Why do you think that it should be passed in as datetime.datetime rather than datetime.date? There is something to be said about maintaining the precision of the observation as being an entire day rather than a specific point in time.

Numpy theoretically supports this notion with time units: http://docs.scipy.org/doc/numpy-dev/reference/arrays.datetime.html#datetime-units

cancan101

cancan101 commented on Nov 5, 2013

@cancan101
ContributorAuthor

Here is perhaps a cleaner example. I do think that auto conversion COULD be performed:

pd.DataFrame([[date(2012,1,1)]]).dtypes
Out[14]: 
0    object
dtype: object

pd.DataFrame([[datetime.combine(date(2012,1,1), time(0,0,0))]]).dtypes
Out[15]: 
0    datetime64[ns]
dtype: object
jtratner

jtratner commented on Nov 5, 2013

@jtratner
Contributor

You can put something together for 0.14 if you want

cancan101

cancan101 commented on Nov 5, 2013

@cancan101
ContributorAuthor

Sounds good. I imagine this should not be too hard.
On Nov 4, 2013 9:38 PM, "Jeff Tratner" notifications@github.com wrote:

You can put something together for 0.14 if you want


Reply to this email directly or view it on GitHubhttps://github.com//issues/4338#issuecomment-27743286
.

jtratner

jtratner commented on Nov 5, 2013

@jtratner
Contributor

I know @jreback has had a pretty good sense of what we want to support for
dates, so I'm going to defer to him on this though.

modified the milestones: 0.15.0, 0.14.0 on Feb 18, 2014
modified the milestones: 0.16.0, Next Major Release on Mar 3, 2015
max-sixty

max-sixty commented on Oct 23, 2015

@max-sixty
Contributor

Is this still active?
@jtratner mentioned @jreback had a design in mind - is that the case?

I'm hitting this in our work - loading a set of datetime.date objects, and getting an object rather than datetime index:

In [75]:

series=pd.Series(range(5), index=[datetime.date.today() + datetime.timedelta(i) for i in range(5)])
series
Out[75]:
2015-10-23    0
2015-10-24    1
2015-10-25    2
2015-10-26    3
2015-10-27    4
dtype: int64
In [76]:

series.index
Out[76]:
Index([2015-10-23, 2015-10-24, 2015-10-25, 2015-10-26, 2015-10-27], dtype='object')

Replacing date with datetime:

In [81]:

series=pd.Series(range(5), index=[datetime.datetime.today() + datetime.timedelta(i) for i in range(5)])
series
Out[81]:
2015-10-23 20:39:36.077137    0
2015-10-24 20:39:36.077156    1
2015-10-25 20:39:36.077169    2
2015-10-26 20:39:36.077174    3
2015-10-27 20:39:36.077178    4
dtype: int64
In [82]:

series.index
series.index
Out[82]:
DatetimeIndex(['2015-10-23 20:39:36.077137', '2015-10-24 20:39:36.077156',
               '2015-10-25 20:39:36.077169', '2015-10-26 20:39:36.077174',
               '2015-10-27 20:39:36.077178'],
              dtype='datetime64[ns]', freq=None)
jreback

jreback commented on Oct 23, 2015

@jreback
Contributor

@MaximilianR

datetime.date could be auto-converted to datetime64[ns] though that would be an API change.

handling datetime.date as an actual dtype, `datetime64[D]``. is possible, though would require a fair bit of work.

It is not on the priority list, though you are welcome to take a look.

modified the milestone: Contributions Welcome on Jul 8, 2018
TomAugspurger

TomAugspurger commented on Oct 25, 2018

@TomAugspurger
Contributor

I don't think we should auto-convert datetime.date to datetimes. They're different things. In the future, we (or a 3rd party) may make an ExtensionArray for dates, and we would have to backtrack this change.

If the user wants datetimes now, then they have pd.to_datetime.

modified the milestone: Someday on Oct 26, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @cancan101@jreback@jorisvandenbossche@TomAugspurger@jtratner

        Issue actions

          ENH: Series Creation Does Not Intelligently Handle datetime.date · Issue #4338 · pandas-dev/pandas