Skip to content

ENH: Introduce type-safe constructors for Timestamp and Timedelta. #58475

Open
@randolf-scholz

Description

@randolf-scholz
Contributor

Feature Type

  • Adding new functionality to pandas

    Changing existing functionality in pandas

    Removing existing functionality in pandas

Problem Description

The default constructors pd.Timestamp.__new__ and pd.Timedelta.__new__ can return NaT, which is a different type. This can lead to silent type errors, depending on the type-checker used. Consider the following example:

import numpy as np
from pandas import Timedelta, Timestamp

t: Timestamp = Timestamp("2024-04-29T18:00:00")
t2: Timestamp = Timestamp(np.datetime64("nat"))  # actually NaTType!
dt: Timedelta = Timedelta(1, "h")
dt2: Timedelta = Timedelta(np.timedelta64("nat"))  # actually NaTType!

Type-checking results:

  • mypy --strict: No errors (w/ and w/o pandas-stubs)
  • pyright with useLibraryCodeForTypes = true: 4 errors (only formally correct result)
  • pyright with useLibraryCodeForTypes = false: no errors

Feature Description

Introduce new constructors timestamp and timedelta (in analogy to how pyarrow does constructors), which are guaranteed to return pd.Timestamp and pd.Timedelta types, or raise an exception in the case when NaT is encountered.

Alternative Solutions

Split pd.NaT into two different types, Timestamp("NaT") and Timedelta("NaT") (as is the case in numpy), which are instances of the respective types. (#24983)

Additional Context

Activity

randolf-scholz

randolf-scholz commented on Apr 29, 2024

@randolf-scholz
ContributorAuthor

These constructors can be very simple wrappers, a rough sketch:

def timedelta(value: Any = ..., unit: Optional[str] = None, **kwargs: Any) -> Timedelta:
    """Utility function that ensures that the constructor does not return NaT."""
    td = (
        Timedelta(unit=unit, **kwargs)
        if value is Ellipsis
        else Timedelta(value, unit=unit, **kwargs)
    )
    if isinstance(td, NaTType):
        raise ValueError("Constructor returned NaT")
    return td


def timestamp(value: Any = ..., **kwargs: Any) -> Timestamp:
    """Utility function that ensures that the constructor does not return NaT."""
    ts = Timestamp(**kwargs) if value is Ellipsis else Timestamp(value, **kwargs)
    if isinstance(ts, NaTType):
        raise ValueError("Constructor returned NaT")
    return ts
jbrockmendel

jbrockmendel commented on Apr 29, 2024

@jbrockmendel
Member

There’s an issue about introducing a separate NaTD specific to Timedelta. If you did that ( and the same for Period), then NaT could become a Timestamp, and you would get type-safety in the constructors without new constructors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    EnhancementNeeds TriageIssue that has not been reviewed by a pandas team member

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @jbrockmendel@randolf-scholz

        Issue actions

          ENH: Introduce type-safe constructors for `Timestamp` and `Timedelta`. · Issue #58475 · pandas-dev/pandas