Skip to content

Unexpected type binding when wrapping functions #12919

Closed
@mailund

Description

@mailund

Bug Report

I admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where mypy and pyre disagree, so it might be a bug.

I am writing code that lifts functions T -> R to Optional[T] -> Optional[R] in various ways, and when I write wrapper functions, type variables get bound in ways I didn't expect. It is particularly bad when I add protocols for various types on top of it, but I can create the unexpected behaviour without it. Basically, the issue is that if I write a function that changes a Callable[[T], R] into a Callable[[Optional[T]], Optional[R]], and then apply such a function to another generic function Callable[[T],R] the type variables are no longer generic, and I cannot apply the wrapped function. This happens both when I work with wrapper functions or when I implement the wrapper as a Generic class.

To Reproduce

If I write a wrapper with a function, it might look like this:

from typing import (
    TypeVar, 
    Protocol, Generic, 
    Optional as Opt
)


_T = TypeVar('_T')

class _F(Protocol[_T]):
    def __call__(self, __x: _T) -> _T:
        ...

# Lifting with a function...
def lift(f: _F[_T]) -> _F[Opt[_T]]:
    # This function would do more, but I just return f for the
    # type checking example
    return f # type: ignore

I get the desired behaviour if I wrap a function with a concrete type

def f(x: int) -> int:
    return x

reveal_type(lift(f))  # _F[Opt[int]] -- as expected

but if I use a generic type, the wrapped function is not generic:

def g(x: _T) -> _T:
    return x

reveal_type(lift(g)) # _F[Union[_T`-1,None]] -- _T is bound (incorrectly?)
                     # pyre thinks this is _F[None] which is also odd...
lift(g)(1)  # so won't accept int

Expected Behavior

I was expecting the wrapped function to have the type Callable[[Opt[_T]], Opt[_T]] and in particular I would expect to be able to call it with an int argument.

To Reproduce

If I use a Generic class for the lifted function, it can look like this:

# Moving generic type to Generic...
class lifted(Generic[_T]):
    """Lift type _T to Opt[_T]."""

    def __init__(self, f: _F[_T]) -> None:
        self._f = f

    def __call__(self, x: Opt[_T], /) -> Opt[_T]:
        """Call lifted."""
        return self._f(x) # type: ignore

reveal_type(lifted[_T])  # _F[T?] -> lifted[_T?]

and it will work for a function with a concrete type, like f: int -> int above

reveal_type(lifted(f))          # lifted[int] -- perfect
reveal_type(lifted(f).__call__) # Opt[int] -> Opt[int] -- perfect
lifted(f)(1)                    # We can call with int
lifted(f)(None)                 # We can call with None

but things get crazy when I use a generic function like g: _T -> _T:

reveal_type(lifted(g))            # lifted[_T`1] -- _T is bound!
                                  # pyre thinks (correctly I think) it is lifted[_T]
reveal_type(lifted(g).__call__)   # Opt[_T`1] -> Opt[_T`1]
                                  # pyre says Any -> Any which I suppose is fine
lifted(g)(1)                      # incompatible type, int isn't an Opt[_T]
                                  # pyre seems to accep this one
lifted(g)(None)                   # but at least None is in Opt[_T]...

reveal_type(lifted[_T](g))          # lifted[_T?] -- Ok
reveal_type(lifted[_T](g).__call__) # def (None) ???
                                    # pyre: Opt[_T] -> Opt[_T] as expected

lifted[_T](g)(1)             # incompatible type "int"; expected "None"
lifted[_T](g)(None)          # but at least None works...
# Pyre doesn't like the [_T] here but accepts the calls without it...

Expected Behavior

Again, I would expect the Generic lifted class to accept types that match the wrapped function's type. This is more important when I need to add protocols to the types, but I would also expect it to work here.

Your Environment

I tested this in mypy playground and pyre playground

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugmypy got something wrong

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions