Description
Bug Report
I admit that I do not know if this is a bug, and not just my incomplete understanding of how the type system works, but I have run into some unexpected behaviour, and behaviour where mypy and pyre disagree, so it might be a bug.
I am writing code that lifts functions T -> R
to Optional[T] -> Optional[R]
in various ways, and when I write wrapper functions, type variables get bound in ways I didn't expect. It is particularly bad when I add protocols for various types on top of it, but I can create the unexpected behaviour without it. Basically, the issue is that if I write a function that changes a Callable[[T], R]
into a Callable[[Optional[T]], Optional[R]]
, and then apply such a function to another generic function Callable[[T],R]
the type variables are no longer generic, and I cannot apply the wrapped function. This happens both when I work with wrapper functions or when I implement the wrapper as a Generic
class.
To Reproduce
If I write a wrapper with a function, it might look like this:
from typing import (
TypeVar,
Protocol, Generic,
Optional as Opt
)
_T = TypeVar('_T')
class _F(Protocol[_T]):
def __call__(self, __x: _T) -> _T:
...
# Lifting with a function...
def lift(f: _F[_T]) -> _F[Opt[_T]]:
# This function would do more, but I just return f for the
# type checking example
return f # type: ignore
I get the desired behaviour if I wrap a function with a concrete type
def f(x: int) -> int:
return x
reveal_type(lift(f)) # _F[Opt[int]] -- as expected
but if I use a generic type, the wrapped function is not generic:
def g(x: _T) -> _T:
return x
reveal_type(lift(g)) # _F[Union[_T`-1,None]] -- _T is bound (incorrectly?)
# pyre thinks this is _F[None] which is also odd...
lift(g)(1) # so won't accept int
Expected Behavior
I was expecting the wrapped function to have the type Callable[[Opt[_T]], Opt[_T]]
and in particular I would expect to be able to call it with an int
argument.
To Reproduce
If I use a Generic
class for the lifted function, it can look like this:
# Moving generic type to Generic...
class lifted(Generic[_T]):
"""Lift type _T to Opt[_T]."""
def __init__(self, f: _F[_T]) -> None:
self._f = f
def __call__(self, x: Opt[_T], /) -> Opt[_T]:
"""Call lifted."""
return self._f(x) # type: ignore
reveal_type(lifted[_T]) # _F[T?] -> lifted[_T?]
and it will work for a function with a concrete type, like f: int -> int
above
reveal_type(lifted(f)) # lifted[int] -- perfect
reveal_type(lifted(f).__call__) # Opt[int] -> Opt[int] -- perfect
lifted(f)(1) # We can call with int
lifted(f)(None) # We can call with None
but things get crazy when I use a generic function like g: _T -> _T
:
reveal_type(lifted(g)) # lifted[_T`1] -- _T is bound!
# pyre thinks (correctly I think) it is lifted[_T]
reveal_type(lifted(g).__call__) # Opt[_T`1] -> Opt[_T`1]
# pyre says Any -> Any which I suppose is fine
lifted(g)(1) # incompatible type, int isn't an Opt[_T]
# pyre seems to accep this one
lifted(g)(None) # but at least None is in Opt[_T]...
reveal_type(lifted[_T](g)) # lifted[_T?] -- Ok
reveal_type(lifted[_T](g).__call__) # def (None) ???
# pyre: Opt[_T] -> Opt[_T] as expected
lifted[_T](g)(1) # incompatible type "int"; expected "None"
lifted[_T](g)(None) # but at least None works...
# Pyre doesn't like the [_T] here but accepts the calls without it...
Expected Behavior
Again, I would expect the Generic
lifted
class to accept types that match the wrapped function's type. This is more important when I need to add protocols to the types, but I would also expect it to work here.
Your Environment
I tested this in mypy playground and pyre playground