Open
Description
I have a number of cases where it would be very useful to use P.args
and P.kwargs
of a ParamSpec
to annotate tuple and dict objects, for example, when extracting arguments to pass to a function in another context.
Here's an example:
from typing import Callable, TypeVar, Tuple, Any
from typing_extensions import ParamSpec
P = ParamSpec('P')
T = TypeVar('T')
def complex(value: str, reverse: bool =False, capitalize: bool =False) -> str:
if reverse:
value = str(reversed(value))
if capitalize:
value = value.capitalize()
return value
def call_it(func: Callable[P, T], args: P.args, kwargs: P.kwargs) -> T:
print("calling", func)
return func(*args, **kwargs)
def get_callable_and_args() -> Tuple[Callable[P, Any], P.args, P.kwargs]:
return complex, ('foo',), {'reverse': True}
call_it(*get_callable_and_args())
In this scenario P.args
and P.kwargs
represent a kind of ad-hoc NamedTuple
and TypedDict
, respectively.
Does this seem like a reasonable extension for ParamSpecs
?
Metadata
Metadata
Assignees
Projects
Milestone
Relationships
Development
No branches or pull requests
Activity
sobolevn commentedon Sep 13, 2022
I remember seing patterns like that in
typeshed/stdlib
. Finding these examples might be helpful to prove your point 👍grievejia commentedon Sep 13, 2022
This option has already been discussed and rejected in PEP 612. To quote from the discussion:
As an example, for the following code:
Should we reveal a tuple of int + empty dict? Or should we reveal an empty tuple and a singleton dict that maps
x
toint
? Even for this extremely simplified example, the's no definite answer here: the existence of "positional-or-keyword" arguments in Python makes things highly ambiguous. And to have the type checker second-guessing the intention of the developer here would be rather expensive (think about the combinatorial blowup if you a long list of arguments).To fully resolve this kind of issue, there needs to be a way in the type system to represent positional-only, positional-or-keyword, and keyword-only parameters separately. But at that point, it's going to be a different language feature, not
ParamSpec
anymore.Kenny2github commentedon Sep 13, 2022
This came up in the context of ORMs - e.g. something vaguely like
grievejia commentedon Sep 13, 2022
@Kenny2github That does not work because, it's invalid to use
P.args
in isolation. To quote from the PEP:So again the issue is that there's no unambiguous way to split a parameter list cleanly into a
P.args
list andP.kwargs
dict. Therefore we enforce the restriction that those two guys must be used together so the type system does not need to deal with such unambiguity.The use case you mentioned is better handled by the (yet-to-be-pepified) "map" operator extension for list variadics.
chadrik commentedon Sep 14, 2022
So then is it solvable if we instead return an object that represents the entire set of arguments, such as a tuple that holds both the args and the kwargs? That object could then be considered
P
itself when used outside of aCallable
.For example:
grievejia commentedon Sep 14, 2022
@chadrik That won't work either:
P
does not work like regular typevars where it represents a single type. It instead represents a "parameter group" that can be passed into other callables, and there's no way in Python to construct the notion it represents at runtime. Therefore, it does not make sense to say something like "the return type of my function isP
". Even if it does, there's no way you could write it, as there could potentially be many, many waysP
gets split into pos+keyword arglist, and you can't just return one particular split and pretend that it will work for any split downstream.chadrik commentedon Sep 14, 2022
What if we create a new type of object which represents "parameters that are compatible with ParamSpec
P
"?For example:
In the above example, it doesn't matter that we don't know all of the ways to split apart
P
, as long as we can vet thatParamsP
is one of those ways when it is instantiated, which seems doable to me.ParamsP
is essentially just aNamedTuple
that amounts to:There is no inference or guarantees about the structure of
ParamsP
, and from a static analysis POV, it would only be valid to use star and double-star expansion withParamsP.args
andParamsP.kwargs
if both are used together.grievejia commentedon Sep 14, 2022
That only works if your function is a "consumer" of
ParamsP
, i.e. it takesParamsP
as argument. But if you are a "producer" ofParamsP
, i.e. you are returning aParamsP
object from your function, you'd need to make sure that what you returns works for any positional/keyword splits of the parameter list, not just one. I am not aware of how you could construct such an object at runtime. What you proposed in your earlier example definitely does not satisfy this criteria.chadrik commentedon Oct 17, 2022
I've been thinking lately that perhaps the best way to pass around a function and its arguments prior to invoking it is a
partial
. Unfortunately, mypy's partial support is extremely lacking: python/mypy#1484The
returns
package provides a mypy plugin that adds full partial validation, but unfortunately I tested it with mypy 0.982 and it is broken.sobolevn commentedon Oct 17, 2022
Yes, we don't have mypy>0.950 support yet, but it is planned.
chadrik commentedon Oct 17, 2022
Here's the ticket for the
returns
mypy plugin error for anyone coming across this conversation in the future: dry-python/returns#1433rmorshea commentedon Apr 30, 2023
I would find something like this to be very useful. I'm am creating a react-like framework in Python and one thing I want is to be able to create a decorator that marks functions as HTML-like element constructors. The naive approach is to use
*args
to describe element children and**kwargs
as the attributes. Doing so has unfortunate syntactic consequences though:Instead, it would be better if the following could be achieved:
If this feature were available, it seems like I could write
component
as:henribru commentedon Jun 3, 2023
This pattern often comes up in task queue libraries. Huey, Celery, Dramatiq and Rq all have a way of turning a function into a task where you call the task using
args
passed as a tuple andkwargs
passed as a dict, instead of them being unpacked, similar to thecall_it
definition in the original comment.Edit:
threading.Thread
from the stdlib is another example of this pattern actually.rmorshea commentedon Jun 3, 2023
I'm realizing that positional or keyword parameters would likely make implementing this rather complicated. The fact that a parameter can be either positional or a keyword, but not both, means that
ParamSpec.args
andParamSpec.kwargs
are linked - if a parameter is specified in one, it must be disallowed in the other. This must happen magically at a distance. For example:It's possible that this is not a significant technical challenge, but I don't know enough about how MyPy works to say.
2 remaining items