A milestone for dropping provisional status?

[ilevkivskyi]

PEP 484 will drop its provisional status at some point. I hope this will happen in time for 3.6.
I propose to create a milestone to estimate how much work should be done before this happens.
Here are some issues that I propose to add to this milestone:

Big things:

• The notorious Kill __subclasscheck__ #136 (and Bare Union, Callable and Tuple should not be classes #133, typing.MutableMapping not a drop-in replacement for collections.abc.MutableMapping #203 that depend on it). This bug requires substantial changes to typing.py. However, it does not require PEP change and there is already a PR Fix generics inheritance and drop __subclasscheck__ #207 so that probably it will not take too much time.
• Protocols (a.k.a. structural subtyping) #11 (Protocols) It looks like a very convenient tool, and it is suitable for Python's duck-typing. There is already a concrete proposal by @JukkaL Protocols (a.k.a. structural subtyping) #11 (comment) that was positively accepted at python-ideas.

Smaller things:

• Introduce an Intersection #213 (Intersection) This will be very easy to implement in typing.py. It will work nicely with protocols. Also I could speculate that mypy implementation will not be too difficult (it should be similar to Union).
• Make Callable more flexible #239 and Proposal: Generalize Callable to be able to specify argument names and kinds #264 (flexible Callable) This feature has been requested several times, and it is really easy to implement in typing.py, there are some detail to discuss.
• Allow subclassing without supertyping #241 and Declaring classes as pure mixins #246 (Implementation) It seems that an agreement on this is quite close.
• Decide how to handle str/unicode #208 how to handle str/unicode.
• A way to spell "SelfType", the type of self #271 SelfType and type annotations of classmethods? I am not sure about this one.

I could volunteer to take care about all of the above except #11 (it probably will be better done by Jukka) and SelfType. By "take care" I mean make PRs to PEPs and typing.py. However, I do not exclude also working on some of these in mypy if necessary. I think it is not necessary for mypy to support all the features at the point when PEP 484 drops provisional status, they could be implemented later.

[vlasovskikh]

And #208, I'm working on it.

[JukkaL]

There's no issue for this yes, but I'd want us to resolve the slowness of expressions like List[int], as each of these constructs a new type object (or even multiple type objects if the type is more complex). I did some quick benchmarking a while ago and it seems that type annotations can slow down program startup significantly. Also, it can affect the performance of code with nested, annotated functions and casts to complex types.

This was discussed at PyCon and somebody (@ambv?) raised this as a potential issue.

[ilevkivskyi]

@JukkaL There was a proposal #130 to cache outcomes of such things. The issue is closed now.
I think that this is in fact also related to #136 . For example, Union[str, int] does not need to be a class, anyway one should not subclass it, and it will not support issubclass etc.

I think that making some typing objects more lightweight together with cashing could give a significant speed-up (also @markshannon will be happy).

I think if @gvanrossum does not object (he closed that issue), then you could reopen #130 to track this.

[JukkaL]

Caching would likely be quite helpful with casts and nested functions, but it might not eliminate enough of the startup overhead.

I think that we should have some (repeatable) benchmark figures so that we understand the scope of the problem before starting to optimize things. Also, perhaps we should also come up with a target acceptable performance level, such as "don't make program startup more than 10% slower compared to unannotated code for typical (?) programs".

[JukkaL]

Another thing we may want to resolve is IO[bytes] vs. BinaryIO (and [IO[str] vs. TextIO). Maybe having IO[AnyStr] would be sufficient?

[gvanrossum]

I've seen many issues caused by IO[] vs. BinaryIO. maybe we could just define BinaryIO as an alias for IO[bytes] and TextIO as IO[str]? (I'd have to look at the code in detail to understand what each of the named classes adds, and how important the distinction is -- but it's caused lots of confusion with in typeshed.)

[ilevkivskyi]

If some issues from the original list are fixed I will just edit it not to bother everyone.

[gvanrossum]

FWIW I don't think we should hold our breath for many of these.

• Kill __subclasscheck__ #136 (subclass checks) OK, this one probably needs to be fixed
• Protocols (a.k.a. structural subtyping) #11 (protocols) is too big to implement any time soon, and I don't think the proposal is small enough to accept without a reference implementation

• 213 (intersection) I'm still lukewarm at best

• Make Callable more flexible #239, Proposal: Generalize Callable to be able to specify argument names and kinds #264 (flexible callable) seems a huge project compared to the benefits

• 241, Declaring classes as pure mixins #246 (implementation) I don't see these as close

• Decide how to handle str/unicode #208 (unicode) Agreed this one should be dealt with
• A way to spell "SelfType", the type of self #271 SelfType if we can do this by supporting (self: T) -> T I think that's a good plan
• slowness (Cache the outcome of e.g. List[T] #130) shouldn't need a PEP update even if we reopen Cache the outcome of e.g. List[T] #130 (I'm fine with that)
• IO[]: I wish BinaryIO could be an alias for IO[bytes] and TextIO for IO[str] but they have additional methods -- please somebody open an issue, I don't think IO[Any] is the solution

[ilevkivskyi]

@gvanrossum I would say that #136 is the most important for me in this list. Protocols, Intersection, and Implementation would be nice to have. Actually, they all three go along the same idea - type checking beyond nominal subtyping (or decoupling types and classes). I particular, Intersection would be more important with protocols (with nominal subtyping C = Intersection(A, B) is practically equivalent to class C(A, B): pass). Also, Implementation allows more fine tuned control of subtyping in contrast to purely nominal type system (where A is a subtype of B iff A is explicitly declared as a subclass of B). Flexible Callable is quite a self contained (but not so big) proposal. Probably, I should agree that these will slow us down. A possible solution would be to consider these two ideas (type checking beyond nominal subtyping and flexible Callable syntax) later after ironing out all the necessary details (maybe as separate PEPs).

Concerning #130, it does not require PEP changes directly, but the solution could change the behavior of issubclas and/or isinstance. My opinion here is that Union, Tuple, Optional should not be classes at all, as well as subscripting them should not create classes (this will give some speed-up). Callable is special in this sense, it should be a class (just an alias for collections.abc.Callable) but Callable[<whatever>] should not be a class. Probably, the only things that should be an actual class is Generic[T].

[gvanrossum]

Yeah, I think you have the right idea. #136 especially needs to be sorted
out before we can call the PEP finalized, and I agree with you on Union
etc. -- they just need to be simple objects that record the parameters.

Pure additions to typing.py can be their own PEP or even discussed in the
Python bug tracker (though I would much prefer to have them in the tracker
here, regardless of whether they need a PEP, since working with
bugs.python.org is so much more painful).

[gvanrossum]

So this was never actually decided, however the 3.6 what's new says typing.py is no longer provisional. I actually now believe that's premature. Can we keep it provisional for 3.6?

[gvanrossum]

I've filed http://bugs.python.org/issue29316 to discuss this.