Description
It would be useful in some cases to have an abstract class where a method's return type is overridable, but a default implementation is still provided for the most common case. Consider an example like this:
from abc import ABC, abstractmethod
from typing import Generic, TypeVar
_InputType = TypeVar('_InputType', contravariant=True)
_IntermediateType = TypeVar('_IntermediateType')
_OutputType = TypeVar('_OutputType', covariant=True)
class GenericBase(Generic[_InputType, _IntermediateType, _OutputType], ABC):
@abstractmethod
def first_step(self, pipeline_input: _InputType) -> _IntermediateType: ...
def second_step(self, state: _IntermediateType) -> _OutputType:
# By default, pass through state unmodified
return state
def execute(self, pipeline_input: _InputType) -> _OutputType:
state = self.first_step(pipeline_input)
return self.second_step(state)
class GenericImpl(GenericBase[str, int, float]):
def first_step(self, pipeline_input: str) -> int:
return len(pipeline_input)
def second_step(self, state: int) -> float:
return state * 1.5
a = GenericImpl()
print(a.execute("aaa"))In this case, I would like GenericBase to provide a default implementation of second_step for the most common cases, but allow implementations to change its return type if they would like. Mypy does not allow for this though, and produces an error:
generic_abstract.py:16: error: Incompatible return value type (got "_IntermediateType", expected "_OutputType")
Without explicit casting, there doesn't seem to be a way to provide a default implementation for second_step that avoids this error. One possible workaround is to define an interface class and make GenericBase a subclass of that:
from abc import ABC, abstractmethod
from typing import Generic, TypeVar
_InputType = TypeVar('_InputType', contravariant=True)
_IntermediateType = TypeVar('_IntermediateType')
_OutputType = TypeVar('_OutputType', covariant=True)
class GenericInterface(Generic[_InputType, _IntermediateType, _OutputType], ABC):
@abstractmethod
def first_step(self, pipeline_input: _InputType) -> _IntermediateType: ...
@abstractmethod
def second_step(self, state: _IntermediateType) -> _OutputType: ...
def execute(self, pipeline_input: _InputType) -> _OutputType:
state = self.first_step(pipeline_input)
return self.second_step(state)
class GenericBase(GenericInterface[_InputType, _IntermediateType, _IntermediateType]):
def second_step(self, state: _IntermediateType) -> _IntermediateType:
# By default, pass through state unmodified
return state
class GenericImpl(GenericInterface[str, int, float]):
def first_step(self, pipeline_input: str) -> int:
return len(pipeline_input)
def second_step(self, state: int) -> float:
return state * 1.5
class OtherGenericImpl(GenericBase[str, int]):
def first_step(self, pipeline_input: str) -> int:
return int(pipeline_input)
a = GenericImpl()
print(a.execute("aaa"))
b = OtherGenericImpl()
print(b.execute("123"))This works, but seems a little un-pythonic. It complicates the class hierarchy and requires us to define an extra class without any real purpose. A better approach would be if, when reading the first definition of GenericBase, Mypy inferred _IntermediateType and _OutputType to be the same type. This would mean that an implementation would be valid iff either _IntermediateType and _OutputType were compatible types, or if it overrode second_step to match the type arguments given to GenericBase. To make this work, type arguments should be evaluated in the implementation, not the base class definition. So the examples I gave above would work, but something like this:
class WrongGenericImpl(GenericBase[str, int, str]):
def first_step(self, pipeline_input: str) -> int:
return len(pipeline_input)would produce an error. There are probably complications I haven't considered in implementing this, but I think the basic idea is worth considering. Otherwise, does anyone have a suggestion for how else I might implement the design pattern in my first example?