typing_extensions >= 4.6.0 causes pip unit test failure

[elfringham]

Click to expand!

Issue Type

Bug

Have you reproduced the bug with TF nightly?

Yes

Source

source

Tensorflow Version

git HEAD

Custom Code

No

OS Platform and Distribution

Ubuntu 20.04

Mobile device

n/a

Python version

3.10

Bazel version

5.3.0

GCC/Compiler version

10.2.1

CUDA/cuDNN version

n/a

GPU model and memory

n/a

Current Behaviour?

//bazel_pip/tensorflow/python/trackable:data_structures_test will fail with typing_extensions >= 4.6.0 installed when run as a pip test against an installed TensorFlow wheel.

Standalone code to reproduce the issue

bazel test --build_tests_only --cache_test_results=no --config=mkl_aarch64_threadpool --copt=-flax-vector-conversions --jobs=75 --test_env=TF_ENABLE_ONEDNN_OPTS=1 --test_env=TF2_BEHAVIOR=1 --define=no_tensorflow_py_deps=true --test_lang_filters=py --test_size_filters=small,medium --test_output=errors --verbose_failures=true //bazel_pip/tensorflow/python/trackable:data_structures_test

Relevant log output

======================================================================
ERROR: testFunctionCaching (__main__.MappingTests)
MappingTests.testFunctionCaching
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/bazel_pip/tensorflow/python/trackable/data_structures_test.runfiles/org_tensorflow/bazel_pip/tensorflow/python/trackable/data_structures_test.py", line 507, in testFunctionCaching
    second_trace = f.get_concrete_function(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 1198, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 1189, in _get_concrete_function_garbage_collected
    concrete = self._variable_creation_fn.get_concrete_function(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 197, in get_concrete_function
    concrete_function, _ = self._maybe_define_concrete_function(args, kwargs)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 172, in _maybe_define_concrete_function
    return self._maybe_define_function(args, kwargs)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 294, in _maybe_define_function
    function_type_utils.make_canonicalized_monomorphic_type(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/function_type_utils.py", line 378, in make_canonicalized_monomorphic_type
    function_type_lib.canonicalize_to_monomorphic(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_type.py", line 481, in canonicalize_to_monomorphic
    _make_validated_mono_param(name, arg, poly_parameter.kind,
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_type.py", line 421, in _make_validated_mono_param
    mono_type = trace_type.from_value(value, type_context)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/core/function/trace_type/trace_type_builder.py", line 142, in from_value
    elif isinstance(value, trace.SupportsTracingProtocol):
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/typing_extensions.py", line 605, in __instancecheck__
    val = inspect.getattr_static(instance, attr)
  File "/usr/lib/python3.8/inspect.py", line 1596, in getattr_static
    instance_result = _check_instance(obj, attr)
  File "/usr/lib/python3.8/inspect.py", line 1543, in _check_instance
    instance_dict = object.__getattribute__(obj, "__dict__")
TypeError: this __dict__ descriptor does not support '_DictWrapper' objects

======================================================================
ERROR: testFunctionCaching (__main__.TupleTests)
TupleTests.testFunctionCaching
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/execroot/org_tensorflow/bazel-out/aarch64-opt/bin/bazel_pip/tensorflow/python/trackable/data_structures_test.runfiles/org_tensorflow/bazel_pip/tensorflow/python/trackable/data_structures_test.py", line 716, in testFunctionCaching
    second_trace = f.get_concrete_function(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 1198, in get_concrete_function
    concrete = self._get_concrete_function_garbage_collected(*args, **kwargs)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 1189, in _get_concrete_function_garbage_collected
    concrete = self._variable_creation_fn.get_concrete_function(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 197, in get_concrete_function
    concrete_function, _ = self._maybe_define_concrete_function(args, kwargs)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 172, in _maybe_define_concrete_function
    return self._maybe_define_function(args, kwargs)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 294, in _maybe_define_function
    function_type_utils.make_canonicalized_monomorphic_type(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/function_type_utils.py", line 378, in make_canonicalized_monomorphic_type
    function_type_lib.canonicalize_to_monomorphic(
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_type.py", line 481, in canonicalize_to_monomorphic
    _make_validated_mono_param(name, arg, poly_parameter.kind,
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/core/function/polymorphism/function_type.py", line 421, in _make_validated_mono_param
    mono_type = trace_type.from_value(value, type_context)
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/tensorflow/core/function/trace_type/trace_type_builder.py", line 142, in from_value
    elif isinstance(value, trace.SupportsTracingProtocol):
  File "/workspace/pip_test/venv_clean/lib/python3.8/site-packages/typing_extensions.py", line 605, in __instancecheck__
    val = inspect.getattr_static(instance, attr)
  File "/usr/lib/python3.8/inspect.py", line 1596, in getattr_static
    instance_result = _check_instance(obj, attr)
  File "/usr/lib/python3.8/inspect.py", line 1543, in _check_instance
    instance_dict = object.__getattribute__(obj, "__dict__")
TypeError: this __dict__ descriptor does not support '_TupleWrapper' objects

----------------------------------------------------------------------
Ran 74 tests in 1.021s

FAILED (errors=2, skipped=4)
================================================================================

[tilakrayal]

@elfringham,
The related PR #60688 was merged and as requested the 'typing_extensions >= 3.6.6', are changed to
'typing_extensions>=3.6.6,<4.6.0' with the mentioned merged PR.

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/pip_package/setup.py#L100.

Could you please confirm whether changes are reflected as mentioned. Thank you!

[hmc-cs-mdrissi]

The rough cause of error is here. The current getattribute for dictwrapper can return TypeError. When an attribute is not found it's instead expected to return AttributeError. Finding place that produces TypeError and replacing it with AttributeError should fix typing extensions incompatibility. typing 4.6 uses inspect.getattr_static which expects that failed attribute access using getattribute only triggers attribute error.

This particular error will also be an error if you try to run tensorflow on python 3.12. The typing extensions change was motivated from here.

edit: Minimal version of this error is,

from tensorflow.python.trackable.data_structures import _DictWrapper

t = _DictWrapper()
object.__getattribute__(t, "__dict__")

The last line raises TypeError. On other hand t.__dict__ properly raises AttributeError. DictWrapper's custom getattribute is rough area that needs bug fix.

The error message comes from here. I've cross posted in cpython side as unsure whether this tensorflow bug or cpython bug.

[stewartmiles]

FYI folks @AlexWaygood over on typing-extensions thinks this is an issue with wrapt, from python/typing_extensions#216 (comment) here's @AlexWaygood's message:

Looks to me like it's probably an issue with wrapt rather than typing_extensions or TensorFlow, actually. We discussed this a bit in python/cpython#105134.

Wrapt raises TypeError if you do object.getattribute(<some_wrapt_ObjectProxy_instance, 'dict'). I know of no way to construct a class using pure Python that has this behaviour (wrapt achieves this behaviour by using a C extension), and it breaks some fundamental assumptions that the stdlib function inspect.getattr_static makes. That, in turn, breaks typing_extensions.

[github-actions]

This issue is stale because it has been open for 7 days with no activity. It will be closed if no further activity occurs. Thank you.