Avoid frequent test failure in greedy_test.py by increasing timeout #7188
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mhucka
changed the title
On my MacOS 15.x Apple M1 laptop, I frequently get a test failure in
`cirq-core/cirq/contrib/routing/greedy_test.py`:
```
[gw2] darwin -- Python 3.11.9 /Users/mhucka/.pyenv/versions/cirq-py311-np1/bin/python3
def test_router_hanging():
"""Run a separate process and check if greedy router hits timeout (5s)."""
circuit, device_graph = create_circuit_and_device()
process = Process(target=create_hanging_routing_instance, args=[circuit, device_graph])
process.start()
process.join(timeout=5)
try:
> assert not process.is_alive(), "Greedy router timeout"
E AssertionError: Greedy router timeout
E assert not True
E + where True = is_alive()
E + where is_alive = <Process name='Process-1' pid=58097 parent=56380 started>.is_alive
cirq-core/cirq/contrib/routing/greedy_test.py:58: AssertionError
```
This happens with Python 3.10 and 3.12 as well.
Since assertion being tested is that the process is _not_ alive, it
seems that 5 seconds can be too short. And indeed, the timeout is
increased to 20 seconds, the error never seems to occur in my local
Mac testing.
It's not obvious whether a short timeout is needed for the purposes
of the test. Shortening the duration to 10 sec also works, but might
not in other people's environments. So, 20 sec seems safer. However,
if it needs to be as short as possible, then 10 seconds seems workable
too.
mhucka
force-pushed
the
mh-fix-greedy-test-timeout
branch
from
2d18654 to
a1c9e6f
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #7188 +/- ##
==========================================
- Coverage 98.14% 98.14% -0.01%
==========================================
Files 1100 1100
Lines 96191 96191
==========================================
- Hits 94409 94405 -4
- Misses 1782 1786 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
viathor
approved these changes
Mar 26, 2025
| @@ -53,7 +53,7 @@ def test_router_hanging(): | |||
| circuit, device_graph = create_circuit_and_device() | |||
| process = Process(target=create_hanging_routing_instance, args=[circuit, device_graph]) | |||
| process.start() | |||
| process.join(timeout=5) | |||
| process.join(timeout=20) | |||
There was a problem hiding this comment.
Please update docstring on line 52 above.
pavoljuhas
approved these changes
Mar 26, 2025
This was referenced May 30, 2025
BichengYing
pushed a commit
to BichengYing/Cirq
that referenced
this pull request
Jun 20, 2025
…uantumlib#7188) * Fix frequent test failure in greedy_test.py On my MacOS 15.x Apple M1 laptop, I frequently get a test failure in `cirq-core/cirq/contrib/routing/greedy_test.py`: ``` [gw2] darwin -- Python 3.11.9 /Users/mhucka/.pyenv/versions/cirq-py311-np1/bin/python3 def test_router_hanging(): """Run a separate process and check if greedy router hits timeout (5s).""" circuit, device_graph = create_circuit_and_device() process = Process(target=create_hanging_routing_instance, args=[circuit, device_graph]) process.start() process.join(timeout=5) try: > assert not process.is_alive(), "Greedy router timeout" E AssertionError: Greedy router timeout E assert not True E + where True = is_alive() E + where is_alive = <Process name='Process-1' pid=58097 parent=56380 started>.is_alive cirq-core/cirq/contrib/routing/greedy_test.py:58: AssertionError ``` This happens with Python 3.10 and 3.12 as well. Since assertion being tested is that the process is _not_ alive, it seems that 5 seconds can be too short. And indeed, the timeout is increased to 20 seconds, the error never seems to occur in my local Mac testing. It's not obvious whether a short timeout is needed for the purposes of the test. Shortening the duration to 10 sec also works, but might not in other people's environments. So, 20 sec seems safer. However, if it needs to be as short as possible, then 10 seconds seems workable too. * Update docstring to match change in code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On my MacOS 15.x Apple M1 laptop, I frequently get a test failure in
cirq-core/cirq/contrib/routing/greedy_test.py:This happens with Python 3.10 and 3.12 as well.
Since assertion being tested is that the process is not alive, it seems that 5 seconds can be too short. And indeed, if the timeout is increased to 20 seconds, the error never seems to occur in my local Mac testing.
It's not obvious to me whether a short timeout is needed for the purposes of the test. Shortening the duration to 10 sec also works, but might not in other people's environments. So, 20 sec seems safer. However, if it needs to be as short as possible, then 10 seconds seems workable too.