Skip to content

Removes protobuf upper version pin #2726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 27, 2025

Conversation

kwlzn
Copy link
Contributor

@kwlzn kwlzn commented Jun 17, 2025

Description

This PR relaxes the upper range of allowed protobuf versions so that IsaacLab can be installed in a modern python environment without downgrading or conflicting with modern protobuf versions.

The original reason for pinning this to <5 was apparently due to transitive breakage in tensorboard, which also had this pinned to <5 - so pinning this in IsaacLab itself would not be necessary if both deps were composed together. Tensorboard has since (in Aug 2024) unpinned this here: tensorflow/tensorboard#6888

So, the original concern should, afaict, be obviated now.

Fixes

This should repair any case where someone wants to install IsaacLab into a modern python environment that uses any of the Google ecosystem (gRPC/protobuf et al) without conflict or forced down-rev'ing to older versions (current version of protobuf is 6.31.1).

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@kwlzn kwlzn force-pushed the kwlzn/protobuf_version_pin branch from 400af1b to 93f5eb4 Compare June 18, 2025 19:14
@kellyguo11
Copy link
Contributor

I'm hitting this error when updating protobuf to the latest, so it seems like there may still be some compatibility issues there?

TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

@kwlzn
Copy link
Contributor Author

kwlzn commented Jun 18, 2025

I'm hitting this error when updating protobuf to the latest, so it seems like there may still be some compatibility issues there?

TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

hey @kellyguo11, thanks for taking a look. how are you getting to that error?

@kellyguo11
Copy link
Contributor

I tried manually upgrading protobuf to the latest on windows and ran any of the training scripts and hit this. seemed to be coming from wandb.

@kwlzn
Copy link
Contributor Author

kwlzn commented Jun 24, 2025

I tried manually upgrading protobuf to the latest on windows and ran any of the training scripts and hit this. seemed to be coming from wandb.

@kellyguo11 hmm, it looks like wandb declares its own compatible versions of protobuf as:

pyproject.toml:    "protobuf>=3.12.0,!=4.21.0,!=5.28.0,<7; python_version < '3.9' and sys_platform == 'linux'",
pyproject.toml:    "protobuf>=3.15.0,!=4.21.0,!=5.28.0,<7; python_version == '3.9' and sys_platform == 'linux'",
pyproject.toml:    "protobuf>=3.19.0,!=4.21.0,!=5.28.0,<7; python_version > '3.9' and sys_platform == 'linux'",

so resolving isaaclab and wandb in the same transitive resolve step should keep this within their allotted bounds. but from a pure dependency management standpoint, if one library (e.g. wandb) requires a certain version of a library it should directly declare that constraint and not rely on e.g. isaaclab to do that for them.

if you are just doing e.g. pip install --upgrade protobuf then it may put an incompatible version of protobuf in your python env. you'd want to do that as one transitive thing (pip install --upgrade protobuf wandb isaaclab etc or use a requirements.txt file) so that wandb's version constraints are applied.

@kwlzn
Copy link
Contributor Author

kwlzn commented Jun 24, 2025

btw, @kellyguo11 I am investigating the test failure in the isaac-lab-premerge-ci logs for /workspace/isaaclab/source/isaaclab_tasks/test/benchmarking/test_utils.py. I'm not able to actually see where that is failing in the AWS CodeBuild log (it seems control-f to search is disabled) - but fwict, there are no actual tests in that file that would be failing (it appears to contain only test utilities).

is the CI failure for that a known issue? I am not seeing how my change could be related to that - and the second test appears to be flaky (did not fail the first CI run before I rebased).

let me know if you can see how or why that might be failing and I'm happy to dig into it.

@kellyguo11
Copy link
Contributor

Yes agreed when we install as a transitive step, we do not end up pulling the latest protobuf, but that means even if we extend the protobuf version support in lab configs, we will not be able to support the latest protobuf out of the box, so I'm not sure if it makes sense for us to relax the version in the lab dependencies.

For the unit test, you can also try running it locally with python -m pytest source/isaaclab_tasks/test/benchmarking/test_utils.py -v -s to see if there are any errors there.

@kwlzn
Copy link
Contributor Author

kwlzn commented Jun 26, 2025

Yes agreed when we install as a transitive step, we do not end up pulling the latest protobuf, but that means even if we extend the protobuf version support in lab configs, we will not be able to support the latest protobuf out of the box, so I'm not sure if it makes sense for us to relax the version in the lab dependencies.

@kellyguo11 I'm not sure I follow when you say "we will not be able to support the latest protobuf out of the box". what dependency does IsaacLab itself have on protobuf that causes it to fail with the latest protobuf? and how does it specifically fail?

I'm not actually seeing the protobuf dep being used in the project sources (maybe I'm overlooking something) - but it sounded from your message like this was scoped to 3rd party dependencies (presumably for tensorboard). if that is the case, those 3rd party deps (e.g. tensorboard or wandb) should be the thing specifying the upper bounds for protobuf transitively in order to get the correct behavior - and it appears to already do that. if IsaacLab itself doesn't directly use protobuf, it should ultimately omit that from it's requirements and let the transitive deps manage that - or at a minimum, unpin the upper range to be more compatible with modern python environments.

For the unit test, you can also try running it locally with python -m pytest source/isaaclab_tasks/test/benchmarking/test_utils.py -v -s to see if there are any errors there.

doesn't appear to be:

============================================================== test session starts ===============================================================
platform linux -- Python 3.10.16, pytest-8.4.1, pluggy-1.6.0 -- /home/kwilson/.cache/uv/archive-v0/q-GaacAoBduFSmhdd5jBn/bin/python3
cachedir: .pytest_cache
rootdir: /home/kwilson/dev/IsaacLab/source/isaaclab_tasks
configfile: pyproject.toml
plugins: anyio-4.9.0, hydra-core-1.3.2
collected 0 items
{
  "overall": {
    "totals": {},
    "successes": {},
    "failures_did_not_finish": {},
    "failures_did_not_pass_thresholds": {},
    "timestamp": "2025-06-26T16:18:04.623532",
    "tag": ""
  }
}

============================================================= no tests ran in 0.01s ==============================================================

I see the same failure on other PRs, so I'm pretty sure it's unrelated to my change.

@kwlzn kwlzn force-pushed the kwlzn/protobuf_version_pin branch from 6f4a37d to d6e11fe Compare June 26, 2025 23:24
@kellyguo11
Copy link
Contributor

Lab shouldn't have a direct dependency on protobuf, but rather the dependencies of lab prevents it from being compatible with the latest version. It looks like it's still able to pull in an older version that's compatible with wandb with these changes, so it shouldn't cause any issues merging this. I'm just not convinced that these changes alone will allow someone to run isaac lab out of the box with the latest version of protobuf.

@kellyguo11 kellyguo11 changed the title Remove protobuf upper version pin. Removes protobuf upper version pin Jun 27, 2025
@kellyguo11 kellyguo11 merged commit 05c22be into isaac-sim:main Jun 27, 2025
4 of 5 checks passed
@kwlzn
Copy link
Contributor Author

kwlzn commented Jun 27, 2025

Lab shouldn't have a direct dependency on protobuf, but rather the dependencies of lab prevents it from being compatible with the latest version. It looks like it's still able to pull in an older version that's compatible with wandb with these changes, so it shouldn't cause any issues merging this. I'm just not convinced that these changes alone will allow someone to run isaac lab out of the box with the latest version of protobuf.

gotcha. well we are working on doing exactly this, so I'll report back or circle back with more PRs if we hit issues. thank you!

btw, do you know when the next release is planned?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants