-
Notifications
You must be signed in to change notification settings - Fork 727
ci: add workflow for rust bench crate #4210
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
The bench crate contains interop testing that would be useful to have in our ci runs. This commit adds a github workflow to run that. It also adds an options to generate the bindings without running the tests, since the tests take several minutes to run.
.github/workflows/ci_rust.yml
Outdated
# our benchmark testing includes interop tests between s2n-tls, rustls, and | ||
# openssl | ||
bench-tests: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mentioned this concern a while back, but you might have forgotten: if I see the name "bench / benchmark tests", I don't assume interop testing. I assume performance benchmarks. Does this actually run any performance benchmarks? Or is it just running interop tests?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://github.com/aws/s2n-tls/blob/main/bindings/rust/bench/src/harness.rs#L426
It runs these tests, which for all of OpenSSL, Rustls, s2n-tls, will test the product of
- handshake type
- cipher suite
- kx type
- cert type
Also session resumption interop tests.
And then checks that the certificate file paths are valid.
So this doesn't do any performance benchmarks.
Do you have any name suggestions? I was just sticking to the lowest denominator in that these are literally the tests in the bench crate. The longer term plan that has been tossed around is that we will shift all of our test harnesses to use this, at which point these tests could live in the integration
crate?
This PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
* adjust test name to remove "bench" and mention interop
* create certs directory if it doesn't exist
The required bounds for traits have recently changed.
The bench crate contains interop testing that would be useful to have in our ci runs. This commit adds a github workflow to run that. It also adds an options to generate the bindings without running the tests, since the tests take several minutes to run.
Description of changes:
Testing:
The workflow should pass.
After this workflow has been added, we should make it a required success for merging.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.