Skip to content

[T145005253] Fix the build scripts infrastructure #1589

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from

Conversation

q10
Copy link
Contributor

@q10 q10 commented Feb 9, 2023

Summary of the changes:

  • Add well-defined functions that perform steps of the installation well, with verbose logging
  • Redefine the nightly builds using the functions defined in setup_env.bash
  • Split up the FBGEMM CI jobs into FBGEMM CI and FBGEMM_GPU CI
  • Enable the nightly builds jobs to run in PR and push mode, BUT with the publish to PyPI step disabled
  • Remove support for building against older CUDA versions
  • Upgrade the build machines for the nightly GPU builds to linux.x12large, thus reducing the build time from over 3.5 hours to 11 minutes
  • Disable the FBGEMM_GPU CIs for now, until we figure out what is broken

@netlify
Copy link

netlify bot commented Feb 9, 2023

Deploy Preview for pytorch-fbgemm-docs canceled.

Name Link
🔨 Latest commit b529d25
🔍 Latest deploy log https://app.netlify.com/sites/pytorch-fbgemm-docs/deploys/63ed887bf5fbf50008ae6f15

@q10 q10 force-pushed the bm/T145005253/1/build-scripts branch 16 times, most recently from 0da05e7 to d70cb2d Compare February 14, 2023 00:25
@q10 q10 changed the title Bm/t145005253/1/build scripts [T145005253] Fix the build scripts infrastructure Feb 14, 2023
@facebook-github-bot
Copy link
Contributor

@q10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@q10 q10 force-pushed the bm/T145005253/1/build-scripts branch from d70cb2d to bca01b0 Compare February 14, 2023 02:43
@facebook-github-bot
Copy link
Contributor

@q10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@q10 q10 force-pushed the bm/T145005253/1/build-scripts branch from 95b5cbd to 02589cb Compare February 15, 2023 00:14
q10 added 5 commits February 14, 2023 17:44
…tions

- Add well-defined functions to `.github/scripts/setup_env.bash` to handle
individual steps in FBGEMM_GPU installation

- Update existing scripts to call the functions in `.github/scripts/setup_env.bash`
as part of execution
- Update the FBGEMM_GPU nightly publish jobs to use functions in
`.github/scripts/setup_env.bash`

- Add new script function to install C/C++ compilers properly
through Conda or Yum
- Updates to `.github/scripts/setup_env.bash` to improve the reliability
of the functions

- Update GitHub action references to v3

- Update FBGEMMCI / build_nvidia_gpu job to use the functions defined
in `.github/scripts/setup_env.bash`
- Split up FBGEMMCI job definitions into `.github/workflows/fbgemm_ci.yml`
and `.github/workflows/fbgemm_gpu_ci.yml`

- Rename the jobs to be less confusing
- Set the nightly builds to use larger instances for more build
parallelization

- Disable the failing jobs in FBGEMM_GPU CI

- Remove support for building against CUDA 11.5 and 11.6

- Disable the actual Publish-to-PyPI step from the nightly build
jobs when run in PR or Push mode

- Disable the cron trigger on push_trigger_wheel jobs
@q10 q10 force-pushed the bm/T145005253/1/build-scripts branch from 727f3ce to c9bef9b Compare February 15, 2023 01:45
@q10 q10 force-pushed the bm/T145005253/1/build-scripts branch 3 times, most recently from 30a3e57 to f7f3afe Compare February 15, 2023 23:43
- Remove commented code

- Clean up the scripts to be compliant with shellcheck linting
@q10 q10 force-pushed the bm/T145005253/1/build-scripts branch from f7f3afe to b529d25 Compare February 16, 2023 01:35
@facebook-github-bot
Copy link
Contributor

@q10 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@q10 merged this pull request in 8d4777e.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants