Move book tests to github runners

[aaruni96]

PR to check how long the book tests take on GH runners.

[aaruni96]

We have this line in the CI.yml

runs-on: ["${{ matrix.os }}", "${{ matrix.group == 'short' && 'high-memory' || 'normal-memory'}}", RPTU]

It adds the label "RPTU" to ensure everything runs on RPTU runners, and also, labels a job "high memory" or "normal memory", depending on whether its the short tests, or everything else. So, I have no easy way to move just the book tests to github runners (alternatively, we get rid of trickery in this line, but then we must figure out another hack to get back the behaviour of short tests on high memory runners).

Should I break out the book tests into another job, like we do with doctests?

@benlorenz

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.41%. Comparing base (31167d4) to head (7ca030c).
Report is 22 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4629      +/-   ##
==========================================
+ Coverage   84.11%   84.41%   +0.29%     
==========================================
  Files         674      674              
  Lines       89781    89880      +99     
==========================================
+ Hits        75522    75874     +352     
+ Misses      14259    14006     -253     

see 61 files with indirect coverage changes

[benlorenz]

Should I break out the book tests into another job, like we do with doctests?

Yes, I think that should be ok. It also looks nicer on the overview with the graph.

[lgoettgens]

You don't need to set the group here, and could instead hardcode it below. This would also remove some duplication in the job title booktest (1.10, book).
But no high priority

[fingolfin]

I am happy with this being moved to another job set, bit just oit of curiosity, couldn't you also just do this:

runs-on: ["${{ matrix.os }}", "${{ matrix.group == 'short' && 'high-memory' || 'normal-memory'}}", "${{ matrix.group == 'book' || 'RPTU'}}]

[aaruni96]

We can't do that because if it gets labeled "normal-memory" or "high-memory", then github will not pick them up either.

Jobs will be assigned to runners which match all of the labels on the job. Github runners are only labelled with the OS, so, ubuntu-latest. Any additional label guarantees it doesn't run on github runner.

Label RPTU is only for future proofing, in case, in the future, we have a self hosted runner for Oscar which isn't hosted in RPTU.

[fingolfin]

This is marked as a draft, so what exactly is to be done or decided?

[aaruni96]

what is to be decided?

This PR started off as a check to how much slower book tests would be on the github runners v/s rptu runners. (Answer: ~10 minutes). Its not yet decided (AFAIK) whether or not you actually greenlight this change yet.

[fingolfin]

If they are just 10 minutes slower, I'd say, let's do this. And then we monitor. And if necessary it is easy enough to undo this, too.

[lgoettgens]

This just needs someone (@fingolfin or @benlorenz) to adapt the list of required CI jobs