Improve CI (pin actions + Dependabot)

[currocam]

Hello Ben,

I have recently learned a bit more about GitHub Actions working in a project of mine, and I have some ideas about how the current SLiM CI could be improved.

Before, there have been issues with CI suddenly failing (see #627, #498).

I’m decently sure now that those are triggered by either (1) some minor version being released (say, for example, changing from setup-msys2@v2.31.1 to setup-msys2@v2.32.0 or (2) someone updating a git tag. Note that currently CI sometimes does not pin to any specific version, e.g. actions/cache@v3, conda-incubator/setup-miniconda@v3, etc.

I propose we change the CI to point to specific commit hashes and follow the format username/repository@commit_hash # tag name.

I made that change in the first commit of this PR.

Possible automation

One would have to keep track of those dependencies and update them from time to time. One automation I've found very useful is dependabot. It's basically a cron job that opens PRs with the updated dependency. The automated PRs look like this (notice that they maintain the format commit hash # version tag and have some useful information in the commit message)

currocam#5

Ideally, one would merge those PR after running the updated CI, rather than unexpectedly finding it in red. The obvious downside is that this complicates the CI more.

Bump versions

To avoid creating many PRs at once and running CI many times, I’ve squashed all changes from the automated PRs, so the third commit bump all the GitHub Actions to the latest version. Feel free to cherry-pick which commit (if any) you would like to merge.

Best,
Curro

[bhaller]

Hi @currocam! This all sounds sensible.

I notice that your dependabot commit addresses only minor and patch updates. So if your plan that we would manually notice major updates to these tools and do them ourselves manually? Or am I misunderstanding? Seems like we probably want to shift forward onto major updates in an automated fashion too, but perhaps not squashed together with the minor/patch updates since we might want to actually think about the major ones (maybe).

Yes, this makes the CI more complex, and no, I don't fully understand it all (and don't really want to :->). If you are happy to continue keeping things running for the foreseeable future, then I'm happy to adopt the full scheme you propose here. If not, I might think harder about it, since I don't like being on the hook to maintain things I don't understand. (But that is true when things break for unknown reasons, too, of course!)

I think modulo your replies to these questions, I'm inclined to merge all of this. Thanks!

[currocam]

(1) The dependabot addresses all updates, I think. What the config file declares is a special group that consists of all minor and patches. Sorry if the comment was not clear. The idea is that each major update gets its own PR (because presumably it breaks something), whereas all small changes are accumulated into a single PR to reduce the noise. Also, running the cron job each month sounds sensible to me, but pls adjust.
(2) Yes, this makes the CI more complex. I’m happy to keep running things :) If at some point it becomes annoying, it should be enough to git rm a single file and close any open PRs.

[bhaller]

OK, this all sounds great, thanks! Merged! You are a life-saver @currocam. :-> I'm actively working in the multitrait branch these days. Should I merge these changes into that branch, or is it best for them to remain only on master?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.62%. Comparing base (4e2be2e) to head (3673b62).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #652      +/-   ##
==========================================
- Coverage   75.86%   75.62%   -0.24%     
==========================================
  Files         114      114              
  Lines       72808    72808              
  Branches    12877    12872       -5     
==========================================
- Hits        55234    55064     -170     
- Misses      17574    17744     +170     

see 13 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.