Track package and paper metrics weekly - #137
Conversation
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…age name Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…right name Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
…sitory Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
| echo "No change this week" | ||
| else | ||
| git commit -m "Weekly package and paper metrics" | ||
| git push |
There was a problem hiding this comment.
A second, more serious problem on this line, separate from the rebuild one already open above: the push will be rejected outright, so the job never records anything.
repos/.../rules/branches/main returns ["deletion","non_fast_forward","pull_request"] — an active ruleset requiring a pull request on the default branch. GITHUB_TOKEN bypasses that only if listed as a bypass actor, and the evidence says it is not: every github-actions[bot] commit on main arrived as a PR merge (#131, #124, #103, #96, #92), never a direct push; update-publications.yaml, the repo's other weekly job writing to _data/, routes through peter-evans/create-pull-request; and release-reminders.yaml uses an App token rather than GITHUB_TOKEN.
Failure scenario: the PR merges, Monday's job collects fine, git commit succeeds, git push is rejected, the run goes red and nothing is written. That repeats weekly with no signal beyond a failed run. Worse, it interacts with the 35-day window added in 5ea6dd2: once the single committed date falls outside it, the page drops all ten CRAN figures and renders "On r-universe" links instead, with nothing to say why. The feature quietly un-does itself about five weeks after merge.
Not fixing it — .github/** stays a human's call. Both options that work are already patterns in this repo: create-pull-request as in update-publications.yaml (fixes the push, not the rebuild), or an App token as in release-reminders.yaml (fixes both, since a push under a non-Actions identity also triggers build-and-deploy).
There was a problem hiding this comment.
Addressed in 772e351, at the maintainer's direction — this file was left alone through the review rounds on purpose, and is changed now because it was asked for rather than because a reviewer suggested it.
The job now generates a token from the same GitHub App release-reminders.yaml uses, hands it to actions/checkout so git push carries it, and passes it to the R code as GITHUB_PAT. That addresses both findings on this line: an App is a distinct identity from github-actions[bot], so its push triggers build-and-deploy, and Apps are what ruleset bypass is normally granted to. The permissions: contents: write block is gone, since the App token carries its own.
Two things this still needs, neither of which is visible from here — both org settings:
- The App must have Contents: read and write on this repository.
- The App must be a bypass actor on the "Protect default branches" ruleset.
Until both are true the push will still be rejected. The cheap check is to dispatch this workflow once from the branch and see whether it pushes.
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Co-authored-by: sbfnk <sebastian.funk@lshtm.ac.uk>
Adds a weekly job that records download counts, GitHub stars and citation counts, and shows the download figure on the software page.
Issue #85 asked for this to support grant applications. The value is the history, which only accumulates from the day the job starts, so this collects the numbers now and keeps the presentation minimal.
What it collects
_data/package-stats.csvgets one row per package per week: stars, forks, open issues, whether the package is on CRAN, and CRAN downloads for the last month. The list of packages comes from the r-universe registry plus_data/software-extras.yml, so it tracks exactly what the software page shows and needs no list of its own._data/paper-citations.csvgets one row per paper per week, with citation counts from OpenAlex. The DOIs come from_data/papers.bib, which the publications workflow already maintains.Rerunning on the same day corrects the rows rather than duplicating them.
What it shows
One addition to the software page: the download figure joins the existing metadata line, reading
R · Updated Aug 2026 · 598 CRAN downloads/month. Naming CRAN matters because only the ten packages on CRAN have a figure, and without it a blank could be read as "not used" rather than "not on CRAN".Nothing else is rendered. In particular there is no citation count anywhere on the site.
Why citations are collected but not shown
Citation counts belong in a grant application, not on a public page. Beside each paper in a list sorted by year they become a column the eye reads down and ranks, including work by different people in the group, and recent papers legitimately show zero: the ten 2025 papers have 19 citations between them, which is normal and looks poor in a table.
The numbers are also concentrated. Of 5,240 citations across 39 papers, 2,830 belong to a single 2020 paper, and the median paper has 15. OpenAlex indexes fewer sources than Google Scholar, so these are a lower bound.
Packages carry no citation column at all. Of sixteen, three declare a citable reference, and one of those is a CRAN package DOI that is almost never cited, so the column would have been blank or misleading nearly everywhere.
Stars are collected but not displayed for the same reason. They measure visibility rather than use, and the two diverge: scoringutils has 934 downloads and 64 stars, EpiNow2 has 598 and 140, contactsurveys has 263 and 2.
Notes
The job commits the CSVs directly to
mainrather than opening a weekly pull request, which needspermissions: contents: write. That commit triggers a site build, so the download figures on the software page refresh themselves.If
_data/package-stats.csvis absent the page still renders, but every package then falls to the "On r-universe" branch rather than showing a download figure, so the file is expected to be present once the job has run.