Extended Configurations for Organizations

[mandreko-bitwarden]

I'm part of an organization with a large number of repositories. We are implementing zizmor in all of them, and we're finding that we have to copy a zizmor.yml configuration file to each repository and maintain them individually when we make changes.

I think it could be a good feature to allow a base configuration, like GitHub and other tools do in an organization-wide .github repository. Another good example is Renovate, which allows you to make a per-repository configuration, which extends a base configuration that you store in a single repository.

Locally, when we run zizmor, we can point to a shared zizmor.yml file, but when running the zizmor-action, it only knows to look in the local repository unless we do something to remotely pull a new config manually.

I'd love to see some functionality like this to help organizations create sane defaults, and then make minimal changes per-repository ;)

[funnelfiasco]

I like the idea of making configs more scalable.

Would you want this only in the action, or would you want to be able to do it from the CLI, too? It would require being online and (potentially, in the case of private repos) authenticating to the repo with the org-wide config, which isn't necessarily the worst thing, but it does add some complexity, IMO.

Also, because I had a similar conversation recently at $dayjob, how would you expect conflicting settings to work? For a dev tool, org -> repo -> flags is the typical approach, but for a security tool, it might be desirable for the org-level settings to take precedent (or for some org-level settings to take precedent and others to be overridable on a per-repo basis).

[mandreko-bitwarden]

I hadn't considered the CLI. I had only been envisioning it for the action so far, since I'm often deep in GitHub workflows ;)

As for conflicts, I would assume you'd have a priority of overrides. Repository specific settings would override the Organization settings. That way you could set up defaults at your organization level, and override specific repository settings per-repository as needed. But that scenario may only work for me, while not others.

[mandreko-bitwarden]

Oh man, I was wondering why I recognized your username :) Fellow hoosier and OG Mario Marathon fan. ❤️

[woodruffw]

Hi @mandreko-bitwarden, thanks for opening a discussion!

FWIW, the way Homebrew does this is by synchronizing a zizmor.yml configuration file across all organization repos with a custom workflow. I don't claim that this is the best way to solve this problem, but it's one that might be worth looking into 🙂

You can see an example of that here: Homebrew/brew-pip-audit#232 (not a zizmor update in that case, but the same principle holds).

There are a couple of interlocking things here:

• I'm broadly hesitant to add support for "hierarchical" configs, e.g. automatically discovering a zizmor config in {org}/.github. The reasons for this are spread across a few different open issues, but the TL;DR is that zizmor's config is really intended to be unique per input "group", since it references filenames that only make sense relative to an input group. That's something that could potentially be addressed in the future, but the upcoming release already has a significant refactor that's aimed at improving the group-config relationship.
• For zizmor-action specifically: I'd be happy to take a PR that exposes a config input! You'll still need to fetch your remote config manually, but you'll at least be able to put it wherever instead of needing to drop it somewhere zizmor knows how to discover.

Some relevant xrefs: #1055 #1094

[mandreko-bitwarden]

Thanks @woodruffw !

How Homebrew handles it is certainly interesting. That's always an option for us. I'll have to analyze that some more, but it seems fairly straightforward enough.

I totally get hierarchical being complex ;) It's more of a wishlist type thing. Maybe one day ;)

In regards to the proposed zizmor-action change, that could also suffice for our needs, while being way less complex. It's not as fancy, but would likely work. I imagine we'll just have an action that downloads the config from a central repo, and utilizes it there. Maybe my first PR could be modifying the zizmor-action to handle a remote config file, to handle the downloading in-action even...

[devantler]

I solved this with a Required Workflow that triggers a Reusable Workflow.

You can even make it opt-in or opt-out via Custom Properties.

1. Create Reusable Workflow that runs Zizmor with a specific configuration.
2. Require the Reusable Workflow to run via org-wide rulesets for all repos, or the ones with a Custom Property set to "Enabled".

It is a bit of an undocumented feature, but it works. Only grudge for me is Required Workflows do not support input, so if you need flexibility you need to duplicate Reusable Workflows, as you cannot call them with an input for dynamically deciding how Zizmor should behave.

[devantler]

I use the https://github.com/zizmorcore/zizmor-action, where you can feed in input as documented here: https://github.com/zizmorcore/zizmor-action#inputs

The Reusable Workflow behind the Required Workflow, then makes an opinionated set of choices (we control the input), and this then runs across the org.

Does zizmor support configuration via env vars?

On standing foot I am not sure, but I never had the need myself given the above.

[devantler]

"via env vars"

And do you mean GitHub Environment secrets/variables, or plain old regular shell env vars?

You can feed in GitHub Environment secrets/variables into Reusable Workflows, so it is possible to obtain some dynamicity that way.

https://docs.github.com/en/enterprise-cloud@latest/actions/how-tos/reuse-automations/reuse-workflows#passing-inputs-and-secrets-to-a-reusable-workflow

[jasonkarns]

shell env vars.

super-linter invokes zizmor via the cli (AFAIK), so in order to allow reusable workflows that run superlinter to be able to drive zizmor differently from one repo to the next, would mean the cli itself would have to respect the variables.

Or I suppose alternatively, using GHA workflow parameters that superlinter respects and passes along as zizmor cli arguments. Which could work but (I believe) would require changes to superlinter.

Tools whose CLIs already respect standard environment variables allow the most flexibility because they don't require every tool in the chain (like superlinter) to have knowledge of each configuration lever.

[devantler]

Ah, yeah I do not know to much about Super Linter. I do use Mega Linter (which is the same, but is implemented in Python instead of bash/shell). In Mega Linter you can do what you want. You have a declarative configuration file, where all its variables can also be fed in as shell env vars.

https://megalinter.io/9.6.0/configuration/ # see the first paragraph, which states shell env var support
https://megalinter.io/latest/descriptors/action_zizmor/ # see the possible configuration variables for Zizmor

I have not checked in depth, if the possible configuration variables for Zizmor in Mega Linter will allow you to configure the settings you aim to control, but your approach seems to be supported there.

[jasonkarns]

(which is the same, but is implemented in Python instead of bash/shell)

I don't think the implementation language is relevent. The important distinction is whether they both use the respective CLIs of the linters/tools they bundle; or if they are implemented by using the corresponding github actions for each tool. (That distinction would affect which settings are available to configure by the end user.) Based on the mega linter documentation, they also use the zizmor CLI, which means we're constrained by which env vars the zizmor respects or the cli command options/arguments that the superlinter and megalinter tools will pass through.

In this case, as you noted with the second link (https://megalinter.io/latest/descriptors/action_zizmor/) mega linter respects specific params which are then passed along to the CLI.

There is a generic ACTION_ZIZMOR_ARGUMENTS variable that megalinter passes along, which would (presumably) allow one to disable specific zizmor rules from the shared reusable workflow file (instead of each repo's calling workflow).

This is something that super-linter could provide as well, which would be useful. (A generic escape hatch where arbitrary CLI arguments could be passed.) But that workaround is only necesary due to the limitation in zizmor itself not respecting zizmor-specific env vars. (Which wouldn't require any custom logic in super linter or megalinter.)

https://megalinter.io/9.6.0/configuration/ # see the first paragraph, which states shell env var support

Env var support of superlinter or megalinter isn't really relevant here unless the underlying tool (zizmor) respects the env vars. We're not trying to configure superlinter or megalinter. We're trying to configure zizmor, and thus are limited by the env vars it respects.

[woodruffw]

#1585 highlights a similar set of user needs, so I've been thinking about this more. One possibility would be a --remote-config=<slug> flag. For example, zizmor --remote-config=acme/anvils would attempt to discover zizmor.yml (with the normal discovery rules) in acme/anvils, and would treat any discovered config as the global config for that zizmor invocation.

The downside to that is that it'd be a single global config, i.e. wouldn't allow merging with any repo-specific config. The latter might be possible as well, but would require a much larger rearchitecting of how zizmor loads and associates configs with its input groups.