[Feature]: regenerate-metadata CLI (refresh Metadata sheet from DHIS2) by gqcorneby · Pull Request #409 · EyeSeeTea/Bulk-Load

gqcorneby · 2026-07-10T11:48:02Z

📌 References

Issue: #869dg8vj6
Depends on: [Feature]: add includeMetadataCodes flag for Metadata Code column #408 - required, must merge first (see below)

📝 Implementation

Add a regenerate-metadata use case + CLI script that refreshes only the Metadata sheet of an existing template from fresh DHIS2 metadata, leaving every other sheet (custom form, dropdowns, VBA) untouched.
SheetBuilder.generateMetadataOnly clears and refills the Metadata sheet in place (not deleted) so the sheet's codeName / VBA binding survives.
CLI flags: --dhis2-url, --auth, --input, --output, --form-id, --form-type, --language, plus the metadata-generation toggles --include-codes, --use-codes, --org-unit-short-name (all default off).
yarn regenerate-metadata alias; README documentation.
Bumps @eyeseetea/d2-api to 1.21.0.
Extracted MaintenanceSection from Settings page and added regenerate option there

🔥 Notes for the reviewer

📹 Screenshots/Screen capture

For testing, i used
PHSM - Policy Tracker.xlsm
this file where these metadata rows are removed

2026-07-29.12-27-11.0.02.-.1.11.mp4

📑 Others

Per-template flag (default false) that adds a "Code" column after "Metadata version" in the generated Metadata sheet, populated with item.code. Additive: existing Metadata columns and the =_<id> defined names are unchanged when off. Enables looking up option codes by name from the Metadata sheet (e.g. for building composite identifiers). Mirrors the generateMetadata wiring: entity -> TemplateWebRepository -> DownloadTemplateUseCase -> SheetBuilder. Adds the first SheetBuilder unit test covering the column on/off behaviour.

Regenerate only the Metadata sheet of a custom template from fresh DHIS2 metadata (with the Code column), leaving every other sheet — custom form, dropdowns, VBA — untouched, so it can be refreshed without rebuilding the whole template. - SheetBuilder.generateMetadataOnly(): loads the input workbook, clears the Metadata sheet IN PLACE (not delete, so its codeName/VBA binding survives) and refills it via fillMetadataSheet with includeMetadataCodes on. - Workbook.clear() and a cross-environment base64 loader/writer (fromBase64Data / writeToBase64) that work in both Node and the browser. - RegenerateTemplateMetadataUseCase wired in CompositionRoot; reusable from the web app. - src/scripts/regenerate-metadata.ts CLI (cmd-ts), same pattern as import-multiple-files.ts. - Unit test covering the refresh + preservation of other sheets.

Bakes in the ts-node --transpile-only --project src/scripts/tsconfig.json flags so the script runs with: yarn regenerate-metadata --dhis2-url ... --input ... --output ... --form-id ...

Expose the three SheetBuilder flags that affect Metadata-sheet output as CLI flags on regenerate-metadata, instead of hardcoding them in the use case: --include-codes (Code column), --use-codes (codes instead of names), --org-unit-short-name (short vs regular OU name). All default off. Other SheetBuilder params only affect full generate(), not metadata-only.

anagperal

thanks @gqcorneby ! Onle a few comments

anagperal · 2026-07-28T11:26:23Z

+            console.debug(`Regenerating Metadata for ${args.input} (form ${args.formId})`);
+
+            const outputBase64 = await compositionRoot.templates.regenerateMetadata(api, {
+                type: args.formType as DataFormType,


Instead of as you can declare:

formType: option({ type: oneOf(dataFormTypes), long: "form-type", defaultValue: () => dataFormTypeMap.trackerPrograms, description: "Data form type: trackerPrograms | programs | dataSets (default: trackerPrograms)", })

Use cmd-ts oneOf(dataFormTypes) so --form-type is validated at parse time and the DataFormType cast at the call site disappears. Also document why the use case imports Settings and SheetBuilder from webapp/logic; fixing the layering needs a wider refactor.

The Code column is written for org units too, but the fixture passed an empty organisationUnits list so that branch was never exercised. Addresses review feedback on #408.

Buffer does not exist in the browser bundle, so the helper threw "Buffer is not defined" for any caller outside Node. It had no callers until now, which is why this went unnoticed.

Shared by more than one use case now, so keep a single definition.

The import page's drop area is now a component, so other screens can reuse it instead of copying the render prop and its styles.

Identifies the template and its data form from the file alone, so a caller does not have to ask the user which template it is. AnalyzeTemplateUseCase resolves the same way but then reads every data value, which is unnecessary when only the template is wanted.

Adds a Regenerate template metadata row to the Maintenance list. The dropped file identifies the template, the generation flags are editable, and the regenerated workbook is downloaded; nothing is persisted. Until now this was only possible through the regenerate-metadata CLI, which needs a checkout and DHIS2 credentials on the command line. The Maintenance section moves out of SettingsFields into its own component to host the new row.

gqcorneby · 2026-07-29T04:44:34Z

Thanks @anagperal! Addressed the comments. The scope of the PR increased a bit 😅 After discussing with Miquel, we'll expose this regenerate in settings page as a maintenance function.

…rate-metadata # Conflicts: # i18n/en.pot # i18n/es.po # i18n/fr.po # i18n/pt.po # i18n/ru.po # src/test/sheetBuilder.spec.ts # src/webapp/logic/sheetBuilder.ts

bundlemon · 2026-07-29T04:49:22Z

BundleMon

No change in files bundle size

Groups updated (1)

Status	Path	Size	Limits
✅	Build Folder _./*/	1.74MB (+2.63KB +0.15%)	+20%

Final result: ✅

View report in BundleMon website ➡️

Current branch size history | Target branch size history

anagperal

thanks @gqcorneby ! a lot of work done here! Just a few comments

anagperal · 2026-07-29T08:46:14Z

+    const loading = useLoading();
+    const classes = useStyles();
+
+    const [resolution, setResolution] = useState<ResolutionState>({ type: "empty" });


This component has also the logic presentation here, move it to a custom hook to have only the render logic

anagperal · 2026-07-29T08:49:56Z

+        private excelRepository: ExcelRepository
+    ) {}
+
+    public async execute(file: File): Promise<ResolvedTemplate> {


This method shares some code with AnalyzeTemplateUseCase. Maybe you can extract it to a helper

I ruled out reusing AnalyzeTemplateUseCase itself but I missed the shared code. Thanks Ana!

AnalyzeTemplateUseCase and ResolveTemplateFromFileUseCase opened with the same load-template and resolve-data-form steps. Kept as two helpers rather than one so the resolve use case can wrap only the lookup in its friendly error, leaving the data form errors distinct. Neither helper handles errors, so both use cases surface the same messages as before. Addresses review feedback on #409.

The dialog is now render-only; the drop handling, flags, regeneration and dismissal guard live in useRegenerateMetadata, following the shape of useMaintenanceCleanup. Addresses review feedback on #409.

Moving strings between files reorders the extraction; msgmerge also re-fuzzed one entry onto an unrelated translation, cleared here so it falls back to English.

anagperal

thanks @gqcorneby !!

gqcorneby added 9 commits July 10, 2026 16:47

feat(templates): include org unit code in metadata sheet

189d3b4

test(templates): simplify sheetBuilder metadata specs

b46eb92

chore(scripts): add yarn regenerate-metadata alias

74c6f5b

Bakes in the ts-node --transpile-only --project src/scripts/tsconfig.json flags so the script runs with: yarn regenerate-metadata --dhis2-url ... --input ... --output ... --form-id ...

chore(deps): bump @eyeseetea/d2-api to 1.21.0

c0ba49c

refactor(templates): dedupe metadata code-column writes

e1dd2c8

docs(scripts): document regenerate-metadata CLI in README

a760ff0

gqcorneby changed the title ~~feat(scripts): regenerate-metadata CLI (refresh Metadata sheet from DHIS2)~~ [Feature]: regenerate-metadata CLI (refresh Metadata sheet from DHIS2) Jul 10, 2026

gqcorneby requested a review from MiquelAdell July 10, 2026 11:53

MiquelAdell requested a review from anagperal July 28, 2026 06:39

anagperal requested changes Jul 28, 2026

View reviewed changes

Base automatically changed from feature/metadata-code-column to development July 28, 2026 11:59

gqcorneby added 7 commits July 29, 2026 10:38

test(templates): cover Code column for org units

2836b71

The Code column is written for org units too, but the fixture passed an empty organisationUnits list so that branch was never exercised. Addresses review feedback on #408.

fix(utils): decode base64 without Buffer in getBlobFromBase64

860a383

Buffer does not exist in the browser bundle, so the helper threw "Buffer is not defined" for any caller outside Node. It had no callers until now, which is why this went unnoticed.

refactor(utils): extract cleanFormula to utils/string

a153e3b

Shared by more than one use case now, so keep a single definition.

refactor(webapp): extract shared TemplateDropzone component

dc5747e

The import page's drop area is now a component, so other screens can reuse it instead of copying the render prop and its styles.

gqcorneby added 2 commits July 29, 2026 12:46

Merge remote-tracking branch 'origin/development' into feature/regene…

fd2bfb2

…rate-metadata # Conflicts: # i18n/en.pot # i18n/es.po # i18n/fr.po # i18n/pt.po # i18n/ru.po # src/test/sheetBuilder.spec.ts # src/webapp/logic/sheetBuilder.ts

style(webapp): format import page mime type list

2d1fdb1

gqcorneby requested a review from anagperal July 29, 2026 04:48

anagperal requested changes Jul 29, 2026

View reviewed changes

gqcorneby added 3 commits July 29, 2026 19:52

refactor(webapp): move regenerate dialog logic to a hook

c392cc6

The dialog is now render-only; the drop handling, flags, regeneration and dismissal guard live in useRegenerateMetadata, following the shape of useMaintenanceCleanup. Addresses review feedback on #409.

chore(i18n): update catalogs after the hook and helper extraction

1f7498e

Moving strings between files reorders the extraction; msgmerge also re-fuzzed one entry onto an unrelated translation, cleared here so it falls back to English.

gqcorneby requested a review from anagperal July 29, 2026 11:58

anagperal approved these changes Jul 29, 2026

View reviewed changes

MiquelAdell approved these changes Jul 30, 2026

View reviewed changes

MiquelAdell merged commit 14feb4e into development Jul 30, 2026
11 checks passed

MiquelAdell deleted the feature/regenerate-metadata branch July 30, 2026 07:47

Navigation Menu

Uh oh!

[Feature]: regenerate-metadata CLI (refresh Metadata sheet from DHIS2) - #409

[Feature]: regenerate-metadata CLI (refresh Metadata sheet from DHIS2)#409
MiquelAdell merged 21 commits into
developmentfrom
feature/regenerate-metadata

gqcorneby commented Jul 10, 2026 •

edited

Loading

Uh oh!

anagperal left a comment

Uh oh!

Uh oh!

anagperal Jul 28, 2026 •

edited

Loading

Uh oh!

gqcorneby commented Jul 29, 2026

Uh oh!

bundlemon Bot commented Jul 29, 2026 •

edited

Loading

Uh oh!

anagperal left a comment

Uh oh!

anagperal Jul 29, 2026

Uh oh!

anagperal Jul 29, 2026

Uh oh!

gqcorneby Jul 29, 2026

Uh oh!

anagperal left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

gqcorneby commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📌 References

📝 Implementation

🔥 Notes for the reviewer

📹 Screenshots/Screen capture

📑 Others

Uh oh!

anagperal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

anagperal Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gqcorneby commented Jul 29, 2026

Uh oh!

bundlemon Bot commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

BundleMon

Uh oh!

anagperal left a comment

Choose a reason for hiding this comment

Uh oh!

anagperal Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

anagperal Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

gqcorneby Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

anagperal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

gqcorneby commented Jul 10, 2026 •

edited

Loading

anagperal Jul 28, 2026 •

edited

Loading

bundlemon Bot commented Jul 29, 2026 •

edited

Loading