Skip to content

project: collect, relink, trim and archive a project's files - #2211

Merged
jcelerier merged 5 commits into
masterfrom
score-packages
Aug 17, 2026
Merged

project: collect, relink, trim and archive a project's files#2211
jcelerier merged 5 commits into
masterfrom
score-packages

Conversation

@jcelerier

@jcelerier jcelerier commented Aug 15, 2026

Copy link
Copy Markdown
Member

Everything a project needs to stop depending on the machine it was authored on: collect its media, find what went missing, drop what nothing points at, and hand the whole thing to someone else as a zip.

One traversal, five policies

Process::FileOperation walks a document's external file references once and reports what it decided about each one (FileEntry / FileAction: Collect, AlreadyThere, KeptInLibrary, Relinked, Trimmed, Unused, Missing, Unsupported, Skipped, Failed). The five operations on top are ~40 lines each, because they only choose a policy — the walking, reporting, summarising and undo all live in one place:

  • Consolidate — copy referenced media into the project folder and repoint the document at it.
  • Re-anchor — rewrite stored paths relative to the project without moving anything.
  • Missing files — scan on load, non-modally, and offer to relink.
  • Unused files — find what sits in the project folder with nothing pointing at it.
  • Trim — rewrite media down to the part the document actually reads.

Because every operation produces the same report shape, FileReportView displays all five and the dialogs differ only in their wording.

How a process declares its files

ProcessModel::mapExternalFiles(ExternalFileMap&) is virtual, and the base implementation already covers file- and folder-valued control ports. Only a process that stores a path outside a port needs to override it — Sound, VST3, Faust, Pd, JS, Clap, Images, Video and the Libav device do — and an override still calls the base. Relocation goes through RelocateFile, so every repoint is one undoable command.

Safety

UnusedFiles moves to <project>/Unused/ rather than deleting, refuses anything outside the project folder, and refuses anything still referenced; scanning the whole folder rather than just known media is opt-in. ProjectArchive writes a zip via the new score/tools/Zip.*.

Tests

13 new files. Six integration tests drive the real dialogs and menu (ProjectConsolidationTest, MissingFilesTest, UnusedFilesTest, MediaTrimTest, ProjectArchiveTest, ProjectFilesMenuTest), three unit tests cover the pieces (ProjectFilesTest, ZipTest, FileIndexTest), plus a score_test::Project fixture and tests/tools/sandboxed-test.sh, which runs a test against a read-only filesystem so the failure paths are exercised rather than assumed.

🤖 Generated with Claude Code

https://claude.ai/code/session_014rZgzE8JjWvHDtaVUhxpLE

jcelerier and others added 5 commits August 16, 2026 18:10
@jcelerier @claude
project: collect a document's media into its own folder
4ca6a33 
Moving a score project between machines currently means hunting down every
media file by hand: paths are stored wherever the process felt like storing
them, and "Save as" into another folder silently redefines what <PROJECT>:
means without moving anything.

Add a consolidation pass that walks the document, copies everything it
references next to the .score file in a per-kind hierarchy (Audio/, Video/,
Images/, Models/, Scripts/...), and rewrites the references to <PROJECT>:.

The traversal is one pass that both reports and rewrites, so the plan shown
in the dialog is by construction what happens on accept. Processes report
their files through Process::ProcessModel::mapExternalFiles; the default
implementation already covers file- and folder-valued control ports, which is
what every avendish process and add-on uses, and models holding a path
elsewhere (sound, video, images, JS, Pd, Faust, VST3, CLAP) override it.
Devices go through a new Device::ProtocolFactory::relocateExternalFiles.

Relocations are ordinary commands bundled in one undoable macro, and each
process picks the command that suits it: reloading a Pd patch or a QML script
rebuilds the ports, so those go through EditScript, which restores the cables.

Placement is content-aware: a file referenced N times is copied once, two
different files with the same name do not collide (including on
case-insensitive filesystems), and a destination already holding identical
bytes is reused -- so a second consolidation is a no-op. Originals are never
moved or deleted, and an existing destination is never overwritten. Files
that cannot be found, plug-in binaries and Faust/QML include folders are
reported rather than silently mangled.

Also fixed along the way:
 - relativizeFilePath matched /proj against /projAB as a prefix;
 - QFileInfo::canonicalPath() answers "." for a file that does not exist, so
   an unsaved document anchored <PROJECT>: to the working directory;
 - Pd serialized its patch path absolutely, so Pd processes never survived a
   move at all;
 - saving into another folder left <PROJECT>: references pointing at nothing;
   score now offers to collect the media, or re-anchors them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSTuRjChdtQXJmLQ4Yf1qp
@jcelerier @claude
project: find missing files, archive to zip, trim media
10c5067 
Three operations that all turn out to be the same shape as consolidating --
walk the document's file references, decide something per reference, maybe
write to disk, maybe repoint -- so they are built as policies over one
traversal rather than three parallel implementations.

Process/FileOperation is that traversal: one report type (FileReport /
FileEntry / FileAction), one command macro, one dryRun flag that suppresses
every rewrite without changing a single reported field. Consolidating,
re-anchoring, scanning, relinking and trimming are now ~40-line policies
each, and FileReportView shows any of their reports.

Missing files. Scanned on every document load and surfaced in a non-modal
window listing what is gone and which process wanted it -- loading a project
must not open with a frightening modal, which is the first thing users of
every other implementation complain about. "Search a folder..." indexes it
recursively and proposes candidates, preferring an exact size match over a
mere name match and offering the rest rather than guessing; "Locate..." is
the manual escape hatch every relink UI ends up needing. Nothing is applied
until asked, and a relink into the project folder is stored relative, so
finding a file also makes the document more portable than it was.

Archive. score/tools/Zip wraps the miniz already vendored for the package
downloader; the archive is built from a consolidation report rather than by
listing the folder, so it holds exactly this document's files -- not the
neighbouring project, not the backups, not trimming's leftovers -- under one
top-level folder. It is written to a .part file and renamed on success: an
interrupted archive that looks finished is how backups get trusted and fail.

Trim. The one operation here that can destroy audio, so almost all of it is
refusals: only files inside the project folder, only regions the process can
bound (Media::Sound derives its from the same mapping the waveform is drawn
with), the union of every region reading a file, generous handles by default
because the first complaint about every trim feature is that it took away the
room to adjust a fade, and the result is discarded unless it is actually
smaller -- writing float WAV from a compressed source easily is not. New
files are written beside the originals, never over them; deleting the
originals is opt-in, confirmed separately, and even then leaves the file
consolidation copied from untouched. Left off, the untrimmed files simply
stop being referenced, so archives shrink anyway and undo still has something
to fall back on.

Trimming's tests run under tests/tools/sandboxed-test.sh: bubblewrap with the
filesystem read-only apart from a fresh /tmp and throwaway config dirs, so a
regression in the delete path cannot reach any real file. score_add_test
gained a SANDBOXED flag for it, and falls back to running unwrapped where
bubblewrap is missing.

Also fixed: Media::Sound::usedFileRange computed seconds through
TimeVal::toSample(1.), which truncates to whole seconds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSTuRjChdtQXJmLQ4Yf1qp
@jcelerier @claude
project: remove the files a project no longer uses
46668cd 
Consolidating copies media in and never takes it out, so a project folder
accumulates every idea that was tried and dropped. File > Remove unused
files... lists what is sitting there with nothing pointing at it, largest
first, and gets rid of what the user ticks.

Everything about this is built around the fact that "unused" is an inference,
not an observation -- it means nothing score knows about references the file:

 - It only looks inside the folders consolidation creates. The rest of a
   project folder holds renders, notes, stems and other people's work that
   score never put there and has no business proposing to delete; searching
   the whole folder is a checkbox, off by default.
 - Moving to <project>/Unused/ is the default, not deleting. Set aside, a file
   stops travelling in archives while remaining one drag from being back, and
   the next scan does not offer it again.
 - The list is checkable, and removeUnusedFiles takes the paths rather than
   recomputing them, so it cannot act on a file the user never saw.
 - It refuses, whatever it is told, anything outside the project folder,
   anything the document still uses, and any .score/.scorebin/.scorejson --
   including a neighbouring document's.
 - The situations where the inference is weaker than it sounds are printed
   above the list rather than left in a manual: references that cannot be
   found (the orphan may be what you were about to relink to), a second
   document sharing the folder, and undo history not being consulted.

Tests run sandboxed like the trim ones: read-only filesystem outside /tmp.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSTuRjChdtQXJmLQ4Yf1qp
@jcelerier @claude
project: group the file commands, and name the two shrinking ones apart
bdef978 
Keeping both trimming and unused-file removal means shipping two commands that
sound like each other. Cubase ships that exact pair -- Minimize Files and
Remove Unused Media -- and it is confusing there, so:

 - the five entries move into a File > Project files submenu, where the pair is
   read side by side rather than one at a time;
 - "Trim media to what is used..." becomes "Shorten media files to what is
   played...", against "Remove unused files...": one shortens files that stay,
   the other removes files that go;
 - each one's help text says what the other does instead.

The submenu goes in above the File menu's last separator. A plug-in's
additions land at the end of a menu, which for File is after Quit -- so the
consolidation entry has been sitting below Quit since it was added. A test
asserts the placement, since it depends on how another plug-in built the menu.

Also tested: the two commands composing. Trimming leaves the untrimmed file
behind unreferenced, which is precisely what makes it safe to keep, and the
cleanup then finds exactly that file and nothing the document still reads.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSTuRjChdtQXJmLQ4Yf1qp
@jcelerier @claude
project: fix the cross-platform build, and what reviewing it turned up
1defa58 
Every CI job failed on the same three things, none of which a build with a
precompiled header and unity off can see:

 - ExternalFiles.cpp and MediaTrim.cpp call ctx.app.interfaces<...>(), but
   DocumentContext only forward-declares GUIApplicationContext. The PCH was
   supplying it locally.
 - FileReportView.cpp and MissingFilesDialog.cpp each declared `enum Column`
   in an anonymous namespace. A unity build merges those into one namespace,
   so the second declaration collided with the first -- and unscoped
   enumerators collide even once the types are named apart, hence
   `enum class ReportColumn : int` following Explorer/Column.hpp.
 - ProjectFilesApplicationPlugin.cpp uses QCoreApplication::processEvents
   without including it; only gcc's include graph noticed.

Verified against a second build tree configured the way CI is
(-DCMAKE_UNITY_BUILD=1 -DSCORE_PCH=0), which reproduces all three, plus a
gcc-14 no-PCH syntax pass over each new file.

Reviewing the whole thing afterwards found six more, in rough order of how
much they would have cost:

 - analyzeUnusedFiles treated a folder reference as a single used path, so
   every file inside a folder a process points at -- a Faust import folder, an
   avendish folder port -- looked unused and was offered for deletion. Folder
   references now cover their contents, in both the scan and the removal
   guard, with a test that fails by deleting the files if the guard goes.
 - "Re-anchor project files" never registered as a command:
   score_generate_command_list_file matches SCORE_COMMAND_DECL with a regex
   whose character class has no '-', so the whole declaration was invisible
   and the command would have failed to deserialize.
 - MissingFilesDialog is deliberately non-modal, and held a DocumentContext&
   that dies with its document. It holds a QPointer<Document> now and closes
   itself instead.
 - DocumentManager reached the plug-ins through GUIAppContext(), which
   downcasts the running ApplicationInterface; score::MockApplication is not a
   GUI one. It goes through AppComponents() instead.
 - Every file operation pushed a macro named "Consolidate project files", so
   undo lied after a relink or a trim. One macro class per operation.
 - The trim dialog listed every image and video in the project as a skipped
   row with a confusing reason; a file no trimmer can read is now out of scope
   rather than declined. Two files used std::find without <algorithm>, the
   same class of bug as the CI failures. An archive of a project whose
   document sits outside the collected folder came out without the document.
   ExternalFileMap::refs was a second, unread copy of the report.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YSTuRjChdtQXJmLQ4Yf1qp
@coveralls

coveralls commented Aug 17, 2026

Copy link
Copy Markdown

Coverage Status

Coverage is 20.35% — score-packages into master. No base build found for master.

@jcelerier
jcelerier merged commit 2463f6b into master Aug 17, 2026
55 of 68 checks passed
@jcelerier
jcelerier deleted the score-packages branch August 17, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jcelerier @coveralls