Skip to content

Prevent \uXXXX escape leakage on paste and enforce UTF-8 song file persistence - #52

Merged
reckel-jm merged 4 commits into
masterfrom
copilot/fix-non-ascii-character-issue
Jun 6, 2026
Merged

reckel-jm merged 4 commits into
masterfrom
copilot/fix-non-ascii-character-issue

Conversation

Copilot AI commented May 31, 2026

Copy link
Copy Markdown
Contributor

Pasting into the song editor could persist non-ASCII text as escaped Unicode literals (e.g. \u4F60\u597D) instead of real characters. This change ensures pasted content is materialized as Unicode text and song files are consistently read/written as UTF-8.

  • Editor paste decoding

    • Added decoding for \uXXXX sequences during paste handling in TfrmDisplaySongContent.
    • Handles valid surrogate pairs (\uD800..\uDBFF + \uDC00..\uDFFF) and leaves invalid surrogate input unchanged.
  • Explicit UTF-8 file I/O in editor flows

    • Switched editor load/save paths to explicit TEncoding.UTF8 in editordisplaysongcontent.pas.
    • Applied UTF-8 writes to song copy/new-save related flow in songeditor.pas.
  • UTF-8 consistency in song/text persistence helpers

    • Updated song export path in lyrics.pas to write UTF-8 explicitly.
    • Updated generic text file wrapper in textfilehandler.pas to load/save with UTF-8.
procedure TfrmDisplaySongContent.memoCodePaste(Sender: TObject;
  var AText: String; var AMode: TSynSelectionMode; ALogStartPos: TPoint;
  var AnAction: TSynCopyPasteAction);
begin
  AText := DecodeEscapedUnicodeSequences(AText);
end;

memoCode.Lines.LoadFromFile(self.openFilePath, TEncoding.UTF8);
memoCode.Lines.SaveToFile(self.openFilePath, TEncoding.UTF8);

@reckel-jm
reckel-jm marked this pull request as ready for review June 6, 2026 12:59
Copilot AI review requested due to automatic review settings June 6, 2026 12:59
@reckel-jm
reckel-jm merged commit baa539f into master Jun 6, 2026
4 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to prevent pasted \uXXXX escape sequences from being persisted literally in the song editor, and to make song/text persistence consistently use explicit UTF-8 encoding across editor and helper flows.

Changes:

  • Added a paste-time decoder that converts \uXXXX (including surrogate pairs) into actual Unicode characters.
  • Switched editor load/save (and copy flow) to explicit TEncoding.UTF8.
  • Updated song export and generic text file helpers to read/write with TEncoding.UTF8.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/generics/textfilehandler.pas Uses explicit UTF-8 for generic text load/save.
src/generics/lyrics.pas Exports song files explicitly as UTF-8.
src/frames/editordisplaysongcontent.pas Adds \uXXXX paste decoding; editor load/save now explicit UTF-8.
src/forms/songeditor.pas Copy-to-new-song write now explicit UTF-8.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/generics/lyrics.pas
Comment on lines 490 to +493
try
outputcontent := TStringList.Create;
outputcontent.Text := self.ExportAsSongFile;
outputcontent.SaveToFile(outputfilename);
outputcontent.SaveToFile(outputfilename, TEncoding.UTF8);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants