fix: read and write config files as UTF-8

[LHMQ878]

Description

open() without an explicit encoding uses the platform's preferred encoding, which on a stock Windows install is a legacy codepage — cp936 on a Chinese system, cp1252 on a Western one — not UTF-8. Every config file trae-agent reads is UTF-8, so any config holding a single non-ASCII byte (a Chinese comment, an accented name, an em dash) fails to load on those machines while loading fine on Linux/macOS.

Observed on Windows 11 with a cp936 locale, using a trae_config.yaml whose only non-ASCII content is a comment:

>>> Config.create(config_file="trae_config.yaml")
UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 4: illegal multibyte sequence

LegacyConfig is worse than a crash. It catches the decode error, prints a warning, and falls back to defaults — so every setting the user wrote is silently discarded:

Warning: Could not load config file trae_config.json: 'gbk' codec can't decode byte 0xad ...
max_steps -> 20   (the file said 42)

The user gets a working agent running on settings they never chose.

More Information

Four sites, all open() calls missing encoding=:

File	Line	Failure mode
trae_agent/utils/config.py	220	YAML config → UnicodeDecodeError, agent won't start
trae_agent/utils/legacy_config.py	90	JSON config → error swallowed, all settings silently reverted to defaults
trae_agent/agent/trae_agent.py	167	patch output, two ways (below)
trae_agent/tools/ckg/ckg_database.py	161, 184	CKG storage info, keyed by absolute codebase paths

The patch-output site fails in two distinct ways, both verified:

# a) a diff containing a character outside the codepage — hard crash
UnicodeEncodeError: 'gbk' codec can't encode character '✅' in position 34

# b) a diff containing non-ASCII text — written in codepage bytes,
#    so any UTF-8 reader (including the evaluation harness) rejects it
bytes on disk: b'diff --git a/x.py b/x.py\r\n+# \xd6\xd0\xce\xc4\xd7\xa2\xca\xcd\r\n'
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd6 in position 29

That output also shows a second defect: \r\n. Universal newline translation rewrites the diff's LF endings to CRLF on Windows, and git apply rejects a patch whose lines end in CRLF. The fix passes newline="" so the diff bytes reach disk unmodified. A comment records why.

ckg_database.py stores a map keyed by codebase_path.absolute().as_posix(). A codebase under a path containing non-ASCII characters — a Chinese username, an accented directory — breaks index reuse and rebuilds the CKG every run.

This is a consistency fix as much as a bug fix: tools/json_edit_tool.py (163, 178), tools/json_edit_tool_cli.py (79, 93) and utils/trajectory_recorder.py (226) already pass encoding="utf-8". The four sites above are the remaining text-mode open() calls in trae_agent/ that don't. docker_manager.py:73 opens in binary mode and is correctly left alone.

Validation

New test file tests/utils/test_config_encoding.py, 3 tests, matching the unittest style of the neighbouring test_config.py.

Reproducing this in CI needs care. Monkeypatching locale.getpreferredencoding does not work — CPython reads the locale encoding at the C level, so open() ignores the patched function (I verified this: the patched call still raised 'gbk' codec, not 'ascii' codec). Instead a legacy_default_encoding context manager replaces open with a shim that supplies a legacy codec in exactly the position CPython would supply the locale's — that is, only when the caller passed no encoding:

def shim(file, mode="r", *args, **kwargs):
    if "b" not in mode and kwargs.get("encoding") is None and len(args) < 2:
        kwargs["encoding"] = codec
    return real_open(file, mode, *args, **kwargs)

Code that names its encoding is untouched by the shim. So these tests fail on a machine of any locale when the encoding= is missing, and pass on a machine of any locale when it is present — they will catch a regression on the project's UTF-8 Linux runners, not only on a Chinese Windows box.

The LegacyConfig test asserts on the resulting values, not on an exception, because that path swallows the error — asserting max_steps == 42 is the only way to catch a silent revert.

Control experiment — implementation reverted to the unpatched open() calls, new tests kept:

2 failed, 1 passed
  FAILED test_yaml_config_with_non_ascii_loads_under_legacy_locale
         UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 4
  FAILED test_legacy_json_config_with_non_ascii_is_not_silently_discarded
         max_steps was 20, expected 42

The one that stays green is test_ascii_yaml_config_still_loads_under_legacy_locale — the ASCII case the old code got right and the fix must not regress.

With the fix: 3 passed. Together with the existing config tests: tests/utils/test_config.py tests/utils/test_config_encoding.py → 14 passed.

No regressions. Whole suite, excluding the three modules that need a live network endpoint (test_ollama_client_utils, test_openrouter_client_utils, test_google_client):

tree	result
this branch	60 passed, 5 failed
unmodified main	57 passed, 5 failed

Identical 5 failures on both — tests/tools/test_bash_tool.py (3) and tests/tools/test_json_edit_tool.py (2), all pre-existing and unrelated (pexpect is not available on Windows). The delta is exactly the 3 new tests.

ruff check → All checks passed. ruff format --check → 49 files already formatted. mypy on the new test file → Success, no issues.

Behaviour is unchanged for every config file that loaded correctly before, so no working setup shifts.

Linked Issues

None — I did not find an existing issue for this (searched encoding, UnicodeDecodeError, utf-8, gbk/cp936/codepage: 0 results). Happy to open one if you'd prefer the issue-first flow.