Skip to content

Fix local executor failing on output lines over 64KiB - #360

Merged
umputun merged 1 commit into
umputun:masterfrom
paskal:executor-local-long-lines
Aug 19, 2026
Merged

Fix local executor failing on output lines over 64KiB#360
umputun merged 1 commit into
umputun:masterfrom
paskal:executor-local-long-lines

Conversation

@paskal

@paskal paskal commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Previously, Local.Run split the captured output with a default bufio.Scanner, so a command printing a single line longer than 64KiB failed with bufio.Scanner: token too long even though the command itself had succeeded. The remote executor has no such limit, since Remote.sshRun splits the buffered output with strings.SplitSeq, so the same playbook behaved differently depending on whether it ran locally or over ssh.

On master:

$ cat longline.sh
#!/bin/sh
awk 'BEGIN{printf "%100000s\n", ""}'

$ spot --local -p longline.yml -t localhost
[0] failed command "emit one long line" on host localhost:0 (localhost): can't run script on localhost:0: bufio.Scanner: token too long

After this change the same playbook completes. The output is fully buffered in memory by the time it is split, so the scanner's bounded reads bought nothing, and splitting the buffer in place returns a line of any length as it is. The splitting keeps bufio.ScanLines semantics, dropping a trailing \r and the empty element after the final newline, so nothing changes for existing playbooks; TestSplitOutputLines pins that down case by case.

@paskal
paskal requested a review from umputun as a code owner August 19, 2026 15:50
Previously, Local.Run split the captured stdout with a default
bufio.Scanner, so a command printing a single line longer than 64KiB
failed with "bufio.Scanner: token too long" even though the command
itself succeeded. The remote executor has no such limit, so the same
playbook behaved differently depending on whether it ran locally or
over ssh.

The buffered output is now split in place, keeping bufio.ScanLines
semantics: a trailing CR is dropped and the empty element after the
final newline is not returned.
@paskal
paskal force-pushed the executor-local-long-lines branch from 18c826b to cd4eda8 Compare August 19, 2026 16:01

@umputun umputun left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not blocking, merging as is.

the new helper keeps blank lines, while Remote.sshRun (remote.go:361) and Dry.Run (dry.go:30) drop them with if line != "", so the three executor.Interface implementations now return different line shapes for the same stdout. git log -S SplitSeq shows the only commit that touched those loops is 7640456, a mechanical Split -> SplitSeq rewrite, so nothing decided this deliberately.

two ways to go. Leave it and keep per-executor behavior, which costs nothing. Or unify on splitOutputLines so all three return the same shape, which changes Remote output: commands.go:564 joins with "; " and would start picking up empty segments. That would be a separate PR.

@umputun
umputun merged commit 7a7c484 into umputun:master Aug 19, 2026
1 check passed
@paskal
paskal deleted the executor-local-long-lines branch August 19, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants