Skip to content

Update the max size of protodelim.Unmarshal to 64MB - #21

Merged
yushan8 merged 9 commits into
mainfrom
update-bazel-query
Feb 25, 2026
Merged

yushan8 merged 9 commits into
mainfrom
update-bazel-query

Conversation

@yushan8

@yushan8 yushan8 commented Feb 24, 2026 •

Copy link
Copy Markdown
Contributor

Why?

  • Unmarshalling the proto with protodelim.UnmarshalFrom causes the proto max size to exceed on large query output error: run 1: query failed: proto: : message size 5531512 exceeded unmarshaler's maximum configured size 4194304
  • This error is not surfaced because the command hangs from reading from the stdout pipe. We should flush out the rest of the pipe to avoid the command from hanging.

What?

  • Keep reading from stdout until the stdout pipe is drained to avoid the command from hanging
  • Update the max proto size to 64MB

Test Plan

INFO: Running command line: bazel-bin/example/cmd/query-bench/query-bench_/query-bench <args omitted>
workspace:  /home/user/go-code
query:      //external:all-targets + deps(//...:all-targets)
runs:       1

2026-02-25T23:26:38.754Z        INFO    bazel/query.go:25       Querying Bazel{workspacePath 15 0 /home/user/go-code <nil>} {query 15 0 //external:all-targets + deps(//...:all-targets) <nil>}
2026-02-25T23:27:28.085Z        DEBUG   bazel/query.go:85       Parsed 2353361 targets from bazel query
run 1: bazel query: 49.332s  (2353361 targets)
run 1: targethasher: 43.097s  (2353616 targets)

@yushan8
yushan8 marked this pull request as ready for review February 24, 2026 22:56
@yushan8
yushan8 requested review from a team as code owners February 24, 2026 22:56
@yushan8 yushan8 changed the title Update bazel query to remove streaming and goroutine calls Update streaming implementation to keep reading from stdout to drain the pipe even on error Feb 25, 2026
@yushan8 yushan8 changed the title Update streaming implementation to keep reading from stdout to drain the pipe even on error Update the max size of protodelim.Unmarshal to 64MB Feb 25, 2026
@yushan8
yushan8 merged commit 7f03954 into main Feb 25, 2026
1 check passed
@yushan8
yushan8 deleted the update-bazel-query branch February 28, 2026 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants