Summary
On nf-quilt 1.0.1 + Nextflow 25.10.x, writeMapToPackage(config, 'config') still crashes when the config contains dynamic process-directive closures (e.g. cpus = { task.attempt * 2 }). The error is now a nesting-depth overflow instead of the empty-bean error from #332:
ERROR QuiltProduct writeMapToPackage.toJson failed: Document nesting depth (1001) exceeds the maximum allowed
(1000, from `StreamWriteConstraints.getMaxNestingDepth()`)
(through reference chain: ConfigMap["cpus"]->...["delegate"]->...->ConfigObject["process"]->ConfigObject["cpus"]->...["delegate"]-> [repeats])
Why #332's fix didn't resolve it
#332 disabled SerializationFeature.FAIL_ON_EMPTY_BEANS (present in 1.0.1, QuiltProduct.groovy:101-103). But FAIL_ON_EMPTY_BEANS was the only thing halting Jackson at the SecretsLoader$1 empty bean. With it disabled, Jackson no longer bails — it follows the closure's delegate/owner bean properties, which form a cycle back to config.process.cpus, and recurses until it hits the 1000-deep StreamWriteConstraints limit. Same root cause (serializing live Groovy closures from the config), new symptom.
Root cause in code
In collectMetadata(), the process block is removed after the config is serialized:
config.remove('executor')
config.remove('params')
config.remove('session')
writeMapToPackage(config, 'config') // process{} closures still present here
config.remove('process') // removed one line too late
Minimal reproducer
main.nf:
process FOO {
cpus { 1 * task.attempt } // any dynamic directive closure triggers it
script: "echo hi > out.txt"
}
workflow { FOO() }
nextflow.config:
plugins { id 'nf-quilt@1.0.1' }
process.cpus = { 1 * task.attempt }
Run with a quilt+s3://… publish target on Nextflow 25.10.x → writeMapToPackage.toJson failed: Document nesting depth (1001)… at onComplete.
Suggested fixes (either)
- Move
config.remove('process') above writeMapToPackage(config, 'config') so the closures are never serialized; or
- Stringify dynamic directives / strip non-serializable closures before
toJson, rather than relying on FAIL_ON_EMPTY_BEANS to (no longer) guard the cycle.
Environment
- nf-quilt 1.0.1, Nextflow ≥25.10.0, AWS Batch / Slurm executor
- Config uses standard nf-core dynamic resource closures (
cpus = { N * task.attempt })
Summary
On nf-quilt 1.0.1 + Nextflow 25.10.x,
writeMapToPackage(config, 'config')still crashes when the config contains dynamic process-directive closures (e.g.cpus = { task.attempt * 2 }). The error is now a nesting-depth overflow instead of the empty-bean error from #332:Why #332's fix didn't resolve it
#332 disabled
SerializationFeature.FAIL_ON_EMPTY_BEANS(present in 1.0.1,QuiltProduct.groovy:101-103). ButFAIL_ON_EMPTY_BEANSwas the only thing halting Jackson at theSecretsLoader$1empty bean. With it disabled, Jackson no longer bails — it follows the closure'sdelegate/ownerbean properties, which form a cycle back toconfig.process.cpus, and recurses until it hits the 1000-deepStreamWriteConstraintslimit. Same root cause (serializing live Groovy closures from the config), new symptom.Root cause in code
In
collectMetadata(), theprocessblock is removed after the config is serialized:Minimal reproducer
main.nf:nextflow.config:plugins { id 'nf-quilt@1.0.1' } process.cpus = { 1 * task.attempt }Run with a
quilt+s3://…publish target on Nextflow 25.10.x →writeMapToPackage.toJson failed: Document nesting depth (1001)…atonComplete.Suggested fixes (either)
config.remove('process')abovewriteMapToPackage(config, 'config')so the closures are never serialized; ortoJson, rather than relying onFAIL_ON_EMPTY_BEANSto (no longer) guard the cycle.Environment
cpus = { N * task.attempt })