Skip to content

writeMapToPackage.toJson fails with "nesting depth (1001) exceeds maximum (1000)" on NF 25.10.x — #332 masked, not resolved, the dynamic-closure case #336

Description

@Austin-s-h

Summary

On nf-quilt 1.0.1 + Nextflow 25.10.x, writeMapToPackage(config, 'config') still crashes when the config contains dynamic process-directive closures (e.g. cpus = { task.attempt * 2 }). The error is now a nesting-depth overflow instead of the empty-bean error from #332:

ERROR QuiltProduct writeMapToPackage.toJson failed: Document nesting depth (1001) exceeds the maximum allowed
(1000, from `StreamWriteConstraints.getMaxNestingDepth()`)
(through reference chain: ConfigMap["cpus"]->...["delegate"]->...->ConfigObject["process"]->ConfigObject["cpus"]->...["delegate"]-> [repeats])

Why #332's fix didn't resolve it

#332 disabled SerializationFeature.FAIL_ON_EMPTY_BEANS (present in 1.0.1, QuiltProduct.groovy:101-103). But FAIL_ON_EMPTY_BEANS was the only thing halting Jackson at the SecretsLoader$1 empty bean. With it disabled, Jackson no longer bails — it follows the closure's delegate/owner bean properties, which form a cycle back to config.process.cpus, and recurses until it hits the 1000-deep StreamWriteConstraints limit. Same root cause (serializing live Groovy closures from the config), new symptom.

Root cause in code

In collectMetadata(), the process block is removed after the config is serialized:

config.remove('executor')
config.remove('params')
config.remove('session')
writeMapToPackage(config, 'config')   // process{} closures still present here
config.remove('process')              // removed one line too late

Minimal reproducer

main.nf:

process FOO {
    cpus { 1 * task.attempt }       // any dynamic directive closure triggers it
    script: "echo hi > out.txt"
}
workflow { FOO() }

nextflow.config:

plugins { id 'nf-quilt@1.0.1' }
process.cpus = { 1 * task.attempt }

Run with a quilt+s3://… publish target on Nextflow 25.10.x → writeMapToPackage.toJson failed: Document nesting depth (1001)… at onComplete.

Suggested fixes (either)

  1. Move config.remove('process') above writeMapToPackage(config, 'config') so the closures are never serialized; or
  2. Stringify dynamic directives / strip non-serializable closures before toJson, rather than relying on FAIL_ON_EMPTY_BEANS to (no longer) guard the cycle.

Environment

  • nf-quilt 1.0.1, Nextflow ≥25.10.0, AWS Batch / Slurm executor
  • Config uses standard nf-core dynamic resource closures (cpus = { N * task.attempt })

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions