Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models

Overview

Current research on Vision-Language-Action (VLA) models predominantly focuses on enhancing generalization through reasoning techniques. While effective, these improvements increase computational complexity and inference latency. Furthermore, these mechanisms are typically applied indiscriminately, wasting resources on trivial tasks while failing to provide the uncertainty estimation necessary to prevent catastrophic failure on out-of-distribution scenarios. Inspired by human cognition, we propose an adaptive framework that dynamically routes VLA execution based on the complexity of the perceived state. Our approach transforms the VLA's vision-language backbone into an active detection tool by projecting latent embeddings into a set of parametric and non-parametric estimators. This allows the system to execute known tasks immediately (\textit{Act}), reason about ambiguous scenarios (\textit{Think}), and preemptively halt execution when encountering physical or semantic anomalies (\textit{Abstain}). We find that a Gaussian Mixture Model fitted to fused vision-language embeddings provides the most reliable task-complexity signal, combining visual novelty with instruction context and cross-modal compatibility. Evaluated on the LIBERO and LIBERO-PRO benchmarks as well as on a real robot, our fused configuration achieves up to 87.5% F1-score across two VLA backbones (SmolVLA and $\pi_0$), retains 83% with as little as 5% of training data, and surpasses state-of-the-art failure detectors.

Warning

The manuscript is under review, and the source code and dataset will be released after publication.

Caution

Temporary Citation (Preprint) - ArXiv: http://arxiv.org/abs/2510.26358

@article{izzo2026act,
  title={Act, Think or Abstain: Complexity-Aware Adaptive Inference for Vision-Language-Action Models},
  author={Izzo, Riccardo Andrea and Bardaro, Gianluca and Matteucci, Matteo},
  journal={arXiv preprint arXiv:2603.05147},
  year={2026}
}

Authors: Riccardo Andrea Izzo, Gianluca Bardaro and Matteo Matteucci
Location: AIRLab (The Artificial Intelligence and Robotics Lab of Politecnico di Milano)