Vopp project

_data/repositories.yml

@@ -6,4 +6,5 @@ github_repos:
   - RDLLab/posggym
   - RDLLab/oppt
   - RDLLab/tapir
-  - AdaCompNUS/sarsop
+  - AdaCompNUS/sarsop
+  - RDLLab/VOPP

_projects/11_project_rdl.md

@@ -2,7 +2,7 @@
 layout: page
 title: "Vectorized Online POMDP Planning (VOPP)"
 center_title: true
-description: "VOPP: a fully GPU-parallelized online POMDP solver that is at least 20× faster than the state-of-the-art parallel solver, using pure tensor operations with no synchronization overhead."
+description: "VOPP: a fully vectorized online POMDP solver that is at least 20× faster than the state-of-the-art parallel solver, using pure tensor operations with no synchronization overhead."
 img: assets/img/project_img/vopp/vopp_overview.png
 importance: 11
 category: active
@@ -44,11 +44,15 @@ do_not_show_post_desc: true
 <!--h2>Abstract</h2-->
 
 <p class="text-justify">
-<strong>Vectorized Online POMDP Planner (VOPP)</strong> is a fully vectorized online POMDP solver that performs planning entirely through batched tensor operations. The Partially Observable Markov Decision Process (POMDP) is a powerful framework for sequential decision-making under uncertainty, capturing the stochastic effects of actions and the noisy, partial information available to autonomous robots. POMDP solving could benefit enormously from massive parallelization on modern multicore hardware, but parallelizing online POMDP solvers has remained difficult: most approaches interleave numerical optimization to find actions with the highest expected total reward and estimation of expected total rewards themselves. This creates dependencies and synchronization bottlenecks during planning that can quickly diminish the gains of parallelization.
+<strong>Vectorized Online POMDP Planner (VOPP)</strong> is a fully vectorized online POMDP solver that performs planning entirely through batched tensor operations running purely CPUs, or purely on GPUs. 
 </p>
 
 <p class="text-justify">
-VOPP takes a fundamentally different approach. It builds on a recent POMDP formulation &mdash; <strong><a href="/projects/7_project_rdl/">Partially Observable Reference Policy Programming (PORPP)</a></strong> &mdash; which introduces <strong>analytical</strong> value functions. This eliminates the need for numerical optimization during planning, leaving only the estimation of expectations to be performed numerically, which can be parallelized efficiently. VOPP exploits this by implementing all online planning operations as fully vectorized computations over a tensor-based planning data structure. The result is a massively parallel online POMDP solver that fully harnesses the immense data-parallel throughput of modern multicore hardware, such as GPUs. In practice, VOPP computes policies using tens of thousands of parallel simulations with no explicit synchronization between simulations required.
+The Partially Observable Markov Decision Process (POMDP) is a powerful framework for sequential decision-making under uncertainty, capturing the stochastic effects of actions and the noisy, partial information available to autonomous robots. POMDP solving could benefit enormously from massive parallelization on modern multicore hardware, but parallelizing online POMDP solvers has remained difficult: most approaches interleave numerical optimization to find actions with the highest expected total reward and estimation of expected total rewards themselves. This creates dependencies and synchronization bottlenecks during planning that can quickly diminish the gains of parallelization.
+</p>
+
+<p class="text-justify">
+VOPP takes a fundamentally different approach. It builds on a recent POMDP formulation &mdash; <strong><a href="/projects/7_project_rdl/">Partially Observable Reference Policy Programming (PORPP)</a></strong> &mdash; which introduces <strong>analytical</strong> value functions. This eliminates the need for numerical optimization during planning, leaving only the estimation of expectations to be performed numerically, which can be parallelized efficiently. VOPP exploits this by implementing all online planning operations as fully vectorized computations over a tensor-based planning data structure. The result is a massively parallel online POMDP solver that fully harnesses the immense data-parallel throughput of modern multicore hardware. In practice, VOPP computes policies using tens of thousands of parallel simulations with no explicit synchronization between simulations required. VOPP is implemented entirely in PyTorch. Unlike previous parallel POMDP solvers, VOPP does not require custom CUDA kernels for GPUs or custom SIMD code for CPUs&mdash;it runs directly on GPUs or CPUs using PyTorch's standard tensor operations. 
 </p>
 
 <div class="row justify-content-center mt-3 mb-4">