feat: improve data migration flow

[hardy-dev-infinilabs]

Summary

• consolidate the data migration changes from console_framework into one business-scoped PR
• improve pipeline task result visibility for migration runs, including last run state and surfaced errors
• harden migration execution with partitioning, bulk indexing continuation/offset handling, disk queue cleanup behavior, metadata/schema initialization, and config reload/path fixes

Included areas

• pipeline migration status/result handling
• elastic partitioning, index/action helpers, metadata/schema init
• bulk indexing migration flow and disk queue support
• config/env/orm fixes needed by the migration path

Excluded on purpose

• challenge login and security primitives
• instance enrollment / reverse channel / token communication work
• metrics-only collection changes