Skip to content

Adding rocpdsna project, V4 writer support and version selection changes at runtime-Part 1 - #5141

Closed
anujshuk-amd wants to merge 3 commits into
developfrom
users/anujshuk-amd/versioned-rocpd-sna-develop_v4Support
Closed

Adding rocpdsna project, V4 writer support and version selection changes at runtime-Part 1#5141
anujshuk-amd wants to merge 3 commits into
developfrom
users/anujshuk-amd/versioned-rocpd-sna-develop_v4Support

Conversation

@anujshuk-amd

@anujshuk-amd anujshuk-amd commented Apr 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Motivation

Rocpdsna currently only supports the V3 (3.0.0) schema. As the profiling ecosystem evolves, a newer V4 (4.0.0) schema is needed to support runtime selection of schema version. Changes support...

Timestamp normalization: V4 replaces direct start/end BIGINT columns with foreign keys to a rocpd_timestamp table, enabling deduplication and cross-table timestamp correlation.

Track-based context: V4 consolidates per-row nid/pid/tid/agent_id/queue_id/stream_id columns into a single track_id FK, reducing redundancy and simplifying queries.

Normalized event metadata: V4 moves embedded JSONB call_stack and line_info out of rocpd_event into dedicated rocpd_call_stack and rocpd_line_info tables with proper FK relationships.

New V4-only tables : rocpd_info_category, rocpd_info_address_range, rocpd_info_source_code, and rocpd_info_pc support richer profiling data.

Runtime schema selection: Users can choose V3 or V4 at runtime without recompilation.
This PR implements the V4 schema writers, adds comprehensive test coverage for both schemas, and cleans up the codebase.

Technical Details

  1. V4 Schema Writer Implementation : Implemented region_writer.hpp, kernel_dispatch_writer.hpp, memory_copy_writer.hpp, memory_alloc_writer.hpp, pmc_event_writer.hpp under source/writers/schema_v4/
  2. Insert Statement Layer
  • schema_v3/insert_statements.hpp: updated V3 insert SQL to align with the versioned backend schema under backends/schema/3.0.0/
  • schema_v4/insert_statements.hpp: new V4 insert SQL covering all 27 V4 tables, including normalized timestamp, track, call stack, line info, and category tables
  1. Entity Registry & Utilities updated
  2. Test Coverage: Added new test cases for v3 and v4 schema validation.
    The rocpdsna depends on [rocpd] Schema Updates #347 PR, the latest version of rocprofiler-sdk schema changes.

JIRA ID

TBA

Test Plan

Added new tests to support new changes.

Test Result

ll rocpd tests and unit tests are passing, and the benchmark is running without any error.

Submission Checklist

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Apr 17, 2026
@anujshuk-amd anujshuk-amd self-assigned this Apr 17, 2026
@anujshuk-amd anujshuk-amd changed the title Adding rocpdsna project-milan changes Adding rocpdsna project, V4 writer support and version selection changes at runtime-Part 1 Apr 17, 2026
@anujshuk-amd
anujshuk-amd marked this pull request as ready for review April 17, 2026 06:49
Copilot AI review requested due to automatic review settings April 17, 2026 06:49
Copilot AI reviewed Apr 17, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds the rocpdsna project with schema-versioned storage/writer plumbing and new SQL/query-builder infrastructure to support runtime selection between ROCpd schema v3 and v4.

Changes:

  • Introduces schema-tag/policy based writer implementations (v3 shown here) and runtime dispatch to v3/v4 at writer construction.
  • Adds a small query-builder library (SELECT/INSERT builders) and integrates it into the build.
  • Adds/updates schema SQL assets (v3 + v4) and supporting infrastructure (registry, validators, key providers, CMake modules, docs).

Reviewed changes

Copilot reviewed 107 out of 142 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
projects/rocpdsna/source/writers/schema_v3/writer_policy.hpp Adds v3 writer policy wiring schema tag, statements, and writer types
projects/rocpdsna/source/writers/schema_v3/region_writer.hpp Adds v3 region writer implementation
projects/rocpdsna/source/writers/schema_v3/pmc_event_writer.hpp Adds v3 PMC event writer implementation
projects/rocpdsna/source/writers/schema_v3/memory_copy_writer.hpp Adds v3 memory copy writer implementation
projects/rocpdsna/source/writers/schema_v3/memory_alloc_writer.hpp Adds v3 memory alloc writer implementation
projects/rocpdsna/source/writers/schema_v3/kernel_dispatch_writer.hpp Adds v3 kernel dispatch writer implementation
projects/rocpdsna/source/writers/schema_v3/common_insert_operations.hpp Adds v3 common insert ops (event/sample/arg/string helpers)
projects/rocpdsna/source/writers/region_writer.hpp Adds forward decl + interface include for region writer
projects/rocpdsna/source/writers/pmc_event_writer.hpp Adds forward decl + interface include for PMC event writer
projects/rocpdsna/source/writers/memory_copy_writer.hpp Adds forward decl + interface include for memory copy writer
projects/rocpdsna/source/writers/memory_alloc_writer.hpp Adds forward decl + interface include for memory alloc writer
projects/rocpdsna/source/writers/kernel_dispatch_writer.hpp Adds forward decl + interface include for kernel dispatch writer
projects/rocpdsna/source/writers/interfaces/region_writer_interface.hpp Adds CRTP interface for region writer
projects/rocpdsna/source/writers/interfaces/pmc_event_writer_interface.hpp Adds CRTP interface for PMC event writer
projects/rocpdsna/source/writers/interfaces/memory_copy_writer_interface.hpp Adds CRTP interface for memory copy writer
projects/rocpdsna/source/writers/interfaces/memory_alloc_writer_interface.hpp Adds CRTP interface for memory alloc writer
projects/rocpdsna/source/writers/interfaces/kernel_dispatch_writer_interface.hpp Adds CRTP interface for kernel dispatch writer
projects/rocpdsna/source/writers/interfaces/info_registration_writer_interface.hpp Expands info registration interface (incl. v4-only methods)
projects/rocpdsna/source/writers/interfaces/api_writer_base.hpp Adds CRTP base helper for writer interfaces
projects/rocpdsna/source/writers/info_registration_writer.hpp Adds forward decl + interface include for info writer
projects/rocpdsna/source/writers/common_insert_operations.hpp Adds forward declaration for common_insert_operations
projects/rocpdsna/source/writer_impl.cpp Implements runtime schema selection and explicit template instantiation for v3/v4
projects/rocpdsna/source/writer.cpp Implements writer_t API forwarding to impl
projects/rocpdsna/source/storage_impl.hpp Adds storage impl with schema version tracking
projects/rocpdsna/source/storage_impl.cpp Implements storage backend factory + schema version plumbing
projects/rocpdsna/source/storage.cpp Exposes storage_t constructors + get_storage_version
projects/rocpdsna/source/reader_impl.hpp Adds reader impl declaration and caches/utilities
projects/rocpdsna/source/reader.cpp Implements reader_t API forwarding to impl
projects/rocpdsna/source/queries/select/table_select_query.hpp Adds fluent SELECT query facade
projects/rocpdsna/source/queries/select/select_query_builders.hpp Adds SELECT builder chain types
projects/rocpdsna/source/queries/select/select_query_builders.cpp Implements SELECT builder chain behavior
projects/rocpdsna/source/queries/query_common.hpp Adds query common enums (sort_order/join_type)
projects/rocpdsna/source/queries/query_builder_base.hpp Adds shared query builder base utilities
projects/rocpdsna/source/queries/query_builder_base.cpp Implements query_builder_base methods
projects/rocpdsna/source/queries/insert/table_insert_query.hpp Adds fluent INSERT query facade
projects/rocpdsna/source/queries/insert/insert_query_builders.hpp Adds INSERT builder chain types
projects/rocpdsna/source/queries/CMakeLists.txt Adds rocpdsna_queries static library build
projects/rocpdsna/source/primary_key_providers.hpp Adds PK providers incl. v4-specific counters
projects/rocpdsna/source/json_serializers.hpp Adds JSON serializer API for callstack/source context
projects/rocpdsna/source/insert_validator.hpp Adds validator interface for required/optional FK checks and PK resolution
projects/rocpdsna/source/insert_validator.cpp Implements validator logic used by writers
projects/rocpdsna/source/entity_utility.hpp Adds generic entity registry utility with string_view optimization
projects/rocpdsna/source/entity_registry.hpp Adds entity registries (v3+ plus v4-only registries)
projects/rocpdsna/source/data_storage/schema_version.hpp Adds schema tag types for compile-time selection
projects/rocpdsna/source/data_storage/backends/schema/4.0.0/summary_views.sql Adds v4 summary views SQL
projects/rocpdsna/source/data_storage/backends/schema/4.0.0/rocpd_views.sql Adds v4 compatibility views SQL
projects/rocpdsna/source/data_storage/backends/schema/4.0.0/rocpd_shema.in Adds v4 schema header template
projects/rocpdsna/source/data_storage/backends/schema/4.0.0/rocpd_metadata.sql Adds v4 metadata insert SQL
projects/rocpdsna/source/data_storage/backends/schema/4.0.0/rocpd_indexes.sql Adds v4 index SQL
projects/rocpdsna/source/data_storage/backends/schema/3.0.0/summary_views.sql Adds v3 summary views SQL
projects/rocpdsna/source/data_storage/backends/schema/3.0.0/rocpd_views.sql Adds v3 compatibility views SQL
projects/rocpdsna/source/data_storage/backends/schema/3.0.0/rocpd_shema.in Adds v3 schema header template
projects/rocpdsna/source/data_storage/backends/schema/3.0.0/rocpd_indexes.sql Adds v3 index SQL (mostly commented)
projects/rocpdsna/source/data_storage/backends/schema/3.0.0/marker_views.sql Adds v3 marker views placeholder
projects/rocpdsna/source/data_storage/CMakeLists.txt Adds data_storage sources to rocpdsna-objects
projects/rocpdsna/source/common/types.hpp Adds internal owned key types + hashes used by registries
projects/rocpdsna/source/common/traits.hpp Adds common traits (string literal, optional, unordered_map, bind-type helpers)
projects/rocpdsna/source/common/string_conversions.hpp Adds to_string helpers used in errors/debug output
projects/rocpdsna/source/common/logger.hpp Adds spdlog-based logger with env configuration
projects/rocpdsna/source/common/directory.hpp Adds filesystem helpers for directory creation
projects/rocpdsna/source/common/debug.hpp Adds LOG_* macros gated by ROCPDSNA_ENABLE_LOGGING
projects/rocpdsna/source/autoincrementer.hpp Adds atomic autoincrementer used by PK providers
projects/rocpdsna/source/CMakeLists.txt Wires sources, adds queries/data_storage subdirs, links rocpdsna_queries
projects/rocpdsna/include/writer.hpp Adds public writer API incl. v4-only info registration methods
projects/rocpdsna/include/storage_types.hpp Adds version_t comparison helpers
projects/rocpdsna/include/storage.hpp Adds public storage API with schema version access
projects/rocpdsna/include/shared_types.hpp Adds shared callstack/source-context types (v4 normalization targets)
projects/rocpdsna/docs/graphs/writer_insert_flow.dot Adds writer insert flow diagram
projects/rocpdsna/docs/graphs/writer_context_dependencies.dot Adds writer context dependency diagram
projects/rocpdsna/docs/graphs/writer_architecture.dot Adds writer architecture diagram
projects/rocpdsna/docs/graphs/system_layers.dot Adds system layer diagram
projects/rocpdsna/docs/graphs/schema_statement_correspondence.dot Adds schema-to-statement correspondence diagram
projects/rocpdsna/docs/graphs/public_api_ownership.dot Adds API ownership diagram
projects/rocpdsna/docs/graphs/backend_abstraction.dot Adds backend abstraction diagram
projects/rocpdsna/cmake/sqlite3.cmake Adds bundled/system SQLite3 selection + amalgamation build
projects/rocpdsna/cmake/spdlog.cmake Adds bundled/system spdlog selection
projects/rocpdsna/cmake/rocprofiler-sdk-rocpd.cmake Adds schema header generation + optional SDK schema sourcing
projects/rocpdsna/cmake/rocpdsna-config.cmake.in Adds install package config template
projects/rocpdsna/cmake/rocm.cmake Adds ROCm prefix path helper
projects/rocpdsna/cmake/nlohmann_json.cmake Adds bundled/system nlohmann_json selection
projects/rocpdsna/cmake/gtest.cmake Adds bundled/system GTest selection
projects/rocpdsna/cmake/coverage.cmake Adds coverage targets and flags plumbing
projects/rocpdsna/cmake/benchmark.cmake Adds bundled/system benchmark selection
projects/rocpdsna/cmake/Findrocpdsna.cmake Adds find-module for rocpdsna targets
projects/rocpdsna/README.md Adds rocpdsna project README with build/install notes
projects/rocpdsna/.gitignore Adds project-local gitignore
projects/rocpdsna/.clang-tidy Adds clang-tidy config
projects/rocpdsna/.clang-format Adds clang-format config

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread projects/rocpdsna/source/writer_impl.cpp
{
// Default to v3 (backward compatible)
m_impl = std::make_unique<writer_impl_polymorphic<writer_policy_v3>>(ctx);
}
Comment thread projects/rocpdsna/source/reader_impl.hpp
Comment on lines +137 to +140
std::unordered_map<size_t, std::string> m_string_info_utility;

std::unordered_map<size_t, reader_types::node_info_ptr_t> m_node_info_utility;
std::unordered_map<size_t, reader_types::process_info_ptr_t> m_process_info_utility;
Comment thread projects/rocpdsna/source/common/types.hpp
Comment on lines +1 to +3
#include <cstddef>
#include <functional>
#include <string>
Comment thread projects/rocpdsna/source/common/types.hpp

struct agent_unique_id_t
{
std::optional<std::string> agent_type;
Comment thread projects/rocpdsna/include/writer.hpp
Comment on lines +21 to +22
writer_t(const writer_t&&) = delete;
writer_t& operator=(const writer_t&&) = delete;
Comment thread projects/rocpdsna/source/queries/insert/table_insert_query.hpp

query_columns_builder& set_table_name(const std::string& table_name)
{
m_ss.str("");
Comment thread projects/rocpdsna/source/queries/insert/insert_query_builders.hpp
Comment on lines +34 to +38
template <typename T>
std::enable_if_t<common::traits::is_string_literal<T>(), std::stringstream&>
process_value(T& value)
{
m_stream << "\"" << value << "\"";
Comment thread projects/rocpdsna/source/data_storage/backends/schema/4.0.0/rocpd_shema.in
Comment on lines +4 to +6
namespace rocpd {
namespace data_storage {
namespace schema_v4 {
Comment thread projects/rocpdsna/source/data_storage/backends/schema/3.0.0/rocpd_shema.in
Comment on lines +4 to +6
namespace rocpd {
namespace data_storage {
namespace schema_v3 {
Comment thread projects/rocpdsna/.clang-tidy

WarningsAsErrors: ''

HeaderFilterRegex: '.*/rocstorage/(include|source)/.*\.hpp$'
@anujshuk-amd

anujshuk-amd commented Apr 23, 2026

Copy link
Copy Markdown
Contributor Author

Refer: #5347

@anujshuk-amd

anujshuk-amd commented Apr 23, 2026

Copy link
Copy Markdown
Contributor Author

All the changes are #5347

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@adjordje-amd adjordje-amd Awaiting requested review from adjordje-amd
@mradosav-amd mradosav-amd Awaiting requested review from mradosav-amd

Assignees

@anujshuk-amd anujshuk-amd

Labels

documentation Improvements or additions to documentation organization: ROCm

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@anujshuk-amd