Skip to content

feat: accept any kernel slice arg in generated launchers - #1286

Closed
drbh wants to merge 1 commit into
NVlabs:mainfrom
drbh:generic-slice
Closed

drbh wants to merge 1 commit into
NVlabs:mainfrom
drbh:generic-slice

Conversation

@drbh

@drbh drbh commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

What this proposes

Allow generated sync launchers to accept custom KernelSliceArg / KernelSliceArgMut implementations in every slice position, including row-width arguments. Move the traits out of the async feature gate and demonstrate cudarc-owned memory.

Review blocker

Safe prepared launches enqueue work without retaining the Rust borrow through device completion. The pointer/length traits do not tell an allocation owner which stream or access mode was used, so they cannot establish ordering and lifetime guarantees for a general third-party allocation.

The cudarc example uses one default stream. Its adapter drops cudarc's event guard before submission and does not track writes, so it does not establish a safe general adapter for other streams.

The current design is recommended for rejection. A follow-up needs a stream-aware use/lifetime contract or a separately scoped unsafe interop API.

Verification

  • Host and macro suites pass with default and all features.
  • The example typechecks with cudarc's CUDA 13.0 binding override.
  • Normal example checking fails on installed CUDA 13.4 because cudarc 0.19.9 rejects that toolkit version.

Related: #968 and #1081.

Sync launchers now accept impl KernelSliceArg and impl KernelSliceArgMut, matching the generic async slice contracts.

RowWidth gains a defaulted buffer type parameter, so existing RowWidth::new(&mut c, n) calls remain unchanged. The slice traits are available without the async feature. Caller-defined views now typecheck through sync, async, and owned launchers.

Retargets the launcher half of NVlabs#1081 and NVlabs#968.

Signed-off-by: David Holtz <david.richard.holtz@gmail.com>

@nihalpasham nihalpasham left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would reject this version. The interop goal is useful, but safe prepared launches need more than a pointer and length:

  • These launchers enqueue work and return; the Rust borrow ends before the GPU necessarily finishes. The slice traits only promise validity for that borrow.
  • The cudarc adapter drops its event guard before the launch and uses the read accessor for writable arguments. cudarc therefore cannot track oxide's reads/writes or wait for them before freeing on another stream.
  • The example uses one default stream, which orders that example correctly. The generated safe API also accepts other streams, where that ordering is absent.

This needs a stream-aware use/lifetime contract, with read/write dependencies recorded around submission and storage kept valid through completion. An explicitly unsafe interop API could also be scoped separately. I haven't patched around the issue because it changes the abstraction itself.

The host and macro suites pass with default and all features. The example also needs a supported cudarc/toolkit selection: its normal check fails on CUDA 13.4 and passes with the CUDA 13.0 binding override.

@nihalpasham nihalpasham added host-apis Host-side runtime APIs (cuda-host, cuda-core, cuda-async) interop Interop with external compilers / frontends needs-changes Review found changes required before this PR can land safety Memory safety, soundness, or undefined behavior labels Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

host-apis Host-side runtime APIs (cuda-host, cuda-core, cuda-async) interop Interop with external compilers / frontends needs-changes Review found changes required before this PR can land safety Memory safety, soundness, or undefined behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants