[APPS] Support importing .backend.ts files as RPC proxies from frontend code#320
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
🎉 All green!❄️ No new flaky tests detected 🔗 Commit SHA: ed163fe | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fcb5014b68
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } catch (error) { | ||
| log.error( | ||
| `Failed to parse exports from ${id}: ${error instanceof Error ? error.message : String(error)}`, | ||
| ); | ||
| return undefined; |
There was a problem hiding this comment.
Throw when backend export analysis fails
Do not swallow export-analysis errors for .backend.* modules here. When extractExportedFunctions rejects (for example on default exports), this code logs and returns undefined, which leaves the original backend module untransformed; the frontend then bundles backend implementation code directly and no backend bundle is produced for that file. In practice this turns a configuration error into a silent runtime/security regression instead of a clear build failure.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d088b2594a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (!exportNames) { | ||
| return undefined; |
There was a problem hiding this comment.
Throw when backend transform cannot derive exports
When parseExportNames yields null (including the explicit default-export error path), this branch returns undefined, so the .backend.* module is left untransformed in the frontend build instead of failing fast. In that case no backend function is registered, but the original backend source can still be bundled client-side, which is a silent and hard-to-debug failure mode for an invalid backend module shape. This should raise a build error rather than skipping transformation.
Useful? React with 👍 / 👎.
| const backendFiles = discoverBackendFiles(context.buildRoot, log); | ||
| const backendOutputs = new Map<string, string>(); | ||
| const hasBackend = backendFunctions.length > 0; | ||
| const hasBackend = backendFiles.length > 0; |
There was a problem hiding this comment.
Derive backend-enabled state from discovered functions
This flag is computed from the initial glob scan only, but backend functions are actually populated later during transform. If transform discovers functions that the pre-scan misses (for example, due to scope/ignore mismatches), backendOutputs can be filled while hasBackend stays false, and the upload path skips adding backend/*.js assets. Using backendFunctions.length/backendOutputs.size at upload time avoids dropping built backend bundles.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5e0dda3366
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } catch (error) { | ||
| log.error( | ||
| `Failed to parse exports from ${id}: ${error instanceof Error ? error.message : String(error)}`, | ||
| ); | ||
| return null; |
There was a problem hiding this comment.
Surface backend export parse failures instead of skipping transform
When extractExportedFunctions throws (for example on export default, which this change explicitly disallows), parseExportNames catches the error and returns null, so the caller in index.ts silently skips proxy generation and function registration. In that case the .backend.* module is left untransformed, no backend bundle is produced, and frontend code can end up executing raw backend implementation instead of executeBackendFunction. This should fail the build (or at least the module transform) rather than degrading into a silent fallback.
Useful? React with 👍 / 👎.
| id, | ||
| context.buildRoot, | ||
| ); | ||
| backendFunctions.push(...functions); |
There was a problem hiding this comment.
Deduplicate backend function registrations across transforms
The transform hook appends discovered functions into a long-lived backendFunctions array every time a .backend.* module is transformed. In dev/HMR, the same file is re-transformed on edits, so this array accumulates duplicate entries indefinitely; then createDevServerMiddleware rebuilds a map from the full array on every request, making request-time work and memory usage grow with edit history. Replacing push-only accumulation with per-file replacement or dedup by encoded query name avoids this degradation.
Useful? React with 👍 / 👎.
| export interface BackendFunction { | ||
| /** The BackendFunctionRef identifying this function */ | ||
| ref: BackendFunctionRef; | ||
| /** Absolute path to the .backend.ts source file */ | ||
| entryPath: string; | ||
| } | ||
|
|
||
| const EXTENSIONS = ['.ts', '.js', '.tsx', '.jsx']; | ||
| export interface BackendFileInfo { |
There was a problem hiding this comment.
I'm going to combine these into one type
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4e7c072f07
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| log.error( | ||
| `Failed to parse exports from ${id}: ${error instanceof Error ? error.message : String(error)}`, | ||
| ); | ||
| return null; |
There was a problem hiding this comment.
Fail build when .backend export parsing fails
Returning null on export-analysis errors causes the transform hook to skip rewriting the .backend.* module, so unsupported patterns (for example export default) are silently shipped as normal frontend code instead of being blocked. In that case no backend function is registered, but the raw backend implementation is still importable by the client bundle, which can lead to incorrect runtime behavior and backend code leakage. This should surface as a hard error for .backend.* files rather than a soft skip.
Useful? React with 👍 / 👎.
| id, | ||
| context.buildRoot, | ||
| ); | ||
| backendFunctions.push(...functions); |
There was a problem hiding this comment.
De-duplicate backend function registrations by file
This append-only push makes backend registrations accumulate across repeated transforms of the same file (common in Vite dev/HMR). If exports are renamed or removed, stale entries remain in backendFunctions, so /__dd/* lookups can still resolve deleted query names and fail later during bundling instead of returning a clean 404; the array also grows indefinitely over edits. Track functions per module id and replace old entries on re-transform.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 887f0698fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| export function generateProxyModule(exports: ProxyExport[]): string { | ||
| const lines: string[] = []; | ||
|
|
||
| lines.push("import { executeBackendFunction } from '@datadog/apps-function-query';"); |
There was a problem hiding this comment.
Remove unresolved runtime import from generated proxy module
Every transformed .backend.* module now imports executeBackendFunction from @datadog/apps-function-query, but this package is not provided by this repo's plugin dependencies, so builds fail as soon as a frontend imports a backend proxy unless the app independently installs that package. In a standard Vite build this surfaces as Rollup failed to resolve import "@datadog/apps-function-query", which blocks production builds for the new feature path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Doesn't this mean we need to add @datadog/apps-function-query in all published's package.json?
(Well, at least @datadog/vite-plugin since we only support this one for now)
There was a problem hiding this comment.
I added @datadog/apps-function-query based on our conversation, but I think I'll want to bump the version of @datadog/apps-function-query to 0.0.2 and update this PR before landing it.
| if (node.type !== 'ExportNamedDeclaration') { | ||
| continue; |
There was a problem hiding this comment.
Reject unsupported export-all syntax in backend discovery
extractExportedFunctions silently ignores non-ExportNamedDeclaration nodes, so a backend file that uses export * from ... yields no discovered functions; the transform then returns undefined and leaves the original .backend source in the frontend graph instead of generating RPC proxies. This can accidentally ship backend implementation code to the client and bypass backend bundling for those exports, so unsupported export forms should throw explicitly (or be fully resolved) rather than falling through.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 49bf9b74ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (exportNames.length === 0) { | ||
| return undefined; |
There was a problem hiding this comment.
Reject unsupported .backend exports instead of passthrough
If a matched .backend.* module uses an unsupported export form (for example export * from './impl'), extractExportedFunctions yields no names and this branch returns undefined, so the file is not replaced by a proxy at all. That means frontend imports can bundle and execute the original backend implementation directly, while no encoded function ref is registered for the dev middleware/build pipeline. This should fail closed (throw an error) rather than silently passing through backend code.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 58266aa2a4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const functions = getBackendFunctions(); | ||
| if (functions.length > 0) { | ||
| backendOutDir = await buildBackendFunctions( |
There was a problem hiding this comment.
Clear backendOutputs before each closeBundle rebuild
closeBundle reuses backendOutputs across runs but never resets it before calling buildBackendFunctions. If backend exports are renamed/removed between rebuilds (e.g., watch mode), stale map entries still reference files from the previous temp output directory, which is deleted in the finally block; handleUpload can then attempt to archive missing or stale backend artifacts. Resetting/recomputing backendOutputs per run avoids cross-build contamination.
Useful? React with 👍 / 👎.
| if (exportNames.length === 0) { | ||
| return undefined; |
There was a problem hiding this comment.
Remove stale backend registry entries on empty exports
When a .backend.* module has no named exports, the transform handler returns early without updating the registry. After an HMR/edit where a file drops all exports, the old functions for that file remain registered, so requests can still resolve removed query names and later rebuilds may attempt to bundle functions that no longer exist. Update the file’s registry entry to an empty list before returning.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d95641706
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (exportNames.length === 0) { | ||
| return undefined; |
There was a problem hiding this comment.
Clear registry when a backend module has no exports
When transform.handler processes a .backend.* file with no named exports, it returns before updating the per-file registry. In watch/HMR scenarios, if a file previously exported backend functions and is edited to only type exports (or no exports), the old entries remain in getBackendFunctions(), so stale function refs can still be bundled/served and may later fail against removed symbols.
Useful? React with 👍 / 👎.
| if (decl.type === 'VariableDeclaration') { | ||
| return decl.declarations | ||
| .filter((d): d is typeof d & { id: Identifier } => d.id.type === 'Identifier') | ||
| .map((d) => d.id.name); |
There was a problem hiding this comment.
Reject non-callable variable exports during function discovery
namesFromDeclaration treats every exported variable identifier as a backend function name, even when the initializer is not callable. For example, export const VERSION = '1' will generate a proxy callable and a backend bundle that executes VERSION(...args), which fails at runtime. Discovery should limit variable exports to callable forms (or error for non-functions) to avoid shipping broken RPC endpoints.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1399c1cf64
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ); | ||
| } | ||
| // handles: export { add, multiply } | ||
| names.push(spec.exported.name); |
There was a problem hiding this comment.
Validate
export { ... } bindings are callable
extractExportedFunctions currently registers every export specifier name as a backend function without checking what it refers to. That means const VERSION = '1'; export { VERSION }; is accepted, a proxy VERSION(...args) is generated, and backend execution fails at runtime because VERSION is not callable. Declaration exports already guard against non-functions, so specifier exports should be validated the same way (or rejected) to avoid shipping broken RPC wrappers.
Useful? React with 👍 / 👎.
| if (node.type !== 'ExportNamedDeclaration') { | ||
| continue; |
There was a problem hiding this comment.
Reject
export * in backend files
All non-ExportNamedDeclaration nodes are skipped, so a .backend.ts module using export * from './impl' yields no discovered functions. The transform path treats that as "no backend exports" and leaves the original module untransformed, so frontend imports don't get RPC proxies and backend code can pass through into the client bundle. ExportAllDeclaration should be handled explicitly with a build-time error instead of being ignored.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
@sdkennedy2 Maybe it's better to throw here for now, and not support this pattern? WDYT?
There was a problem hiding this comment.
It's most likely the same issue for export class MyClass....
There was a problem hiding this comment.
I'd be down to throw. This seems like a good idea.
There was a problem hiding this comment.
This should be fixed now because we throw when we receieve export * from './impl'
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9669dc867e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // handles: export { add, multiply } | ||
| names.push(spec.exported.name); |
There was a problem hiding this comment.
Reject non-callable export specifiers
The export-specifier path currently trusts any identifier in export { ... } and registers it as a backend function without checking whether it is callable. In a .backend.* file, const VERSION = '1.0'; export { VERSION }; will pass discovery here, generate a proxy, and later fail at runtime when the backend virtual entry executes VERSION(...args). This should be rejected during discovery the same way non-function export const declarations are rejected.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
We name handle local declarations so that if you have:
const VERSION = '1.0'; export { VERSION }; It will correctly check if the VERSION variable declaration is callable. In this case we will throw.
The only area thats a little shaky is we allow the following but we don't verify that handler is callable. It seemed like a reasonable tradeoff though. Alternatively we could throw in this scenario.
import {handler} from '..';
export {handler};| if (exportNames.length === 0) { | ||
| // Clear any previously registered functions for this file | ||
| // so stale entries don't persist across HMR re-transforms. | ||
| setBackendFunctions(id, []); | ||
| return undefined; |
There was a problem hiding this comment.
Prevent untransformed .backend modules from reaching frontend
When a .backend.* file has zero discovered exports, the transform clears the registry but returns undefined, which leaves the original module source in the frontend bundle instead of replacing it with a proxy/empty module. In practice, temporary edits or unsupported export forms can cause backend-only code to be shipped to the client unexpectedly; the hook should still return a transformed module (or throw) after clearing stale registry state.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
There seems to be a gap here (ref codex's review).
| const functions = getBackendFunctions(); | ||
| if (functions.length > 0) { | ||
| backendOutDir = await buildBackendFunctions( |
There was a problem hiding this comment.
Clear stale backend outputs before each bundle close
closeBundle now reads a dynamic function set via getBackendFunctions(), but backendOutputs is reused across runs and never reset. In watch/HMR rebuilds, removing or renaming a backend export leaves old entries in the map, and those paths point to temp files deleted after the previous run, so handleUpload() can try to archive stale/missing backend files. Resetting backendOutputs before rebuilding avoids stale uploads and intermittent archive failures.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Given that we specifically support the dev mode, this looks legit too.
Summary
Testing
|
…on discovery Add BackendFunctionRef, hashed query names, multi-export discovery, and proxy codegen. Uses unplugin's transform hook with this.parse() instead of esbuild+acorn for AST parsing. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
…st, orphaned JSDoc
apps-runtime.ts imports from backend/client and runs in the browser, so it belongs in the DOM compilation context alongside the client code. Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
fe42d08 to
d564740
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d564740ec9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| context.inject({ | ||
| type: 'file', | ||
| position: InjectPosition.MIDDLE, | ||
| value: path.join(__dirname, './apps-runtime.mjs'), |
There was a problem hiding this comment.
Point runtime injection to a file available in source builds
context.inject currently references path.join(__dirname, './apps-runtime.mjs'), but this change only adds src/built/apps-runtime.ts (the .mjs artifact exists only after packaging). When the plugin is executed from source (workspace/test/local-link flows), the injected file path does not exist, so the runtime global is not installed and generated .backend.* proxies will fail at call time when reading globalThis.DD_APPS_RUNTIME.executeBackendFunction.
Useful? React with 👍 / 👎.

Motivation
The previous backend function discovery model required files to live in a fixed
backend/directory, with function names derived from filenames. This was rigid — it coupled project layout to the plugin's conventions, only supported one export per file, and required discovery to happen synchronously before the build started.This PR replaces that with unplugin's
transformhook, which leverages the bundler's built-in parser (this.parse()) to discover exports lazily as files are processed. Backend files are now identified by a.backend.{ts,tsx,js,jsx}extension and can live anywhere in the project tree, with multiple named exports per file.It also absorbs the backend-call runtime that used to live in the separately-published
@datadog/apps-function-querypackage into the apps plugin itself — removing one published dependency and letting the apps plugin own the full codegen-to-runtime path (stacked on #322).Changes
Transform-hook discovery (
index.ts)The plugin registers a
transformhook with a filter matchingBACKEND_FILE_RE(.backend.{ts,tsx,js,jsx}). For each matching file, it:this.parse(code)(provided by unplugin — uses the bundler's own parser).extractExportedFunctions(), which walks the ESTree AST and rejects default exports.BackendFunctionentries in a registry (keyed byentryPathfor HMR re-transform safety).Backend function registry (
createBackendFunctionRegistry)A
Map<entryPath, BackendFunction[]>tracks all discovered functions. Keyed by absolute file path so HMR re-transforms replace stale entries instead of duplicating them. When a file is re-transformed with no exports, the registry entry is cleared.Proxy codegen (
backend/proxy-codegen.ts)Generates the frontend replacement module. Each exported function becomes a thin wrapper that reads from a runtime exposed on
globalThis.DD_APPS_RUNTIMEby a new injected script:No
importstatement in the generated code — the runtime is delivered by the apps plugin itself, not a user-facing npm package.Runtime injection (
src/built/apps-runtime.ts+src/index.ts)A new file
src/built/apps-runtime.tsimportsexecuteBackendFunctionfromsrc/backend/client/(copied over in #322) and assigns it toglobalThis.DD_APPS_RUNTIMEas a side-effect module. The apps plugin'sgetPlugins()callscontext.inject({ type: 'file', position: InjectPosition.MIDDLE, value: path.join(__dirname, './apps-runtime.mjs') })— the same mechanism@dd/rum-pluginuses for its SDK init. ThetoBuildentry in@dd/apps-plugin/package.jsonmakes the published@datadog/vite-pluginrollup pipeline also build the runtime as a sibling file indist/src/apps-runtime.mjs.InjectPosition.MIDDLEis used instead ofBEFOREbecause Vite's dev server only injects MIDDLE content (it does so viatransformIndexHtml, emitting a<script type="module" src="/@id/__datadog-helper-file">in head-prepend). BEFORE is served through Rollup'sbanner()output hook which only fires at build time, leaving the runtime undefined duringvite(dev).File organization
Backend-function code is organized into two directories:
proxy-codegen.tsandencodeQueryName.tsare build-time only (pure string/hash operations). Theclient/subtree ships to the browser viaapps-runtime.mjs. This separation keeps thevite/directory for genuinely vite-specific code (dev-server middleware, build config, vite sub-plugin).Opaque query names (
backend/encodeQueryName.ts)SHA-256 hashes the file path so backend file structure never leaks into frontend bundles. Format:
{sha256(relativePath)}.{exportName}.Removed
backendDirconfig option (no longer needed — the.backend.*file extension replaces it).discoverBackendFunctionsthat scanned a fixed directory before the build.BackendFunctionReftype (flattened intoBackendFunction).@datadog/apps-function-queryruntime dependency from@datadog/vite-plugin/package.jsonand its integrity exception inpackages/tools/src/commands/integrity/dependencies.ts.QA Instructions
Local dev:
cd ~/dd/build-plugins && yarn devin one terminal (runsprepare-link+ auto-rebuild watcher).dd-auth --domain="dd.datad0g.com" --actions-api -- npx vite— open in browser.window.DD_APPS_RUNTIME.executeBackendFunctionis a function;POST /__dd/executeActionreturns 200; proxy calls return expected data.Unit:
yarn workspace @dd/tests test:unit packages/plugins/apps— 13 suites / 136 tests.End-to-end verified locally against
dd.datad0g.comusingtest-action-catalog-app: both a simplegetGreetingbackend function and an Action-Catalog-backedlistHostsfunction returned expected responses with zero console errors.Blast Radius
@dd/apps-pluginand the published@datadog/vite-plugin.backend/myHandler.tstomyHandler.backend.tswith explicit named exports.backendDirconfig option removed.@datadog/apps-function-querydep is removed from@datadog/vite-plugin— consumers no longer need it transitively; generated proxies use the injected runtime instead.@datadog/apps-function-queryfrom their node_modules will resolve cleanly after this lands.globalThis.DD_APPS_RUNTIMEbecomes a reserved global name on any app using backend functions. Matches theDD_RUMprecedent.Documentation