feat(runtime,datasource): default-datasource factory seam + adopt pre-built drivers via one connect path (#3826)

.changeset/default-datasource-adopt-seam.md

@@ -0,0 +1,43 @@
+---
+"@objectstack/runtime": minor
+"@objectstack/service-datasource": minor
+"@objectstack/verify": patch
+---
+
+feat(runtime,datasource): the default-datasource connect seam accepts a host driver factory — adopt pre-built instances without forking the verdict (#3826)
+
+ADR-0062 D1's open-core convergence (#3869/#3886) left one structural question
+open: a host whose `default` needs a driver the shared factory cannot build —
+the cloud distribution's `turso`, or an instance pooled BEYOND one kernel (the
+cloud control-plane driver doubles as the proxy base of every environment
+kernel; per-environment drivers are cached across kernel rebuilds) — had only
+two options, both bad: stay on the legacy pre-built `DriverPlugin` path, whose
+connect verdict lives in `ObjectQLEngine.init()` (the second implementation
+#3826 exists to retire), or fork the connect orchestration. Either re-opens the
+#3741 → #3758 drift this whole line of work is about.
+
+Two additive pieces close it:
+
+- **`DefaultDatasourcePlugin` accepts an injected `IDatasourceDriverFactory`**
+  (defaults to the shared open-core factory, byte-for-byte unchanged when
+  omitted). The factory only changes what `create()` returns — the policy-free
+  init connect, `bootCritical` fail-fast, `OS_ALLOW_DRIVER_CONNECT_FAILURE`
+  escape hatch, and the start() replay into retained admin state are identical
+  either way, and the new tests pin that (an adopted instance that cannot
+  connect takes the exact same verdict).
+- **`createPrebuiltDriverFactory(driver, { driverId?, fallback? })`** in
+  `@objectstack/service-datasource` — the "adopt an existing driver" seam the
+  first #3826 pass found missing, landed AS a factory so it composes into the
+  one connect path instead of becoming a second entry point. `create()` returns
+  the SAME instance every call: construction, pooling, and reuse stay host
+  concerns; only the verdict converges. Not for the common case — a `default`
+  expressible as `{ driver, config }` should stay a plain definition.
+
+The `@objectstack/verify` dogfood harness now boots through
+`DefaultDatasourcePlugin` (declared `sqlite-wasm` definition) instead of a
+pre-built `DriverPlugin` — so the dogfood gate exercises the same declared
+-default connect path `objectstack dev`/`serve` use, which is the §Risk
+mitigation ADR-0062 promised ("behind the dogfood gate") and did not yet have.
+The degraded-boot parity guard stays: `ObjectQLEngine.init()`'s verdict is
+still live for the boot re-verification, `DriverPlugin` escape-hatch drivers,
+and the cloud compositions until they converge onto this seam.

content/docs/data-modeling/drivers.mdx

@@ -21,7 +21,11 @@ Drivers can be selected in two ways:
    override the inference (e.g. force `sqlite` for an ambiguous path).
 
 2. **Programmatic** — register a driver instance as a kernel plugin via
-   `DriverPlugin` (the same path the CLI uses internally):
+   `DriverPlugin`. This is the escape hatch for pre-built or auxiliary
+   drivers — the CLI itself no longer boots the primary this way: it declares
+   the `default` datasource as a definition and connects it through the shared
+   datasource path ([#3826](https://github.com/objectstack-ai/objectstack/issues/3826),
+   described further down this page):
    ```typescript
    import { DriverPlugin } from '@objectstack/runtime';
    import { SqlDriver } from '@objectstack/driver-sql';

docs/adr/0062-external-datasource-runtime.md

@@ -1,6 +1,6 @@
 # ADR-0062: External Datasource Runtime — connection lifecycle, credentials, visibility & query completeness
 
-**Status**: Accepted (2026-06-22) — D2–D8 implemented (`service-datasource` connection service + opt-in-safe gate + fail-closed connect policy; native-SQL declines external per D6; D7 lint in `validate-expressions.ts`). **D1 partially implemented**: declared datasources auto-connect through the one service, but the `default` driver still has its own connect + failure path — see the status correction under D1 (#3826).
+**Status**: Accepted (2026-06-22) — D2–D8 implemented (`service-datasource` connection service + opt-in-safe gate + fail-closed connect policy; native-SQL declines external per D6; D7 lint in `validate-expressions.ts`). **D1 implemented for every open-core boot path** (#3826: the standalone/artifact `default` and the CLI serve fallback are declared definitions connected through the one service — see the status notes under D1); the cloud repo's own stack composition is the last remaining second site, and the parity guard stays until it converges.
 
 **Supersedes the runtime portions of**: ADR-0015 §18 addendum (kept as the historical record). ADR-0015 remains the canonical spec/binding decision; this ADR is the canonical *runtime* decision.
 
@@ -68,7 +68,9 @@ Introduce a single service that, given a datasource definition, builds a driver
 >
 > **Config-load fallback converged too (#3826, third pass).** `createStorageDriver` is gone: the CLI's serve fallback (a host `objectstack.config.ts` with objects but no driver plugin) now emits a definition via `resolveStorageDefinition` and hands it to the same `DefaultDatasourcePlugin`. `mysql` joined the shared factory for it; the dev loosen-only self-heal (#2186) rides as `config.autoMigrate` and the CLI's wasm persistence mode as `config.persist` — host-composition passthroughs the factory honours, never part of the app-facing datasource spec. `turso`/libSQL keeps its loud typed failure at *resolution* (nothing is constructed to fail later). The `telemetry` sibling datasource deliberately stays a pre-built `DriverPlugin` (the documented escape hatch for named auxiliary drivers): it is best-effort, dev-oriented, and its own `resolveSqliteDriver` step-down check replaces the old primary-resolution coupling.
 >
-> **Remaining second site: the cloud stack's own composition** (cloud repo, `cloud-stack.ts`), which also owns the `turso` driver. Until it converges, `packages/runtime/src/degraded-boot-parity.test.ts` remains load-bearing: it pins both connect paths to the same operator-visible contract (fail-fast by default, identical `OS_ALLOW_DRIVER_CONNECT_FAILURE` parsing, `DEGRADED BOOT` on stderr). #3741 → #3758 was exactly the miss it exists to catch.
+> **Remaining second site: the cloud repo's own compositions** — the real primary-driver sites are `environment-kernel-factory.ts` (each environment's per-tenant driver) and the control-plane preset fed by `cloud-stack.ts`'s `buildControlDriver`, which also owns the `turso` driver. Until they converge, `packages/runtime/src/degraded-boot-parity.test.ts` remains load-bearing: it pins both connect paths to the same operator-visible contract (fail-fast by default, identical `OS_ALLOW_DRIVER_CONNECT_FAILURE` parsing, `DEGRADED BOOT` on stderr). #3741 → #3758 was exactly the miss it exists to catch.
+>
+> **The convergence seam (#3826, fourth pass).** Two properties of the cloud composition made "just declare it" impossible: its driver kinds live outside open-core (`turso`), and its instances are *pooled beyond one kernel* (the control-plane driver doubles as the proxy base of every environment kernel; per-environment drivers are cached across kernel rebuilds — reconstruction per boot would multiply pools). So `DefaultDatasourcePlugin` now accepts an **injected `IDatasourceDriverFactory`** (defaulting to the shared open-core factory), and `createPrebuiltDriverFactory` wraps an already-built instance as a factory — the "adopt an existing driver" entry point this ADR's first pass found missing, landed *as a factory* so the connect orchestration (policy-free init connect, `bootCritical` verdict, shared escape hatch, start() replay into retained state) stays this one implementation. Construction and pooling remain host concerns; only the verdict converges. The `@objectstack/verify` dogfood harness also boots through the declared default now (not the `DriverPlugin` escape hatch), making the §Risk mitigation — "behind the dogfood gate" — actually true for the converged path.
 
 ### D2 — Connect is opt-in-safe: existing managed apps are byte-for-byte unchanged
 

packages/runtime/src/default-datasource-plugin.test.ts

@@ -19,13 +19,18 @@ async function assemble(opts: {
   withAdminPlugin?: boolean;
   connectPolicy?: any;
   bundle?: any;
+  /** Host-injected driver factory (the cloud/EE seam) — see the last cases. */
+  factory?: any;
 } = {}) {
   const { ObjectQLPlugin } = await import('@objectstack/objectql');
   const runtime = new Runtime({ cluster: false });
   const kernel = runtime.getKernel();
   // Order matters for START: the default datasource must connect before
   // ObjectQLPlugin.start() runs boot schema-sync.
-  await kernel.use(new DefaultDatasourcePlugin({ driver: opts.driver ?? 'memory' }));
+  await kernel.use(new DefaultDatasourcePlugin(
+    { driver: opts.driver ?? 'memory' },
+    opts.factory ? { factory: opts.factory } : {},
+  ));
   await kernel.use(new ObjectQLPlugin());
   if (opts.bundle) await kernel.use(new AppPlugin(opts.bundle));
   if (opts.withAdminPlugin !== false) {
@@ -135,6 +140,60 @@ describe('DefaultDatasourcePlugin — the default datasource as a declaration (#
     }
   }, BOOT_TIMEOUT);
 
+  it('adopts a HOST-BUILT driver instance through an injected factory — same path, host pooling (#3826 seam)', async () => {
+    // The cloud composition's shape: the host already holds a constructed
+    // (possibly pooled/shared) driver instance the open-core factory cannot
+    // rebuild. createPrebuiltDriverFactory wraps it; the plugin's connect
+    // orchestration must register THAT instance — never a rebuilt copy.
+    const { createPrebuiltDriverFactory } = await import('@objectstack/service-datasource');
+    const { InMemoryDriver } = await import('@objectstack/driver-memory');
+    const hostBuilt = new InMemoryDriver();
+    const kernel = await assemble({
+      driver: 'turso', // a kind the SHARED factory does not support — proves dispatch
+      factory: createPrebuiltDriverFactory(hostBuilt, { driverId: 'turso' }),
+      bundle: {
+        manifest: { id: 'com.test.adopted-ds', name: 'Adopted DS', version: '1.0.0' },
+        objects: [{ name: 'note', label: 'Note', fields: { title: { type: 'text' } } }],
+      },
+    });
+    try {
+      await kernel.bootstrap();
+      const engine = kernel.getService<any>('data');
+      const defaultName = engine.getDefaultDriverName();
+      expect(defaultName).toBeDefined();
+      // Identity, not equivalence: the engine's default driver IS the host's instance.
+      expect(engine.getDriverByName(defaultName)).toBe(hostBuilt);
+      await engine.insert('note', { title: 'through-the-adopted-default' });
+      const rows = await engine.find('note');
+      expect(rows.map((r: any) => r.title)).toContain('through-the-adopted-default');
+    } finally {
+      try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+    }
+  }, BOOT_TIMEOUT);
+
+  it('an injected factory rides the SAME failure verdict — fail-fast, same escape hatch', async () => {
+    // The seam must never fork the verdict: an adopted instance that cannot
+    // connect takes the identical bootCritical fail-fast (and the identical
+    // OS_ALLOW_DRIVER_CONNECT_FAILURE override) as a factory-built default.
+    const { createPrebuiltDriverFactory } = await import('@objectstack/service-datasource');
+    const failing = {
+      name: 'turso',
+      async connect() { throw new Error('connect ECONNREFUSED 127.0.0.1:8080'); },
+      async disconnect() {},
+    };
+    const kernel = await assemble({
+      driver: 'turso',
+      factory: createPrebuiltDriverFactory(failing, { driverId: 'turso' }),
+    });
+    const err = await kernel.bootstrap().then(
+      () => { throw new Error('bootstrap() resolved but should have thrown'); },
+      (e: unknown) => e as Error,
+    );
+    expect(err.message).toMatch(/boot-critical/);
+    expect(err.message).toContain('OS_ALLOW_DRIVER_CONNECT_FAILURE');
+    try { await (kernel as any)?.stop?.(); } catch { /* noop */ }
+  }, BOOT_TIMEOUT);
+
   it("rejects an app bundle that declares a datasource named 'default' (host-reserved name)", async () => {
     const kernel = await assemble({
       bundle: {

packages/runtime/src/default-datasource-plugin.ts

@@ -5,6 +5,7 @@ import {
   DatasourceConnectionService,
   createDefaultDatasourceDriverFactory,
   type ConnectableDatasource,
+  type IDatasourceDriverFactory,
 } from '@objectstack/service-datasource';
 
 /**
@@ -52,12 +53,37 @@ import {
  * drivers — it is no longer how the standalone default boots.
  */
 export interface DefaultDatasourceDefinition {
-  /** Driver id the shared factory can build (`sqlite`, `sqlite-wasm`, `postgres`, `mongodb`, `memory`). */
+  /** Driver id the (injected or shared) factory can build (`sqlite`, `sqlite-wasm`, `postgres`, `mongodb`, `memory`). */
   driver: string;
   config?: Record<string, unknown>;
   label?: string;
 }
 
+export interface DefaultDatasourcePluginOptions {
+  /** Arms the shared factory's dev sqlite step-down (#2229) + loosen-only self-heal passthroughs. */
+  dev?: boolean;
+  /**
+   * Host-injected driver factory. Defaults to the shared open-core factory
+   * (`createDefaultDatasourceDriverFactory`). The seam exists for hosts whose
+   * `default` needs a driver the open-core factory cannot build — the cloud
+   * distribution's `turso`, or an already-pooled instance adopted via
+   * `createPrebuiltDriverFactory` — WITHOUT forking the connect + failure
+   * -verdict orchestration this plugin owns (the #3741 → #3758 drift). The
+   * factory only changes what `create()` returns; the policy-free init
+   * connect, `bootCritical` verdict, escape hatch, and start() replay are
+   * identical either way.
+   *
+   * Note the start() replay goes through the SHARED `'datasource-connection'`
+   * service, whose factory is the host's admin-plugin one — after a NORMAL
+   * boot that replay resolves `already-registered` before any factory is
+   * consulted, but a degraded boot's real retry (init failed under
+   * `OS_ALLOW_DRIVER_CONNECT_FAILURE`) retries with the shared factory. A
+   * host injecting a factory for a kind the shared one cannot build should
+   * expect that retry to report `skipped-unsupported` rather than reattempt.
+   */
+  factory?: IDatasourceDriverFactory;
+}
+
 export class DefaultDatasourcePlugin implements Plugin {
   name = 'com.objectstack.runtime.default-datasource';
   version = '1.0.0';
@@ -71,10 +97,12 @@ export class DefaultDatasourcePlugin implements Plugin {
 
   private readonly def: DefaultDatasourceDefinition;
   private readonly dev?: boolean;
+  private readonly factory?: IDatasourceDriverFactory;
 
-  constructor(def: DefaultDatasourceDefinition, opts: { dev?: boolean } = {}) {
+  constructor(def: DefaultDatasourceDefinition, opts: DefaultDatasourcePluginOptions = {}) {
     this.def = def;
     this.dev = opts.dev;
+    this.factory = opts.factory;
   }
 
   private record(): ConnectableDatasource {
@@ -90,7 +118,7 @@ export class DefaultDatasourcePlugin implements Plugin {
 
   init = async (ctx: PluginContext) => {
     const connection = new DatasourceConnectionService({
-      factory: () => createDefaultDatasourceDriverFactory({ dev: this.dev }),
+      factory: () => this.factory ?? createDefaultDatasourceDriverFactory({ dev: this.dev }),
       engine: () => {
         try {
           return ctx.getService('data');

packages/runtime/src/degraded-boot-parity.test.ts

@@ -3,20 +3,28 @@
 // framework#3826 — DRIFT GUARD for the two "a datasource would not connect"
 // implementations.
 //
-// ADR-0062 D1 asks for exactly one "definition → live driver" path. The
-// *construction* half converged (both build through the shared datasource driver
-// factory), but the *connect + failure verdict* half did not:
+// ADR-0062 D1 asks for exactly one "definition → live driver" path, and for
+// every OPEN-CORE boot that is now true: the `default` datasource is a declared
+// definition connected by `DefaultDatasourcePlugin` through
+// `DatasourceConnectionService.connect()` (#3869 standalone/artifact, #3886 the
+// CLI serve fallback) — `handleFailure` (#3758) is its one failure verdict.
+// But the ENGINE-side verdict — `ObjectQLEngine.init()` → `DriverConnectError`
+// (#3741) — is still live code, on purpose, for three callers:
 //
-//   - the `default` driver is registered as a `driver.*` kernel service and
-//     connected by `ObjectQLEngine.init()`     → DriverConnectError (#3741)
-//   - declared datasources are connected by
-//     `DatasourceConnectionService.connect()`  → handleFailure  (#3758)
+//   - the boot re-verification of the already-connected default (the #3741
+//     fail-fast role D1 deliberately keeps in `init()`),
+//   - drivers registered via the pre-built `DriverPlugin` escape hatch
+//     (the telemetry sibling, tests, dynamic/proxy drivers), and
+//   - the cloud repo's own compositions (`environment-kernel-factory`,
+//     control-plane preset), until they converge onto the shared path.
 //
-// They agree today only because two separate pieces of code were each written
-// correctly — and the last time they disagreed, the gap survived three months
-// and a second bug report (#3741 fixed one layer, #3758 was the other). Until
-// the paths are actually merged, this test is what notices a divergence:
-// it pins the operator-visible contract both of them owe.
+// So two verdict implementations still coexist, and they agree only because
+// each is written correctly — the last time they disagreed, the gap survived
+// three months and a second bug report (#3741 fixed one layer, #3758 was the
+// other). This test pins the operator-visible contract both of them owe:
+// fail-fast by default, one identically-parsed escape hatch, DEGRADED BOOT on
+// stderr. It outlives the cloud convergence: as long as `init()` can throw a
+// connect verdict at all, that verdict must match the service's.
 //
 // This lives in `runtime` because it is the only package that depends on both.
 

packages/runtime/src/index.ts

@@ -18,7 +18,7 @@ export type { DefaultHostConfigOptions, DefaultHostConfigResult } from './defaul
 // Export Plugins
 export { DriverPlugin } from './driver-plugin.js';
 export { DefaultDatasourcePlugin } from './default-datasource-plugin.js';
-export type { DefaultDatasourceDefinition } from './default-datasource-plugin.js';
+export type { DefaultDatasourceDefinition, DefaultDatasourcePluginOptions } from './default-datasource-plugin.js';
 export { AppPlugin, collectBundleHooks, collectBundleFunctions, collectBundleActions } from './app-plugin.js';
 export { SeedLoaderService } from './seed-loader.js';
 // Boot-summary seed outcome accumulator (#3415/#3430) — the single writer