Repository navigation
Add the AI SDK with coroutine-safe execution and framework integrations - #670
Merged
Merged
Conversation
Escape quotes and backslashes in metadata string literals and map whereNotIn to a parenthesized chain of inequalities. Normalize shorthand filter keys to strings at FileSearch so numeric PHP array keys retain the declared provider-independent shape. Describe the four query-builder operators precisely without adding runtime guards. Preserve numeric value formatting, model defaults, media validation and upstream tool mapping behavior. Port the Gemini tool mapping tests and their shared fixtures. Regression tests fail on unescaped literals and missing exclusion handling, and numeric shorthand fails without its boundary cast. They pass with the fixes. Upstream request mapping cases remain tracked until GeminiGateway is available; no tests are skipped.
Apply the repository formatter to the two empty foreach loops used to consume failing conversation streams. The loops still advance the complete stream and leave exception assertions unchanged. The available AI suite includes this file; execution remains tracked against the pending Anthropic and Groq gateways.
Preserve the upstream schema transformation for Anthropic native structured output, including recursive definitions, union types, supported formats and constraint descriptions. Add native constant and closure types without changing the schema contract. Port every upstream sanitizer test to PHPUnit, covering nested schemas, nullable properties and unsupported constraints. The focused sanitizer suite passes; source analysis has no diagnostics for this implementation.
Bring the Anthropic request builders, attachment and history mappers, tool definitions, response parsing and streamed event handling into the AI package. Retain the existing shared Http transport and upstream protected provider signatures. Describe the built-in provider types and validated tool-choice modes for static analysis without adding runtime checks. Encode list-shaped top-level tool arguments as JSON objects so numeric argument names remain properties. Preserve associative mapper output and the upstream nested argument representation. Keep replay blocks, thinking signatures, provider tool events, prompt caching and structured output behavior intact. Port the corresponding upstream feature tests, tool-search mapping tests and fixtures to PHPUnit. The numeric-argument regression fails with the original mapping and passes with the correction. Queue tests cover encrypted provider objects passed singly or in a failover list. Formatting and available behavioral checks pass. Source analysis and provider-tool integration retain explicitly tracked missing dependencies from the incomplete package port; no temporary skips, analysis suppressions or substitute classes are introduced.
Implement retrieval, multipart uploads and deletion through the Anthropic Files API, preserving file metadata, provider options and authentication headers. Select the existing bounded AI Http connection for each fresh request. Keep credentials and request headers on the operation while allowing transport reuse, and preserve the upstream protected client extension point. Port and run all upstream file gateway cases, including multipart provider options. Formatting and source analysis for the gateway pass.
…lures Add the upstream approval fixtures and Anthropic guard tests for conversational, client-replayed and non-resumable agents. Preserve coverage of ungated side effects before an approval pause. Adapt the AG-UI persistence rejection fixture to recordApprovalResult, the claim-owned write boundary used by resumed tools. It still rejects after execution and before a tool-result event; the original RUN_ERROR sequence, approval_mismatch payload and exception-reporting assertions are unchanged. The real Anthropic rejection path now passes. Remaining cases requiring the unported OpenAI gateway, Vercel adapter or fake conversation store remain visible and tracked without skips.
Encode empty and list-shaped argument arrays as objects in tool-input-available events. PHP converts numeric property names to integer keys, so serializing those arrays directly incorrectly produces JSON lists. Keep ordinary associative input and the public tool argument API unchanged. The regression inspects serialized stream frames without associative JSON decoding, preserving the distinction between objects and lists. It fails with the original expression and passes with the corrected mapping. Nested object preservation remains unchanged; this fix only restores the top-level tool-input object shape.
Record that nested JSON objects may lose their type during associative decoding, including empty objects becoming arrays. This can affect schema validation in MCP and Vercel integrations. Keep the upstream array API and request handling for now, and consider upstream reports and fixes during normal synchronization. Record the decision in the active port plan and a brief README Known Issues entry without creating a separate follow-up task. Distinguish that retained limitation from the top-level object-encoding corrections, which preserve the current API.
Add caller-supplied middleware to the AI port plan without changing existing agent contracts or declared middleware behavior. Specify invocation snapshots, serializable closure ownership through prompts and step options, and the empty-list fast path. Include focused coverage for lazy execution, middleware ordering and queued payloads, plus updates to the canonical AI documentation.
Copy the upstream step-capture gateway and in-memory conversation store into the AI test fixtures, adapting namespaces and signatures to the framework contracts. These fixtures support the upstream middleware coverage, including step input changes, provider continuation tokens, and conversation identifiers on completed streams. They also provide the missing conversation-store fixture for protocol tests. Formatting passes. The AI suite exercises the available paths; cases requiring provider gateways still being ported remain visible in the remaining-file checklist.
Add Promptable::withMiddleware() to append caller-supplied middleware outside the agent-declared pipeline. Capture list membership with each prompt or stream so later configuration cannot change an existing lazy invocation, including its failover attempts. Carry serializable closures through prompt revisions, immutable generation options, queued agents and queued events. Unwrap once per run through the existing step executor. Preserve existing contracts and factory signatures, with no extra options copy for calls without runtime middleware. Port the complete upstream middleware test file and add focused coverage for ordering, instance persistence, lazy capture, queued execution, direct options and event serialization. Formatting and type fixtures pass. Source analysis and the AI suite retain only explicitly tracked missing dependencies from the incomplete package port.
Extend the existing middleware documentation with the fluent API, ordering relative to declared middleware, instance lifetime and lazy stream capture. Explain queue serialization requirements and the boundary between model steps and tool execution. Keep the existing upstream explanations and examples, updating namespaces within the touched section. This additive extension requires no Laravel migration guidance.
Use the upstream JSON representation for individual vectors and complete responses. Remove the packed float64 codec and leave compression to the configured cache connection, avoiding an unproven format optimization and integer-to-float conversion on cache hits. Retain scoped provider identities, precomputed keys, batch cache operations, and response metadata restoration. Simplify cache coverage to ordinary vector values, metadata, usage, and account isolation rather than byte-level floating-point representation. Verified both cache modes with the contract-provider tests. Remaining Cohere integration cases await the gateway port.
An HTTP response can end successfully without the provider finishing its message. Require the existing message_delta stop reason before constructing a completed step, and report premature EOF through the existing stream error path. Add a regression that fails with the former completion behavior. Correct the tool-loop fixture to return SSE for its second streamed request and verify the final answer. Give prompt-cache tests their own HTTP fakes so an earlier catch-all response cannot hide the streaming or structured-output fixture. The affected Anthropic test files pass, and the broader AI suite has only the remaining unported dependency failures.
Port request construction, message and attachment mapping, provider tools, text steps, streaming, audio, images, transcription, and embeddings from Laravel AI 1.x. Reuse the existing named HTTP client and provider ownership boundaries, preserving upstream extension signatures with native types. Join all response text parts instead of retaining only the final message. Preserve flat and nested provider errors, handle failed and incomplete terminal events, and prioritize response-level interruption over completed tool items. Reject premature stream EOF through the existing error path. Encode top-level tool argument maps as JSON objects. Port the core upstream request, streaming, error, audio, embedding, and fake tests and their fixtures. Regression cases fail with the original parsing behavior and pass with the corrections. Tests configure credentials after provider registration so shipped defaults remain under test. Formatting and type fixtures pass. Source analysis and the broader AI suite retain only explicitly tracked failures from unported classes; remaining provider test files stay on the port checklist.
Implement the decisions endpoint for boolean, choice, and score questions, including inline image attachments and provider options. Preserve upstream wire formats and validation while applying native types and the shared provider HTTP client. Copy the upstream classification tests and image fixture, preserving attachment variants, answer mapping, option precedence, and unsupported input behavior. All classification cases pass with the shipped provider configuration.
Add file upload, retrieval and deletion plus vector-store creation, expiration, retrieval and file membership operations. Use the existing request clients, provider-option resolution, file preparation and immutable date facade. Encode nonempty file attributes as JSON objects so numeric metadata keys retain the API-required shape. Preserve store and file provider identity when constructing response objects. Port upstream file, store and custom-base-URL tests. Add a numeric metadata regression against serialized request JSON; it fails without the object cast. All affected test files pass.
Specify upstream JSON cache values and delegate compression to the configured cache connection. Remove packed-format benchmarks and codec-specific acceptance criteria while retaining provider and account isolation requirements. Record the required audit boundary after the current checkpoint: independently examine completed AI-port additions for justified complexity and maintenance, reconcile concrete findings, and report recommendations before resuming implementation. Preserve worthwhile correctness and scalability mechanisms rather than optimizing for the smallest codebase.
Replace the completed audit instructions with the agreed remaining work: remove the manager's operation-started registration restriction while retaining boot-only configuration, duplicate-registration checks and invalid-result handling. Keep bounded caching of fixed skill directories. Require measurements before adding broadcast batching, accounting for delivery latency and scheduling costs, and make its configuration and verification requirements conditional on that decision. Document the conversation store's write-policy extension with an authorization example. Record the aggregation helper coupling for the source comment and remove transport trailer coverage from the AI integration test plan. No runtime code changes are included.
Remove the operation-started flag and its writes from provider, partition, embedding-scope and context lookups. Resolving a value during bootstrap no longer prevents another hook from being registered later in bootstrap. Retain boot-only registration, per-hook duplicate checks and validation of resolved values. Resolver-backed providers already bypass the fixed instance cache, and the other hooks resolve on use, so no cache invalidation mechanism is needed. Remove the test for the rejected lockout and make duplicate-registration errors describe the remaining checks.
Identify the four public combine helpers mirrored by the single-pass response aggregation. Changes to those helpers must also be reflected in this pass, since their source diffs do not appear in the response class. Preserve the existing aggregation behavior and performance.
Add the Azure v1 text, image and embedding gateway using the existing OpenAI Responses mapping and framework HTTP client. Preserve deployment selection, API-key authentication, tool schemas, stateful continuations and upstream extension signatures. Port the complete upstream generation test coverage to PHPUnit, including response events, finish reasons, provider options, attachments, tool loops, image usage and failures. Keep shared streaming fixtures under Fixtures and configure test providers through Testbench. Formatting and type fixtures pass. Azure and OpenAI provider tests pass except the tracked provider-tool cases awaiting the AI MCP adapter. Source analysis has only the remaining unported dependency diagnostics.
Reuse the OpenAI file and store implementations with Azure v1 routing and API-key authentication. Preserve the assistants upload purpose and resolve upload options with the Azure provider identity. Port upstream tests for multipart uploads, provider-specific purpose selection, vector store retrieval and creation, and Azure file-search filter behavior. Both test files pass; formatting and source analysis introduce no new diagnostics.
Azure gateways use the v1 endpoint and do not send an api-version query parameter. Remove the unused setting from shipped configuration and additionalConfiguration so it cannot suggest a request control that does nothing or unnecessarily alter embedding cache identities. Retain the existing tests that verify version-free text and embedding URLs. The focused Azure suite has no new failures; its outstanding errors remain the unported AI MCP adapter.
Present measured broadcasting costs, differences and the agreed recommendation in a concise chat summary, then notify the owner and await their decision before implementing batching. Keep the existing requirements for evaluating buffering, scheduling, latency and maintenance costs.
Give each provider a credential holder shared by its gateways and header clones. Retain completed credentials instead of pending SDK promises, so overlapping coroutines cannot drive the same promise or inherit another operation’s refresh client. Serialize refresh through the coroutine locker, release idle database leases before waiting, and clear stale credentials before a refresh. Preserve short-lived results for existing waiters and release ownership after failures or cancellation. Cover concurrent refresh and waiter recovery with focused tests.
Adapt signed PSR requests to fresh framework pending requests while reusing registered HTTP connections. Return settled promises to the AWS SDK so network operations stay within the calling coroutine and AWS retains responsibility for retries. Preserve request bodies, protocol options, retry delays, response-bearing failures and cancellation. Deliver AWS transfer statistics alongside the effective user callback without replacing the framework collector or changing request-option precedence. Tests cover error parsing, retries, transport failures and statistics.
Preserve upstream authentication precedence and protected client helpers while replacing the assume-role client registry with provider-owned completed credentials. Build clients per operation with current headers and timeouts; route Bedrock through its provider preset and explicit STS requests through the shared AI preset. Keep the AWS default credential chain and stock assume-role loader. Translate both direct Bedrock and wrapped STS connection failures into provider failover errors while preserving cancellation. Port upstream tests and verify real STS-to-Bedrock signing through the HTTP adapter. Retain the documented skipped regression for AWS SDK issue 3370 without a production workaround.
Explain credential reuse for fixed and dynamically resolved providers, distinguish remote credential lookup costs from local static sources, and document the shared HTTP preset used by STS. Align the active port plan with the credential holder and synchronous HTTP adapter. Track the AWS assume-role cancellation defect, its regression test and the dependency update needed to remove the skip. Remaining gateway ports and integration measurements remain part of the AI port.
Normalize the bare-vector and typed-float response formats used by Cohere and its Bedrock models. Preserve the upstream rejection of unsupported typed responses and its behavior for absent embeddings. Port the corresponding upstream parser tests to PHPUnit, retaining all cases. The focused parser and Bedrock suites pass.
Buffer complete backtick runs across streaming chunks so multi-backtick text is emitted without losing characters. Track fenced code and preserve its contents literally, including fences indented within lists; only whitespace may follow a closing fence. Extend the existing every-split and bytewise formatting test with code containing headings, arithmetic, backticks, and an indented list example. Keep ordinary text streaming immediately without adding a general Markdown parser.
Align the documentation with provider URL settings, compatible-endpoint streaming usage and diarization, Bedrock document prompts, and deferred tool fallback behavior. Correct the provider enum spelling, optional Vercel message ID example, and chronological conversation ordering. Explain that sub-agent text becomes a tool result while parent usage counts its own steps, document abandoned-turn settlement, and use the required closures in image/audio negative fake assertions.
Member
Author
|
@cubic-dev-ai review |
@binaryfire I have started the AI code review. It will take a few minutes to complete. |
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
No issues found across 1217 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
View guided diff | Turn on auto-fix | Re-trigger cubic
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Use the existing tool-choice resolver when native structured output and tools are combined. Explicit none, required, and named choices now reach the request instead of being replaced with auto; the extended-thinking restriction applies consistently. Extend request-mapping coverage for each choice and the structured-output thinking rejection. Preserve synthetic-schema tool selection and provider-option overrides.
Map automatic, required, and named choices into Converse toolConfig. Reject none with a clear error because Converse cannot represent it, rather than silently allowing tool calls. Apply the mapping after tool configuration is built, preserving protected method signatures, synthetic-schema selection, and subsequent provider overrides. Extend parameter tests and cover the unsupported choice.
Fail before sending a chat request when mapped tools are present and an explicit choice other than auto is requested. Ollama cannot enforce those choices, so silently ignoring them could execute tools contrary to the request. Keep automatic and omitted choices unchanged. Place the check in the request builder shared by prompt and stream calls, and add a regression using the existing tool-choice fixture.
Re-yield events produced by buffered step middleware instead of delegating their generator keys. Default iterator_to_array collection now retains every step event and the terminal event rather than overwriting entries when keys restart. Add a two-step regression that checks the complete event sequence and both text deltas. Document why delegation is unsuitable at this boundary.
Cancel pending title work before persisting a recordable failed or abandoned turn. Reuse a title that has already completed; otherwise use the prompt prefix without starting or waiting for another provider request. Expose a non-waiting read on the operation-owned pending title and preserve successful-turn behavior. Cover serial and concurrent titles on both failure and abandonment while checking that completed tool results are still saved.
Track the opening backtick run length so shorter fenced examples inside a code block cannot close the outer block. Matching or longer closing runs retain the existing whitespace and indentation handling. Extend the chunk-boundary regression with a four-backtick block containing a three-backtick example, followed by formatted text. Verify the same output for every split and bytewise input.
Explain failed and abandoned first-turn title handling in the general titles section, including reuse of completed titles and cancellation of unfinished work. Clarify that stream cleanup does not launch or wait for title generation. Document the Bedrock none limitation and model-dependent forced-tool support alongside runtime tool choice, and note that Ollama supports automatic choice only.
Compare cache lifetimes only when the corresponding instruction or tool cache point is present in the request. Attributes for absent targets no longer reject valid requests, including requests using automatic caching. Restrict forced-tool rejection to manual enabled thinking. Explicitly disabled and adaptive thinking retain their supported tool choices, and disabled thinking no longer prevents forcing the synthetic output tool. Extend request and cache tests for missing targets and the distinct thinking modes while retaining the existing invalid-order and manual-thinking rejection coverage.
Run cache-order validation only when both system and tool cache targets are present in the final Converse parameters. Preserve the protected validator signature and provider-option overrides. Select the synthetic structured-output tool automatically when manual thinking is enabled, since forcing that tool produces an invalid provider request. Leave ordinary explicit tool choices and existing structured response parsing unchanged. Cover requests with only one cache target and verify that manual thinking retains both its settings and the synthetic tool definition without forcing a call.
Restore normal title generation for recordable provider and tool failures. Only cancellation or deliberate stream abandonment skips new title work, keeping a completed concurrent title or using the prompt prefix. Identify abandonment with a dedicated AI exception at the existing cleanup site and keep the stopping decision inside the new-conversation title branch. No method signatures or additional operation flags are needed. Narrow stopped-stream coverage to serial and concurrent abandonment plus recordable cancellation. The existing failed-turn title regression remains unchanged and passes with the restored behavior.
Distinguish cancellation and abandonment from ordinary errors when documenting conversation titles. Explain the permanent prompt-prefix fallback and how concurrent generation can preserve an already completed title. Describe cache lifetime ordering only for targets present in a request, including automatic caching. Document one-hour caching on supported Bedrock Claude models and replace the incomplete thinking example with a supported effort option.
Member
Author
|
@cubic-dev-ai review |
@binaryfire I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
No issues found across 1218 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
You've manually re-run cubic several times on this PR. Each manual re-review checks the full PR again and counts toward your usage quota. To preserve your usage limits, we recommend letting cubic automatically review new commits.
View guided diff | Turn on auto-fix | Re-trigger cubic
Synchronize conversation and stored-message IDs in finally blocks around completion callbacks, including callbacks registered after a stream has completed. A callback can save a turn before throwing, so its committed state must remain visible on the retained stream response. Keep iteration failures and public producer catch callbacks separate. Extend the existing callback regression to cover immediate and deferred callbacks while preserving exception identity and failure reporting.
Copy inner conversation state to the outer failover response after iterator cleanup, whether generation completes, fails, or is abandoned. This preserves IDs assigned by custom stores and also clears offered IDs when a turn is not remembered. Release the temporary outer-response reference before another attempt can suspend. Add a real provider-stream regression with a completed tool followed by failure, and extend ownership coverage to remembering agents using failover without a conversation participant.
Catch cancellation only while resolving the opening title, join unfinished title work, and save with a ready title or prompt prefix before rethrowing the same exception. Preserve the original generation outcome, completed tool results, pending approvals, and storage errors without retrying persistence. Resolve titles for completed streams in the completion callback so a late cancellation cannot turn a completed or paused response into a failed partial record. Apply stored IDs before rethrowing and use the captured conversation selection as the authority for whether the conversation exists. Remove redundant pending-ID mutation and restore the upstream void return on rememberFailedTurn. Update extension guidance and cover real coroutine cancellation, custom-store IDs, pending approvals, repeated iteration, and synchronous title cancellation. Existing failed-turn title expectations remain unchanged.
Use the existing JSON instruction path when manual extended thinking prevents forcing the synthetic output tool. Retain real tools and provider options, and decode final JSON text in both response parsers without depending on the model name. Synthetic structured output takes precedence when present. Reject required or named tool choices while manual thinking is enabled instead of sending an invalid Converse request. Replace the incomplete synthetic-auto expectation with request-and-response coverage, tool-loop coverage, streamed JSON decoding, and unsupported-choice cases.
Explain that cancellation preserves work already done without starting or waiting for further title generation, including cancellation that arrives during an existing title request. Document that Anthropic and Bedrock cannot force a tool while manual extended thinking is enabled, alongside the existing provider tool-choice limitations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds
hypervel/ai, porting Laravel's AI SDK to Hypervel with coroutine-safe execution, persistent conversations, and integrations with Scout, Sentry, and Telescope. Applications can use the familiar agent and provider APIs while running concurrent requests in long-lived workers.The package includes text and structured output, streaming, tools and approvals, conversation history, images, speech and transcription, embeddings, reranking, classification, file stores, skills, testing fakes, and queued generation. It also includes runtime middleware and tools, provider failover, and AG-UI and Vercel AI SDK chat inputs and stream protocols. Provider support covers OpenAI, Anthropic, Azure OpenAI, Bedrock, Gemini, Cohere, DeepSeek, ElevenLabs, Groq, Jina, Mistral, Ollama, OpenAI-compatible endpoints, OpenRouter, TypeSafe, VoyageAI, and xAI, according to each provider's capabilities.
Execution and provider ownership
Request state belongs to an operation rather than a shared worker. Provider instances and their credentials survive failover, deferred execution, and file-store operations without being replaced by a fresh lookup of the provider name. Repeated provider/model entries remain separate failover attempts.
Applications can resolve provider configuration for each operation, scope embedding caches, partition stored conversations, and capture application context for deferred work. These hooks support applications with different credentials and data boundaries for different customers without adding a tenant framework to the package. Runtime middleware can be attached to an agent instance without modifying its class; each invocation captures the instance's middleware.
Provider traffic uses Hypervel's HTTP client. Streaming stays incremental, including tool follow-up requests, and releases response bodies on completion, failure, cancellation, or abandonment. Bedrock keeps credential reuse and refresh coordination with the provider while creating SDK clients for individual operations. AWS signing and credential-chain behavior remain with the AWS SDK.
Creating fresh Bedrock clients has a measured cost. With inherited AWS shared configuration, client CPU per buffered step was 3.98 ms versus 0.60 ms in a shared-client experiment; streamed steps were 4.99 ms versus 1.47 ms. Without that configuration, reuse saved 0.8–0.9 ms. Fresh clients keep credentials and concurrent request promises owned by their operations without adding shared-client coordination.
Remote attachments use the HTTP client's shared destination checks and a configurable decoded-download limit, defaulting to 32 MiB. Embedded URL credentials are rejected. Explicitly trusted hosts can be listed in
ai.remote_files.allowed_hosts. If an outgoing proxy resolves destination hostnames, that proxy must filter the addresses it connects to.Provider parsing also fixes concrete failures: incomplete streams no longer look like successful responses, terminal errors reach callers, multiple text parts are retained, zero-valued content is preserved, and top-level tool arguments remain JSON objects. Other corrections cover reasoning replay, Gemini file-search filters, and image attachment capability checks.
Streaming CPU and memory
Stream response aggregation runs in one pass. Response callbacks no longer form reference cycles that retain completed streams until cyclic garbage collection. Replay retains the original events.
SSE and protocol responses enable disconnect cancellation to stop provider work on supported whole-connection closes. An individual HTTP/2 stream reset is detected only on a failed write, completion, or timeout with the current runtime.
The following measurements combine those AI changes with the receive-buffer copying fix already merged into
0.4in #669. They show the combined effect; they aren't an isolated measurement of this PR against current0.4.Heap is measured at the end of a run, not per request, and excludes native allocations. The large response includes the provider protocol's terminal complete-response payload.
Conversation storage and tool approvals
A conversation turn is stored atomically. The store writes the conversation timestamp once while still calling the public message-write methods, so subclass overrides continue to work. Parent-first row locking gives concurrent writes a consistent lock order. Conversation partition checks apply to reads, writes, and restoration; deleting a conversation cascades to its messages.
Stored approval resumes use an exclusive claim. Each completed tool result is recorded before proceeding, so a later failure doesn't discard earlier results. This costs additional commits on approval resumes: recording five results took 28.4 ms on PostgreSQL and 63.8 ms on MySQL in the storage fixture below. It does not promise exactly-once external side effects: a process can fail after an external action but before saving its result. An uncertain execution isn't silently retried.
Paused-message scans are paged, and replay cleanup reads flagged message IDs instead of decoding every paused message. The migration uses UUID identifiers, configurable participant key types, long-text payloads, approval claim and replay fields, and indexes matched to the store's queries.
This compares two Hypervel paths, not Hypervel against Laravel. The database timings were collected on a nearly full Docker volume and shouldn't be treated as general database latency predictions.
Caching and optional concurrency
Fixed skill directories are cached for the worker lifetime; other skill sources stay dynamic. Deployments reload services when cached skill files change. Embedding cache identities are computed once for a batch and distinguish provider configuration and application scope.
Tool execution and conversation-title generation can run concurrently through
#[ConcurrentTools(max: …)]andai.conversations.concurrent_title. Tools have bounded concurrency and retain result order. Title generation overlaps the response and is joined or cancelled with its owning operation. Enabling concurrent titles can incur a title request even when an attempt fails, including a separate request for each failover attempt. Child operations carry application context and the scoped database connection name without sharing the parent's transaction.These are local filesystem and controlled-wait measurements, not estimates of model response times. Tool and title concurrency remain opt-in.
Broadcast batching
Broadcasts deliver immediately by default; callers can still choose queued delivery with
now: false. Adjacent text and reasoning deltas are batched by default with a 50 ms window and a 4,096-byte limit on JSON-encoded delta content. The dedicatedai.broadcasting.batchingconfiguration can change these limits or disable batching.One bounded sender handles a stream. Structural events keep their order, original events remain available for replay, and SSE delivery is unchanged. During applicable transactions or event deferral, and outside coroutines, the caller sends events individually without batching. The window bounds deliberate batching delay, not total delivery time under backpressure.
The experiment below used sixteen streams producing 576 original events against a local Pusher-compatible HTTPS receiver. Connection retention was already enabled in both variants. The measured batching helper informed the production implementation; these are experiment results, not a new measurement of the final implementation.
Batching cuts sends/jobs by roughly 58–61% and client CPU by 52–58% here. The tradeoff is visible in the fast immediate case: individual deltas arrive later, although the last event is accepted at about 663 ms either way. In the slower immediate case, last-event acceptance falls from about 1,989 ms to 818 ms. Queued producers and workers shared a process in this fixture; these aren't forecasts for independently deployed workers.
Scout embeddings and vector search
Scout can generate embeddings through the SDK for Meilisearch, Typesense, and Turbopuffer, alongside their existing native and precomputed embedding modes. The database engine adds PostgreSQL/pgvector semantic and hybrid search, embedding updates, and pagination. Hybrid search loads up to 1,000 text and 1,000 semantic candidates and paginates the first 1,000 fused results. It preserves query customizations and model hydration; applications can select only the fields they need to reduce candidate memory use.
Database-vector imports limit child concurrency using the imported models' actual connection pool, reserving a slot for the parent chunk reader. A one-connection pool executes inline. Before this limit, concurrency 50 with a 10-connection pool failed a 5,000-row import after 4,500 updates. The fix has regression coverage but wasn't benchmarked for throughput. All concurrent Scout imports now preserve
DB::usingConnection()selection.Embedding writes retain per-row commits: a 100-row transaction experiment reduced generated import time from 22.65 to 18.65 seconds but increased overlapping writer p99 from 170 to 1,455 ms. Those figures share the database-storage caveat above. Models using the
Searchabletrait gainsearchableEmbeddingColumn()automatically; direct implementers ofSearchableInterfacemust add it.Sentry tracing and Telescope redaction
Sentry records invocation, model-step, tool, embedding, and classification spans. Model steps use already-resolved instructions and tools, and HTTP spans sit under the relevant step. Streaming spans follow consumption and close on success, failure, cancellation, or abandonment. Nested agents restore the outer agent's current span and conversation, and each failover attempt keeps its own spans.
The
gen_ai*tracing switches are enabled by default when the AI package is installed, subject to Sentry's tracing and sampling settings. With the defaultdata_collection: null, prompts, responses, tool payloads, and embedding/classification content are omitted unlesssend_default_piiis enabled. An explicitdata_collectionconfiguration instead controls them throughgen_ai.inputsandgen_ai.outputs, which default to true if omitted.The integration avoids work when tracing or individual span families are disabled. Enabled listeners check tracing state before formatting payloads, and AI lifecycle event construction is guarded by
hasListeners(). History formatting is lazy and bounded, attachment contents aren't loaded for telemetry, and bounded JSON remains valid rather than being cut mid-document. Bounded maps cap abandoned records; weak references keep those records from retaining their coroutine tracing scopes. These are architectural resource controls; no Sentry throughput improvement is claimed from the AI benchmarks.The AI events are also usable independently of Sentry:
StartingStepincludes resolved instructions and tools, streamed tools dispatchToolFailed, and newAgentStreamAbandoned,ClassificationFailed, andEmbeddingsFailedevents expose terminal paths to listeners.Telescope masks additional AI, AWS, OAuth, and search-provider credentials in headers and payloads. Query credentials are also hidden in recorded URLs, redirects, relevant headers, and session navigation URLs without rewriting unrelated query bytes.
Supporting framework fixes
saveOrIgnore()as well as ordinary inserts. Post-insert listeners can update a newly created model without being mistaken for a partition change.anyOfcompositions. Databasetake()andskip()accept null consistently withlimit()andoffset().The package includes an isolated
dogfood/aibrowser demo for contributors, package discovery/configuration wiring, and upstream-derived tests and fixtures. PostgreSQL CI services include pgvector for real vector-search coverage.At the root package level,
symfony/yamlbecomes a runtime dependency, and a Composer conflict excludesaws/aws-sdk-phpversions below 3.369.1 for allhypervel/componentsinstallations.The AI SDK documentation is ported alongside updates to the HTTP client, Scout, Sentry, Telescope, Permission, JSON Schema, and Laravel porting pages.
Compatibility and validation
The usual public generation APIs follow Laravel. Intentional differences are documented in the package README and porting guide, including provider/model pair returns, protected extension points, remote attachments, skill caching, broadcasting, and custom approval-store requirements. Existing Laravel conversation tables need the documented schema migration; they aren't interchangeable with the new schema.
Formatting, static analysis, targeted package tests, Testbench, and the demo tests passed. Coverage includes real loopback streaming, cancellation and cleanup, database locking and partition isolation, pgvector, queued execution, and Sentry span ownership. The final full-suite run stalled in a worker on Swoole 6.3.0RC1, which has a tracked shutdown hang; that run isn't reported as a completed pass. Live paid-provider tests and optional MCP integration weren't exercised locally.
Benchmarks use local mock providers and services, with no paid AI accounts. Repeated streaming and broadcast results are medians from alternating fresh-process comparisons on PHP 8.4.25/Swoole 6.3.0RC1 with CLI OPcache enabled and JIT disabled. Reproduction commands, full results, and measurement limits are in tests/Benchmarks/Ai. Client CPU excludes separate service processes. One MySQL process that hit the normal pool checkout deadline was excluded from the storage medians.
Known AWS SDK cancellation and event-stream decoder defects have focused skipped regressions with upstream issue links rather than local workarounds. Bedrock streams can fail with a checksum error or lose the final usage/error event; see the known issues. The upstream limitation around nested JSON object/list shapes in tool arguments is also documented. The included MCP adapters require the Hypervel MCP package, which isn't available yet.
Note
Add
hypervel/aiSDK with providers, agents, tools, and framework integrationsfake/assertAPIs) for agents, audio, embeddings, files, images, reranking, stores, transcription, and classification, plus a Vercel/AG-UI chat input adapter and streaming SSE/AG-UI response protocols.hypervel/aiandsymfony/yamlas root dependencies in composer.json and conflicts with AWS SDK >= 3.369.1;Env::encodeandparse_environment_variablesnow emit plaintrue/falseinstead of parenthesized strings, affecting anything parsing testbench env output; queuedfailed()handlers inCallQueuedListener,SendQueuedMailable,SendQueuedNotifications, andCallQueuedClosurenow accept a nullableThrowable; Testbench Workbench setup now runs before database traits viasetUpTheTestEnvironmentTraitToBeIgnored.Macroscope summarized 1544117.