Sync master with upstream release b10142 - #601
Merged
jan-service-account merged 35 commits intoJul 27, 2026
Merged
Conversation
…ggml-org#26049) * hex-l2: use dirty ranges for flushing * hex-l2: simplify range based flush logic * hex-l2: optimize dirty range scans * hex-hvx: support for reduce_max_i32 * hex-mm: optimize fused MUL_MAT+ADD to use vtcm for bias when it fits * hex-mmid: optimize mmid row-mapping generation * hex-mmid: optimize mmid row-mapping generation * hex-mmid: optimize mmid row-mapping generation (round2) * hmx-mm: optimize output proc by tiling (col-chunking) * hex-fa: start the next q dmas a bit earlier * hex-fa: prefetch Q even earlier * hvx-fa: optimize softmax to keep things in hvx registers * hex-fa: hoist const register init in softmax loop * hmx-fa: kick off next-qkv DMAs before o-proc * hmx-fa: hoist various checks out of the inner loop * hmx-fa: adjust the cost model to better balance softmax work across hvx threads * hmx-fa: overlap diag rescale build with last HMX task * hmx-fa: optimize idx update in output proc * hmx-fa: unroll the softmax loops for improved perf * hmx-fa: overlap qk-dot with softmax, double-buffer p and s tiles * hex-trace: double the default number of trace entries * hex-trace: add trace events for opbatch and buffer mgmt * hex-trace: overhaul tracing to simplify runtime event handling and support opbatch stats * hex-trace: replace ascii timeline diagram with pipeline bubbles detector * hex-trace: handle missing start/stop events * hex-dma: always log stop/start trace events even for dummy dmas * hex-scripts: fix flake warnings
…f a response schema is provided (ggml-org#26018)
…rs (Settings panel) (ggml-org#26002)
* skill: add new model * add common pitfalls * add code review skill * nits * add codeowners * add security review section * mention about skills in agents.md * add design review * exclude ggml-gh-bot * Apply suggestions from code review Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com> Co-authored-by: Xuan-Son Nguyen <thichthat@gmail.com> * conversion-time weight modification * nits --------- Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com>
* support "reasoning_effort": "none" in OAI API * handle reasoning.effort: "none" in OAI responses API * clarify non-"none" values of reasoning_effort have no effect * use json_value instead of body.at
) * ui: fix MCP server display name conflicts in tools lists Tool groups were keyed by display label so two servers reporting the same name broke the keyed each blocks and only one was visible. Key rendering, expand state and toggles by the stable server id instead, and suffix duplicate labels with a counter in config order. * ui: customizable MCP server display name with autofill Add a display name field to the MCP server form, add and edit alike. The custom name takes precedence over the server-reported one, so two servers reporting the same name can be told apart; clearing the field returns to the automatic label. In the add dialog a debounced preview handshake prefills the field with the server-reported name: a manual edit freezes the autofill, stale responses are discarded, failures stay silent, and an unedited prefill is not persisted so the label keeps following the server. * ui: fix recursive fetch passthrough in the client test setup The original fetch was captured inside beforeEach, where it is the previous test's spy since vi.spyOn returns the existing one, so the default passthrough recursed on itself for any URL outside the mocked set. Capture the real fetch once at module load.
* Start building graph - reuse deepseek32 * Enable kv cache and rotation for glm_dsa architecture Just follow Deepseek 3.2 for now. * Reuse prev_top_k for "shared" indexer layers * GLM 5.2 uses LLAMA_ROPE_TYPE_NORM for the indexer. This is transformers' `apply_rotary_pos_emb_interleave` * Default indexer types to GLM pattern Previous converted GGUFs like https://huggingface.co/unsloth/GLM-5.2-GGUF write indexer weights to _all_ layers, even if they are only required for "full" types. This PR relies on a new key "%s.attention.indexer.types"; if absent, it will use the default GLM 5.2 schedule as defined in https://huggingface.co/zai-org/GLM-5.2/blob/main/config.json#L26. Note that conversion is not saving this key yet. * Save indexer types to gguf, restore on load * Use ggml_lightning_indexer when cparams.fused_lid Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com> * GLM 5 and 5.1 use full indexers Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com> * Fix indentation * Ensure array is zero-filled * Prefer explicit std::fill * Assert prev_top_k exists for shared indexer --------- Co-authored-by: fairydreaming <166155368+fairydreaming@users.noreply.github.com>
* ui: remove viewport fade in and smooth autoscroll bottom snap fadeInView mounted every message and markdown block at opacity 0 and relied on an IntersectionObserver to reveal it. When the observer never fires (blocks mounted offscreen during long agentic loops) the content stays invisible forever while still present in the DOM. Remove the action, its orphaned isElementInViewport util and all call sites: blocks now render visible immediately. AutoScrollController.scrollToBottom defaulted to behavior smooth and is invoked every 100 ms while streaming. Each tick restarts an easing animation toward a moving scrollHeight, producing a random elastic bump of a few pixels when the user reaches the bottom and autoscroll reengages. Default to instant scrolling; the user facing scroll down button keeps its smooth behavior. * ui: skip rendering of offscreen chat messages via content-visibility Apply content-visibility auto with contain-intrinsic-size to chat messages so the browser skips layout and paint for messages outside the viewport. The DOM stays complete: component state, find-in-page, text selection, and the mutation based autoscroll are unaffected, and browsers without support simply ignore the properties. * ui: remove conversation switch fade Switching conversations faded the message list out and in over 500 ms plus a 300 ms route delay, deferring the message refresh behind two requestAnimationFrame calls. Remove the fade, its navigation hooks and dead state, and refresh messages directly so switching is only bound by actual render time. * ui: describe present behavior in comments and drop unused parameter * ui: anchor the context gauge popup to the form with plain CSS The stats card was portaled to body and repositioned in script on every ancestor scroll event, trailing the page by one frame while streaming. Render it as an absolutely positioned sibling of the input box inside the already relative form, so nothing runs during scroll. The card sits just above the dial, centered on it and overlapping the textarea, from a single measurement of the dial center and top taken when it opens; the dial and the card share the same positioning frame, so the values stay exact while the card is open. Mouse pointers open on hover with a grace delay to reach the card, touch pointers toggle on tap, any press outside the card and the dial closes it, and Enter and Space toggle from the keyboard. The card lives outside the input box because its overflow-hidden and backdrop-filter would clip any positioned descendant. * ui: extract context gauge popup constants and relocate its state store Move the placement values and the close grace delay to lib/constants/context-gauge-popup.ts, matching the auto-scroll constants layout, and move the popup state module from the component folder to lib/stores where runes modules live in this codebase. * ui: declare the context gauge popup card ref as $state
* performance harness - the empirical root Assisted-by: Claude Opus 4.8 * 210.36ms -> 2.67ms per streamed token Assisted-by: Claude Opus 4.8 * 11.58ms -> 0.62ms per streamed token Assisted-by: Claude Opus 4.8 * 22.02ms -> 3.33ms per streamed token Assisted-by: Claude Opus 4.8 * 3.07ms -> 1.36ms per streamed token at 40 messages Assisted-by: Claude Opus 4.8 --------- Co-authored-by: Zach Winter <dmtommy@icloud.com>
…et sampler (ggml-org#25544) * common : extract trie/ac to a separate file * common : support multiple token sequences in the reasoning budget sampler * common/trie : return matched word index * common/trie : rename "word" to "pattern" * common/reasoning-budget : expose matched end sequence * common/sampling : replay end sequence when reasoning budget is done * cont : update to use multiple end sequences * cont : clean up
…gguf_writer_base (ggml-org#25867) Without a virtual destructor, deleting a derived object through a base-class pointer only invokes the base destructor, skipping the derived one.
Signed-off-by: Adrien Gallouët <angt@huggingface.co>
… in generation_settings (ggml-org#25830) These two parameters were overlooked when task parameters were being JSONized within `generation_settings` and have been added. A regression test has been added to prevent the problem from recurring and it passes. Fixes ggml-org#25803
The INI parser creates an implicit default section for top-level metadata. After reserved keys such as `version` are skipped, that section can have no model options but was still added and exposed in router mode. Skip only the empty implicit default while preserving real default presets, named presets, and the global `[*]` settings. Signed-off-by: JS van Dijk <267467744+hogeheer499-commits@users.noreply.github.com> Co-authored-by: JS van Dijk <267467744+hogeheer499-commits@users.noreply.github.com>
* hexagon: add IM2COL op Add Hexagon IM2COL support targeting only patch-embedding convolutions. * hexagon: im2col refactor and cleanup * hex-im2col: instrument and update im2col. * hex-im2col: add local htp_vtcm_layout computation.
* move server_pipe to common * init impl * vendor: update subprocess.h * add server_mcp_stdio * stderr drain * server_mcp_transport * server_mcp_stdio is now framing-only, no json * internal/mcp-stdio: integration + tests + fixes (ggml-org#26075) * server-mcp: harden transport and wire up the tool integration Builds on the transport/manager architecture (server_mcp_transport + server_pipe) with the hardening and integration the draft did not yet have. Hardening: * Reader and stderr pumps are polled (running-aware) instead of blocking on a read that only ends at EOF. subprocess_terminate() SIGKILLs only the direct child, so a grandchild the MCP server spawned that inherited the pipe would otherwise keep the write end open and hang teardown (both warmup shutdown at startup and process shutdown). The writer is likewise non-blocking + polled. * Windows: resolve the command through PATHEXT so "npx" (npm ships npx.cmd, never npx.exe) spawns, matching POSIX's PATH search; and enumerate the parent environment as UTF-8 (GetEnvironmentStringsW) instead of the active code page. * server_pipe gains an opt-in max_size (default unbounded, so the router's streaming use is unchanged); the MCP reply queue uses it so a server that streams unsolicited notifications between requests cannot grow it without bound. Integration: * --mcp-servers-config / --mcp-servers-json flags; enabling MCP restricts default CORS to localhost, same as --tools. * MCP tools are exposed through /tools (and chat-completions) as <server>_<tool>, skipping names that collide with a built-in or another MCP tool. * Manager lifecycle wired into llama_server(): warmup at start, shutdown() from the signal handler before the HTTP server drains, blocking teardown in clean_up(). * SIGPIPE ignored so a child dying mid-write yields EPIPE rather than killing us. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com> * server-mcp: add MCP test suite with grandchild deadlock regression test 21 tests over the /tools endpoint: tool discovery/invocation, timeouts, crash recovery and respawn cooldown, warmup partial failure, malformed and batched notification+response output, tool-definition shape, and prompt shutdown during a slow call. The last test spawns an MCP server that leaves a grandchild inheriting its stdout/stderr and asserts the server both starts and stops promptly. Verified it fails (5s SIGKILL fallback on a deadlocked reader-join) when the pump is made to ignore the running flag, and passes with the polled reader. Assisted-By: Claude Opus 4.8 <noreply@anthropic.com> * clean up * clean up 2 * even stricter life cycle * nits * nits 2 --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co> * fix some edge cases * fix last_error data race * fix response schema + docs * server: fix MCP zombie leak and timeout-induced transport teardown join_pumps() never reaped the child, leaking one zombie per spawn: call subprocess_join() before subprocess_destroy(). A per-call timeout permanently closed from_server and got a healthy transport evicted: add close_on_stop to server_pipe::read() and pass false from send_rpc(), where should_stop is a per-request deadline and a late reply is already skipped on id mismatch. Also drop the unreachable disconnect cancellation in server_mcp_tool::invoke(): support_stream is false, st is always null. (cherry picked from commit e6de1ec) Assisted-by: Claude Opus 4.8 * server: make MCP test fixtures JSON-RPC 2.0 compliant Add the missing notification guard to mcp_malformed_server.py and mcp_burst_server.py (the latter treated id 0 as a notification and replied to unknown ones; its notification table is now unused). Return -32602 instead of -32601 for unknown tools: tools/call is a valid method, the tool name is the invalid parameter. Also fix the test module docstring: tools are named <server>_<tool>. (cherry picked from commit 74a08e8) Assisted-by: Claude Opus 4.8 --------- Co-authored-by: Piotr Wilkin (ilintar) <piotr.wilkin@syndatis.com> Co-authored-by: Pascal <admin@serveurperso.com>
* Fix emscripten compilation with openmp * Separate wasm job to its own workflow * Add flags necessary for newer emsdk * Just disable openmp * Update triggers
…nd (ggml-org#26099) The context gauge card starts monitoring like the dial does, because its own processing state instance only follows the live stream while its monitoring flag is set. It also gets back the text-sm and ring classes the removed hover card wrapper used to inject, which restores its layout. Routing to a conversation now lands at the bottom instantly and keeps the pin one frame at a time until the page height settles, since content-visibility size realizations and syntax highlight passes grow the page without DOM mutations.
…rg#24908) * Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style leading-dense and routed/shared experts, and swigluoai activation. Sparse attention is not yet supported (dense fallback); vision tower and MTP heads are dropped. * MiniMax-M3 vision tower (mmproj + clip graph) * Delete m3_vision_ref.py * Update clip.cpp * MSA * Update constants.py * Update minimax.py * Cache creation. Working withotu flash attention * Added flash attention for sparse layers * Decomposed slow cpu OP into GPU + CPU ops. Massive speedup over long ctx * Rewrote indexer op to be cuda native. Modified flash attention to match per group block picking * Implement sparse attention calc out of stock ops. * Fix a cache allocation and cont issue * Fixed -fa auto crash, flagged debug spots * Delete vocab.json * Delete model.safetensors.index.json * Delete generation_config.json * Delete Minimax directory * Handled multi stream case to fall back on Dense Attention * Development scaffolding cleanup. No functional change to the decode or 4-way paths. Full debug harness remains at <8136a9c68ed7a5eb009aa67bba3fda8062f4648f> for reproducing the selection-parity validation. * Remove redundant comment from minimax-m3.cpp * Changed 3 Gelu Ops for vision into Gelu_erf ops * Assert that n_kv is multiple of 128 * Rename MSA index tensors to indexer convention Note: All GGUFs generated before this change will need to be regenerated. * Fix incorrect Assert * Review driven changes (#3) * Remove comment from conversion minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove whitespaces from constants.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Tighten comment in minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * inherit MiniMax-M3 from MiniMax-M2 * drop dead text_config fallbacks * Add indexer writer methods * Reuse LLM_FFN_SWIGLU_OAI_MOE * Remove duplicate indexer setters, add only block_size/local_blocks, follow value naming convention * Fix conversion error /gguf_writer.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update gguf-py/gguf/gguf_writer.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update conversion/minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update conversion/minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove whitespace in src/llama-kv-cache.cpp Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove Whitespace in Update src/llama-model.h Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove whitespace in src/llama-hparams.h Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * remove multimodal code upon maintainer request. Will be made as a separate PR * Whitespace clean in tensor_mapping.py * Log cache size on launch, block ctx shift, support prompt caching Log indexer cache size on launch Disallow ctx shift Support prompt caching * Update minimax-m3.cpp * Optimize implementation, add multi stream support. Fully rewrote minimax-m3.cpp for speed and buffer size gains: Unified the 4-way + decode, 1 FA call per layer instead of 4, with the groups mapped onto ne[3] Custom CPU op now emits block-level mask, expanded on GPU, which causes CPU to GPU transfer to shrinks at prefill Decode: ~25 nodes/layer vs ~50, no per-group concats/conts Unified selection semantics, so both regimes rank bs + local bias (position-anchored local force), which means prefill/decode can no longer disagree on selection can_reuse on the MSA bias input. Graph reuse at decode restored (was rebuilding the full graph every token) In-place mask adds, shrinking compute buffer ~6.8 to ~4.2 GiB at ub2048/62k Multi-stream: MSA now runs with -np N when kv_unified=false. Decode stays batched across streams (still 1 FA call), prefill loops per stream. dense fallback only for --kv-unified + multi-seq Measured effect on expert offload bound setup: decode 6.2(4WAY)–7.15(MSA_decode) -> 7.7~7.8 t/s, flat from 5k to 60k+. prefill around 10% faster. buffer about 20% smaller, multi-user support. * set default cache type to F32 * Fix potential DSA double indexer cache allocation bug, only allocate in-cache k_idx for archs that opt in * remove F16 downcasts in MSA attention, force F32 indexer score accum * Add Minimax eos to llama vocab * Guard edge case where idx cache can become stale after a tail trim * Update llama-kv-cache.h * Update llama-kv-cache.cpp * Update llama-kv-cache.cpp * Update llama-kv-cache.h * Update llama-kv-cache.cpp * Review driven changes * style fix * indexer hparams are required * fix tests * fix lint --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
Co-authored-by: Eric Hartford <eric@quixi.ai>
…g#26121) * ui: detect the conversation import format from file contents iOS resolves every accept entry to a UTI and has none for ".jsonl", so the picker greyed out exported conversations. Drop the accept filter and pick the parser from the file contents: ZIP magic bytes, then a first "session" record for JSONL, otherwise the legacy JSON format. Also remove the unused importConversations() picker and an orphan doc comment, and cover each format with unit tests. * ui: report what a conversation import actually wrote The import summary echoed the selection back, so re-importing conversations already in the database claimed success while nothing was written and only a console warning said otherwise. Return the imported and skipped conversations from the database layer, list the written ones in the summary, and count the rest in a toast. * ui: name the literals of the JSONL conversation format Introduce SessionRecordType and SESSION_HARNESS, and reuse the existing NEWLINE constant, so the record format lives in one place. This also covers the writer side, which predates the import path under review and carried the same literals: an enum stated by the reader alone lets the two sides drift. Values are unchanged, so an export stays byte identical.
…g#26144) * Keep Minimax's indexer tensors at F32 for speed and accuracy * name -> new_name
…g#26006) The in-conversation system-message edit textarea had a fixed min-height and no auto-resize, so long messages were crammed to 2 lines. Reused existing autoResizeTextarea helper on input and when the editor opens, and added max-height to cap growth.
* Add preliminary MiniMax-M3 support Text-only port that re-uses existing components: MiniMax-M2 style GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style leading-dense and routed/shared experts, and swigluoai activation. Sparse attention is not yet supported (dense fallback); vision tower and MTP heads are dropped. * MiniMax-M3 vision tower (mmproj + clip graph) * Delete m3_vision_ref.py * Update clip.cpp * MSA * Update constants.py * Update minimax.py * Cache creation. Working withotu flash attention * Added flash attention for sparse layers * Decomposed slow cpu OP into GPU + CPU ops. Massive speedup over long ctx * Rewrote indexer op to be cuda native. Modified flash attention to match per group block picking * Implement sparse attention calc out of stock ops. * Fix a cache allocation and cont issue * Fixed -fa auto crash, flagged debug spots * Delete vocab.json * Delete model.safetensors.index.json * Delete generation_config.json * Delete Minimax directory * Handled multi stream case to fall back on Dense Attention * Development scaffolding cleanup. No functional change to the decode or 4-way paths. Full debug harness remains at <8136a9c68ed7a5eb009aa67bba3fda8062f4648f> for reproducing the selection-parity validation. * Remove redundant comment from minimax-m3.cpp * Changed 3 Gelu Ops for vision into Gelu_erf ops * Assert that n_kv is multiple of 128 * Rename MSA index tensors to indexer convention Note: All GGUFs generated before this change will need to be regenerated. * Fix incorrect Assert * Review driven changes (#3) * Remove comment from conversion minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove whitespaces from constants.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Tighten comment in minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * inherit MiniMax-M3 from MiniMax-M2 * drop dead text_config fallbacks * Add indexer writer methods * Reuse LLM_FFN_SWIGLU_OAI_MOE * Remove duplicate indexer setters, add only block_size/local_blocks, follow value naming convention * Fix conversion error /gguf_writer.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update gguf-py/gguf/gguf_writer.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update conversion/minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update conversion/minimax.py Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove whitespace in src/llama-kv-cache.cpp Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove Whitespace in Update src/llama-model.h Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Remove whitespace in src/llama-hparams.h Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> * Update minimax_m3.cpp Rewrite code comment based on feedback and to better reflect the actual architecture, and reuse existing build_vit * Rename minimax_m3.cpp to minimax-m3.cpp * Update CMakeLists.txt * Remove debug code from clip.cpp * Update clip.cpp * Update comments in tools/mtmd/models/minimax-m3.cpp * Permute Q/K at conversion, drop precomputed sin/cos * Log cache size on launch, block ctx shift, support prompt caching Log indexer cache size on launch Disallow ctx shift Support prompt caching * Update minimax-m3.cpp * Optimize implementation, add multi stream support. Fully rewrote minimax-m3.cpp for speed and buffer size gains: Unified the 4-way + decode, 1 FA call per layer instead of 4, with the groups mapped onto ne[3] Custom CPU op now emits block-level mask, expanded on GPU, which causes CPU to GPU transfer to shrinks at prefill Decode: ~25 nodes/layer vs ~50, no per-group concats/conts Unified selection semantics, so both regimes rank bs + local bias (position-anchored local force), which means prefill/decode can no longer disagree on selection can_reuse on the MSA bias input. Graph reuse at decode restored (was rebuilding the full graph every token) In-place mask adds, shrinking compute buffer ~6.8 to ~4.2 GiB at ub2048/62k Multi-stream: MSA now runs with -np N when kv_unified=false. Decode stays batched across streams (still 1 FA call), prefill loops per stream. dense fallback only for --kv-unified + multi-seq Measured effect on expert offload bound setup: decode 6.2(4WAY)–7.15(MSA_decode) -> 7.7~7.8 t/s, flat from 5k to 60k+. prefill around 10% faster. buffer about 20% smaller, multi-user support. * set default cache type to F32 * Fix potential DSA double indexer cache allocation bug, only allocate in-cache k_idx for archs that opt in * remove F16 downcasts in MSA attention, force F32 indexer score accum * Add Minimax eos to llama vocab * Guard edge case where idx cache can become stale after a tail trim * Update llama-kv-cache.h * Update llama-kv-cache.cpp * Update llama-kv-cache.cpp * Update llama-kv-cache.h * Change resize Pad to none, resize alg to Bicubic Pillow * Review driven changes * Update llama-kv-cache.cpp * rm unrotated pos_t * fused rope w + pad * rename merge --> merger for consistency * add review skill for mtmd * graph should use hparams n_merge * fix lint --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com> Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates dev branch with latest release (b10142) from ggml-org/llama.cpp