Repository navigation
Sync master with upstream release b10326 - #614
Merged
jan-service-account merged 28 commits intoAug 8, 2026
Merged
Conversation
…tning_indexer (ggml-org#26646) - In MSL, declaring an array of matrix types like `threadgroup half4x4` causes a 'no matching constructor' compilation error because MSL matrix types do not have zero-argument default constructors and threadgroup variables cannot have initializers. - Fix this by declaring a POD `threadgroup half` array instead and casting to `threadgroup half4x4 *` for matrix indexing. Signed-off-by: JamePeng <jame_peng@sina.com>
1. Abort CI if build requirements are missing. 2. Add check to make sure Git LFS has been configured. 3. Add trailing newlines to log messages.
…#26515) * support fp16 to fp16/fp32 * support all missed data types in set_rows * refactor the code to support all data types
…DSV4_HC_PRE (ggml-org#26568) * support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PREwq * update ops.md * fix format issue
* tests: add SWIGLU perf cases perf mode had no GLU coverage. Adds SWIGLU at 17408 columns, 512 and 2048 tokens, f16 and f32, with the operands both fused and split. * sycl: consolidate fused-GLU kernels They differed only in which op_* they called, so take the op as an argument and share a common launcher. Their block sizes were all 256, so launch geometry is unchanged; SYCL_GELU_BLOCK_SIZE and SYCL_SILU_BLOCK_SIZE lose their last users so are dropped. * sycl: contiguous fast path for the fused GLU ops o0 == n and o1 == n collapse the de-interleave index math to the identity, so dispatch a flat kernel in that case. It fires for ggml_glu_split with packed operands; a fused [gate|up] tensor keeps the strided path. test-backend-ops perf -o SWIGLU on an Arc Pro B70: split +14% f16 and +4% f32, fused unchanged.
The NVFP4 quantization format stores a scaling factor for every group of 16 weights, packed into a single UE4M3 byte. The SYCL GPU code was converting these scale values using the E4M3 path, but that's *signed*, and these are unsigned values.
* ui: read model modalities from the router model list The router advertises input modalities for every model, loaded or not. Reading them at list build time lets the UI accept image and audio uploads for a model selected through ?model=, which has no /props yet. * enum
…ml-org#25554) * ggml : add fallback definitions for missing aarch64 HWCAP bits * ggml : require HWCAP_ASIMDHP for the aarch64 fp16 cpu variants Also rename has_fp16_va to has_fp16, the field gates the whole FEAT_FP16 extension, scalar and vector half-precision arithmetic together.
…gml-org#26706) The reference implementation has two mutually exclusive prompt layouts. In non streaming mode the prefill carries the whole utterance text plus tts_eos summed with codec_pad, and the trailing text hidden collapses to a single tts_pad row. In streaming mode the prefill carries only the first text token and the trailing rows stream the rest of the text followed by tts_eos. The pipeline built the non streaming prefill but the streaming overlay, so the talker saw the utterance a second time during generation and read it twice before emitting codec_eos. The overlay is now the single tts_pad row that matches the prefill.
* add lru_sched * handle coalescing (req leaves waiting queue) * add tests * fix stream case * address review comments
…ggml-org#26711) * ui: set npm `min-release-age` to protect against supply-chain attacks * ui: bump to 7 days
…26707) * webui: load the model selected via ?model= Opening the WebUI with ?model= selects the model but doesn't load it. The load only starts when you send your first message, so you wait for it then. This loads it as soon as the page opens, while you're still typing your prompt. It's what the model dropdown already does, and it isn't awaited, so the UI still works while the model loads. This is the path the Llama macOS app uses to open the WebUI, so it's a common way in. * webui: gate the load behind ?load=true Loading on landing is opt-in, so a plain ?model= link behaves as before and doesn't allocate memory on its own. * webui: name the chat URL params Collects the query params the chat routes read into a URL_PARAMS constant, instead of repeating the literals across three files. NEW_CHAT_PARAM folds into it.
* mtmd: fix longest_edge ignoring min/max pixels * nits
* base : @-mention picker foundation - glob search, picker nav, highlight * feat : @-mention file/folder picker and mention badges in message bubbles * fix: Imports * feat : wire the @-mention picker into the chat form * fix: Bound the glob-search result cache key and prune stale entries
…oup (ggml-org#26708) ggml_metal_op_norm sized the threadgroup with `nth = std::min(nth, args.ne00_t)`, which can leave nth not a multiple of the simdgroup size. The kernels finish their row reduction with a cross-simdgroup step where each lane of the last simdgroup reads one per-simdgroup partial sum out of shmem_f32: if (tiisg == 0) { shmem_f32[sgitg] = sumf; } threadgroup_barrier(mem_flags::mem_threadgroup); sumf = shmem_f32[tiisg]; sumf = simd_sum(sumf); When the last simdgroup is partial it has fewer lanes than the threadgroup has simdgroups, so the tail of the partial sums is never read and the row sum is too small. For ne00_t = 33 nth becomes 33: two simdgroups, but only one lane in the second, so one of the two partial sums is dropped. The mean and variance are then wrong for the whole row. Round ne00_t up to a whole number of simdgroups instead. Rounding up rather than dropping the clamp keeps the threadgroup as small as possible: deleting the line would raise nth to the next power of two (ne00_t = 544 -> 1024 instead of 544), which costs idle lanes on 26 row lengths below 8192 that were already correct, including 1536 and 3584. GGML_OP_NORM is affected as well as GGML_OP_RMS_NORM - both dispatch through ggml_metal_op_norm. No mainstream LLM hidden size hits this: ne00_t is ne00/4 on the vectorized path, so 4096, 8192, 2048 and friends all give a multiple of 32. It is reachable from other norm shapes, e.g. 320-channel norms. Add NORM and RMS_NORM cases for ne0 = 33, 132 and 260 across the existing eps values. 33 exercises the scalar path and 132/260 the vectorized one, since only those divide by 4. Before, on M3 Pro: test-backend-ops test -b MTL0 -o NORM 25/50 test-backend-ops test -b MTL0 -o RMS_NORM 26/51 After: test-backend-ops test -b MTL0 -o NORM 50/50 test-backend-ops test -b MTL0 -o RMS_NORM 51/51 test-backend-ops test -b MTL0 13943/13943
test-backend-ops perf -o SSM_CONV on an Arc Pro B70, interleaved A/B against master, 6 reps, us/run: ne_a=[515,3328,1,1] ne_b=[4,3328,1,1] n_t=512 97.68 -> 52.95 1.85x ne_a=[937,8192,1,1] ne_b=[4,8192,1,1] n_t=934 516.16 -> 276.13 1.87x ne_a=[4,3328,1,1] ne_b=[4,3328,1,1] n_t=1 2.73 -> 2.71 flat llama-bench on qwen35 27B Q4_K - Medium (48 of its 64 blocks run ssm_conv), -ngl 99 -fa 1 -ctk f16 -ctv f16, interleaved passes of r=3: -b 2048 -ub 2048 pp2048 1045.1 / 1043.5 / 1043.7 -> 1069.5 / 1066.3 / 1065.9 +2.2% -b 2048 -ub 512 pp2048 771.8 / 772.7 -> 785.5 / 786.6 +1.8% -b 2048 -ub 512 tg128 23.81 / 23.88 -> 23.87 / 23.86 flat
* base : slash-command/misc foundation - model icon and focus-selector constants * feat : slash-command picker and command parsing helpers * refactor : wire command and @-mention pickers into the chat form * ui : improve model selector keyboard navigation and load/dismiss * feat: Unify markdown/raw-text rendering under one setting with migration * fix: Misc fixes - tool-call subtitle, assistant wrap, progress guards * feat: Clamp and style numeric settings inputs from registry bounds
* tests : speed-up test suite 3x * cont : print 30 slowest tests
* feat: Add contenteditable tokenizer for badge/code-chip chat input * feat: Add source-space undo/redo history for the rich input * feat: Split text glued to a closing code fence onto its own line * feat: Add ChatFormContenteditable rich input renderer * feat : wire the contenteditable into ChatForm with auto-switch gating
get_output runs the waveform work the pipeline defers to it, from a single trailing window to a full pass depending on the model. Measuring it keeps the reported total and the audio to process ratio honest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates dev branch with latest release (b10326) from ggml-org/llama.cpp