winget install --id ggml.llamacppAbout llama.cpp
LLM inference in C/C++
What's new in b10357
opencl: transpose the K tile in local memory for FA prefill kernels (#26428) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework Linux: - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.14) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) Android: - Android arm64 (CPU) Windows: - Windows x64 (CPU) - Windows arm64 (CPU) - Windows arm64 (OpenCL Adreno) - Windows x64 (CUDA 12) - CUDA 12.4 DLLs - Windows x64 (CUDA 13) - CUDA 13.3 DLLs - Windows arm64 (CUDA 13) (preview) - CUDA 13.4 DLLs - Windows x64 (Vulkan) - Windows x64 (OpenVINO) - Windows x64 (SYCL) - Windows x64 (ROCm 7.14) openEuler: - DISABLED - openEuler x86 (310p) - openEuler x86 (910b, ACL Graph) - openEuler aarch64 (310p) - openEuler aarch64 (910b, ACL Graph) UI: - UI
Version history
| Version | Updated | Notes |
|---|---|---|
| b10357 | opencl: transpose the K tile in local memory for FA prefill kernels (#26428) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x... | |
| b10333 | Unknown | ggml-cpu : fix missing Q5_0 dispatch in SpaceMiT backend (#26792) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS X... |
| b10331 | Unknown | server: report the isolate working directory from get_info (#26773) - server: report the isolate working directory from get_info Without an explicit cwd, get_info fell back to the server process working directory even wh... |
| b10326 | Unknown | tts: account for the vocoder pass in the timings line (#26733) get_output runs the waveform work the pipeline defers to it, from a single trailing window to a full pass depending on the model. Measuring it keeps the repo... |
| b10299 | Unknown | metal : avoid threadgroup matrix array instantiation in kernel_lightning_indexer (#26646) - In MSL, declaring an array of matrix types like threadgroup half4x4 causes a 'no matching constructor' compilation error because... |
| b10293 | Unknown | ci : onboard AMD ROCm CI with gfx1151 fixes (#26544) - ci: prepare for amd rocm ci Signed-off-by: Aaron Teo aaron.teo1@ibm.com - ci: fix editorconfig-checker Signed-off-by: Aaron Teo aaron.teo1@ibm.com - ci: fix device n... |
| b10278 | Unknown | build : remove GGML_METAL_USE_BF16 from all build scripts (#26604) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS... |
| b10236 | Unknown | metal: implement DSv4 Lightning Indexer (#25893) - metal: implement F16 Lightning Indexer - Implement GGML_OP_LIGHTNING_INDEXER for 128-dimensional, 64-head inputs with F32 queries and weights plus F16 keys and masks. -... |
| b10223 | Unknown | test: fix some CI errors (#26415) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework Linux: - Ubuntu x64 (... |
| b10216 | Unknown | vulkan: add POOL_1D op (#25431) - vulkan : add pool1d push constants and pipeline field Declared data structures needed for POOL1D OP, which are the vk_op_pool1d_push_constants struct and pipeline_pool1d_f32 field. - vul... |
| b10199 | Unknown | server: support inp embd to generate next token (#26313) - server: support embd for sampled token - fix ~server_batch() Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64,... |
| b10173 | Unknown | model: Add Laguna-S-2.1 LLM_TYPE (#26233) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework Linux: - Ubun... |
| b10155 | Unknown | mtmd: support MiMo-V2.5 audio input (RVQ-based model) (#26190) - gguf converter for mimo audio - fix conv - cpp impl - nits - nits 2 Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Sil... |
| b10144 | Unknown | server + ui: fix stream routes for model names containing a slash (#26137) - server + ui: refactor resumable stream routes to query string conv_id The conversation id can embed a model name containing slashes (ggml-org/.... |
| b10107 | Unknown | hexagon: fix Windows crash when op_poll is enabled (#26029) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFrame... |
| b10103 | Unknown | metal : add f16 type support to leaky relu (#25981) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework Lin... |
| b10092 | Unknown | ggml: enable PowerPC backend variants on AIX (#25983) - ggml: enable PowerPC backend variants on AIX Allow the PowerPC CPU backend variants to be built on AIX by extending the platform check in the CMake configuration. T... |
| b10081 | Unknown | common: resolve draft repo to its requested sidecar (#25955) With -hfd pointing to a repo shipping speculative sidecars, the draft resolved to the main model of that repo, since find_best_model() excludes sidecar files,... |
| b10075 | Unknown | hexagon: add CLAMP op (#25934) Website: - https://llama.app macOS/iOS: - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework Linux: - Ubuntu x64 (CPU... |
| b10068 | Unknown | model: rotate injected K/V cache for DFlash (#25823) - dflash: rotate injected K/V cache when using K/V quantization - Update src/models/dflash.cpp Co-authored-by: Georgi Gerganov ggerganov@gmail.com - clearer format - r... |