Filter by type:
Filter by tags:
qwen3-4b-dflash
Qwen3-4B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. This is the canonical DFlash pairing documented upstream (`z-lab/Qwen3-4B-DFlash` + `Qwen/Qwen3-4B`).
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3-4B target; the ~0.5 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3-4B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.5-4b-dflash
Qwen3.5-4B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.5-4B target; the ~0.6 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3.5-4B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.5-9b-dflash
Qwen3.5-9B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.5-9B target; the ~1 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3.5-9B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.6-27b-dflash
Qwen3.6-27B (dense) paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. DFlash gives its largest speedups on dense targets like this one.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.6-27B target; the ~1.8 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-dflash
Qwen3.6-35B-A3B (Mixture-of-Experts, ~3B active per token) paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. DFlash speedups on MoE targets are smaller than on dense models, but still useful.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The UD-Q4_K_M file carries the full Qwen3.6-35B-A3B target; the ~0.4 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
vllm-omni-wan2.2-t2v
Wan2.2-T2V-A14B via vLLM-Omni - Text-to-video generation model from Wan-AI. Generates high-quality videos from text prompts using a 14B parameter diffusion model.
Repository: localaiLicense: apache-2.0
vllm-omni-wan2.2-i2v
Wan2.2-I2V-A14B via vLLM-Omni - Image-to-video generation model from Wan-AI. Generates high-quality videos from images using a 14B parameter diffusion model.
Repository: localaiLicense: apache-2.0
qwen3.6-27b-nvfp4-dflash-vllm-cpp
Qwen3.6-27B NVFP4 on vllm.cpp with DFlash block-diffusion speculative
decoding: the fastest configuration of this model the engine ships.
Where MTP drafts one token at a time, DFlash drafts a whole 16-token block in
a single non-autoregressive pass from a separate 3.5 GB drafter, then the
target verifies the block in one step. At concurrency 1 that measures 2.9x
the throughput of the same weights with speculation off, and at or above
vLLM's own DFlash-on decode.
Both checkpoints are installed for you: the target as a revision-pinned
snapshot, the drafter into models/Qwen3.6-27B-DFlash, which is where the
backend looks when speculative_config.model names it. The drafter shares the
target's embed_tokens and lm_head, so the two are not independently
swappable.
Needs a Blackwell-class NVIDIA GPU and roughly 28 GB of weights in total.
Repository: localaiLicense: apache-2.0
minimax-h3-fl2va-unpruned-q4-ggml
MiniMax-H3 served by stable-diffusion.cpp. It generates video AND stereo
audio jointly from a text prompt, so a clip comes back as an MP4 with a
soundtrack. Muxing the finished frames needs ffmpeg on the host.
Runtime: stable-diffusion.cpp (backend stablediffusion-ggml). Partition:
fl2va. DiT layout: unpruned. Format: Q4_K_M.
This entry is the FL2VA partition: text-to-video, first-frame (I2VA) and first/last-frame (FL2VA) conditioning. It refuses reference conditioning (reference image or audio): that needs minimax-h3-ref2va-pruned-q4-ggml, which is a separate checkpoint.
Render status: verified render (the stable-diffusion.cpp page shows a text-to-video and
an image-to-video render with this file).
Weights are the Q4_K_M quantisation of the DiT (~18.8 GB), the
Qwen3-VL-32B MiniMax-H3 text encoder (~18 GB), the fp16 video VAE (~5 GB)
and the fp32 audio VAE (~0.6 GB). MiniMax-H3 runs at 24 fps and the width
and height are rounded up to a multiple of 32. The frame count is
rounded up to the 17k + 5 grid.
Repository: localaiLicense: other
minimax-h3-fl2va-pruned-q4-ggml
MiniMax-H3 served by stable-diffusion.cpp. It generates video AND stereo
audio jointly from a text prompt, so a clip comes back as an MP4 with a
soundtrack. Muxing the finished frames needs ffmpeg on the host.
Runtime: stable-diffusion.cpp (backend stablediffusion-ggml). Partition:
fl2va. DiT layout: pruned. Format: Q4_K_M.
This entry is the FL2VA partition: text-to-video, first-frame (I2VA) and first/last-frame (FL2VA) conditioning. It refuses reference conditioning (reference image or audio): that needs minimax-h3-ref2va-pruned-q4-ggml, which is a separate checkpoint. It is the smaller, pruned counterpart of minimax-h3-fl2va-unpruned-q4-ggml.
Render status: rendered with stable-diffusion.cpp on CUDA (text-to-video, 512x288, 22 frames, with audio). Not yet rendered through LocalAI.
Weights are the Q4_K_M quantisation of the DiT (~11.4 GB), the
Qwen3-VL-32B MiniMax-H3 text encoder (~18 GB), the fp16 video VAE (~5 GB)
and the fp32 audio VAE (~0.6 GB). MiniMax-H3 runs at 24 fps and the width
and height are rounded up to a multiple of 32. The frame count is
rounded up to the 17k + 5 grid.
Repository: localaiLicense: other
minimax-h3-ref2va-pruned-q4-ggml
MiniMax-H3 served by stable-diffusion.cpp. It generates video AND stereo
audio jointly from a text prompt, so a clip comes back as an MP4 with a
soundtrack. Muxing the finished frames needs ffmpeg on the host.
Runtime: stable-diffusion.cpp (backend stablediffusion-ggml). Partition:
ref2va. DiT layout: pruned. Format: Q4_K_M.
This entry is the REF2VA partition, the one that takes REFERENCE conditioning: reference images, reference videos (a directory of frames, with an optional paired soundtrack) and reference audio steer the subject and the voice of the clip. It does not serve first/last-frame keyframes, and keyframes cannot be mixed with references. For plain text-to-video or keyframes use minimax-h3-fl2va-unpruned-q4-ggml.
Render status: verified render (the stable-diffusion.cpp page shows a reference-to-video
render with this file).
Weights are the Q4_K_M quantisation of the DiT (~11.4 GB), the
Qwen3-VL-32B MiniMax-H3 text encoder (~18 GB), the fp16 video VAE (~5 GB)
and the fp32 audio VAE (~0.6 GB). MiniMax-H3 runs at 24 fps and the width
and height are rounded up to a multiple of 32. The frame count is
rounded up to the 17k + 5 grid.
Repository: localaiLicense: other
realesrgan-x4plus-anime-6b
RealESRGAN x4plus anime 6B is a six-block ESRGAN model for anime images,
with a native 4x scale. Uses the official PyTorch .pth checkpoint through
stable-diffusion.cpp, without conversion. Upscale-only: not a text-to-image model.
Repository: localaiLicense: BSD-3-Clause
stable-diffusion-x4-upscaler
Stable Diffusion x4 Upscaler is Stability AI's diffusion-based super-resolution model. It enlarges low-resolution images by four times while reconstructing image detail.
Repository: localaiLicense: openrail++
dream-org_dream-v0-instruct-7b
This is the instruct model of Dream 7B, which is an open diffusion large language model with top-tier performance.
Repository: localaiLicense: apache-2.0
dreamshaper
A text-to-image model that uses Stable Diffusion 1.5 to generate images from text prompts. This model is DreamShaper model by Lykon.
Repository: localaiLicense: other
stable-diffusion-3-medium
Stable Diffusion 3 Medium is a Multimodal Diffusion Transformer (MMDiT) text-to-image model that features greatly improved performance in image quality, typography, complex prompt understanding, and resource-efficiency.
Repository: localaiLicense: stabilityai-ai-community
wan-2.1-t2v-1.3b-ggml
Wan 2.1 T2V 1.3B — text-to-video diffusion model, GGUF-quantized for the
stable-diffusion.cpp backend. Generates short (33-frame) 832x480 clips
from a text prompt. Cheapest Wan variant, suitable for CPU-offloaded
inference with ~10 GB of usable RAM.
Repository: localaiLicense: apache-2.0
wan-2.1-i2v-14b-480p-ggml
Wan 2.1 I2V 14B 480P — image-to-video diffusion, GGUF Q4 quantization.
Animates a reference image into a 33-frame 480p clip. Requires more
RAM than the 1.3B T2V variant; CPU offload enabled by default.
Repository: localaiLicense: apache-2.0
wan-2.1-flf2v-14b-720p-ggml
Wan 2.1 FLF2V 14B 720P — first-last-frame-to-video diffusion, GGUF Q4_K_M.
Takes a start and end reference image and interpolates a 33-frame clip
between them. Unlike the plain I2V variant this model feeds the end
frame through clip_vision as well, so it conditions semantically (not
just in pixel-space) on both endpoints. That makes it the right choice
for seamless loops (start_image == end_image) and clean narrative cuts.
Native 720p but accepts 480p resolutions; shares the same VAE, t5xxl
text encoder, and clip_vision_h as I2V 14B.
Repository: localaiLicense: apache-2.0
wan-2.1-i2v-14b-720p-ggml
Wan 2.1 I2V 14B 720P — image-to-video diffusion, GGUF Q4_K_M.
Native 720p sibling of the 480p I2V model: animates a single
reference image into a 33-frame clip at up to 1280x720. Trained
purely as image-to-video (no first-last-frame interpolation path),
so motion is freer and better-suited to single-anchor animation
than repurposing the FLF2V 720P variant for i2v. Shares the same
VAE, umt5_xxl text encoder, and clip_vision_h as the I2V 14B 480P
and FLF2V 14B 720P entries.
Repository: localaiLicense: apache-2.0
sd-1.5-ggml
Stable Diffusion 1.5
Repository: localaiLicense: creativeml-openrail-m
Page 1