Filter by type:
Filter by tags:
qwen3.5-0.8b-artemis-q5
TurinTech Artemis Q5_K_M quantization of Qwen3.5 0.8B for text chat,
reasoning, and tool use. This text-only build omits the vision encoder
and MTP layer. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
dirk-qwen3.8-27b-q5
Dirk in the higher-quality Q5_K_XL GGUF format, with MTP speculative
decoding and the shared F16 vision projector for multimodal prompts.
Repository: localaiLicense: apache-2.0
huihui-qwen3.8-27b-abliterated-q5
Huihui Qwen3.8 27B in Q5_K GGUF format, with the shared BF16 vision
projector and MTP speculative decoding through llama.cpp.
Repository: localaiLicense: apache-2.0
ornith-1.5-9b-q5
Ornith-1.5-9B in the Q5_K_M GGUF format, with the shared BF16
vision projector for multimodal prompts.
Repository: localaiLicense: mit
ornith-1.5-35b-a3b-q5
Ornith-1.5-35B-A3B in the Q5_K_M GGUF format, with the shared
BF16 vision projector for multimodal prompts.
Repository: localaiLicense: mit
nex-n2.5-mini-q5
Nex-N2.5-mini is a 35B-parameter Qwen3.5 MoE model for coding, tool use,
and computer and browser tasks with image input. This Q5_K_M GGUF build
includes the F16 vision projector and uses the embedded chat template.
Repository: localaiLicense: apache-2.0
tiel-coder-35b-a3b-q5-mtp
Tiel-Coder-35B-A3B in Q5_K_XL format with MTP speculative decoding and a
BF16 vision projector.
Repository: localaiLicense: mit
sharp-spark-x2.5-4b-q5
Sharp-Spark is an imatrix quantization of XHToken's Spark-X2.5-4B text model
with an adjusted chat template for coding. This Q5_K_XL build uses the
embedded Sharp-Spark template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
qwen3.8-35b-a3b-distill-q5
Empero's Qwen3.8 distillation into Qwen3.6-35B-A3B has 35B total
parameters and about 3B active per token. This Q5_K_M GGUF build
uses the embedded reasoning template and includes the F16 vision
projector. Vision is inherited from the base and was not evaluated
by the publisher.
Repository: localaiLicense: apache-2.0
gemma-4-12b-it-q5
Gemma 4 12B IT is Google's 12B instruction-tuned model for chat,
reasoning, tool use, and image understanding. This Unsloth Q5_K_M
GGUF build includes the F16 vision projector and uses the embedded
chat template. It defaults to a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-e4b-hauhaucs-aggressive-q5
HauhauCS Aggressive is an uncensored derivative of Google's Gemma 4 E4B
instruction model. This Q5_K_M GGUF supports text chat and image input
through the publisher's bundled F16 multimodal projector.
Uses the embedded chat template and the publisher's sampling settings.
The default context is 8192 tokens. The publisher declares the Gemma license.
Repository: localaiLicense: gemma
lythri-7b-a4b-q5
Lythri 7B A4B is a Gemma 4 E4B fine-tune for conversational companionship,
with 7.46B total parameters and 4.5B active parameters.
This Q5_K_M GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
zero-gemma4-e4b-openzero-q5-k-m
Zero Gemma4-E4B OpenZero is a Gemma 4 E4B instruction fine-tune for
local coding, research, chat, and agentic workflows. It was trained on
2,033 curated OpenZero examples and is distributed as one merged Q5_K_M
GGUF file.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
lfm2.5-24b-a2b-q5
LiquidAI LFM2.5-24B-A2B is a text-only reasoning model with 24B total
parameters and 2.45B active parameters for multilingual chat and tool use.
This entry uses the official Q5_K_M GGUF quantization.
Repository: localaiLicense: other
neohorse-1-4b-q5
NeoHorse-1-4B is TokenRhythm's text-only Qwen3.5-4B fine-tune for coding,
reasoning, and agentic tasks. This official Q5_K_M GGUF build uses the
embedded Jinja chat template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
minimax-h3-fl2va-pruned-q5
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly, so a clip comes back as an MP4 with a soundtrack.
Ask for speech in the prompt and the model lip-syncs it.
Runtime: vllm.cpp (backend vllm-cpp). Partition: fl2va. DiT layout:
pruned. Format: Q5_0.
This entry is the FL2VA partition, which serves text-to-video (t2va) and first/last-frame conditioning (fl2va). It refuses reference conditioning: for that use the matching ref2va entry. The DiT is the pruned DiT (AdaLN curve table in place of the per-block modulation projection; about 20B instead of 33B parameters) (~13.9 GB).
Render status: loads, not rendered. The vllm.cpp model page says the pruned Q5_0 file loads through the same path as Q8_0; only Q8_0 has been rendered.
The vllm.cpp page records its renders with the DiT expanded to bf16. This
entry does not expand it and keeps the quantised weights resident, so a
render in that mode is not covered by the page.
The weights are one DiT, the Qwen3-VL-32B encoder (~14.6 GB), the video VAE
(~10.4 GB), the audio VAE and the tokenizer, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768 at 124 frames
and 24 fps, about 5.2 seconds. Generation is slow: roughly 176 s per denoise
step at that canvas on a 20-SM device. Muxing the finished frames needs
ffmpeg on the host.
Repository: localaiLicense: other
minimax-h3-ref2va-pruned-q5
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly, so a clip comes back as an MP4 with a soundtrack.
Ask for speech in the prompt and the model lip-syncs it.
Runtime: vllm.cpp (backend vllm-cpp). Partition: ref2va. DiT layout:
pruned. Format: Q5_0.
This entry is the Ref2VA partition, which takes REFERENCE conditioning (reference images, clips and audio) so the subject or voice carries into the clip. It refuses plain text-to-video and first/last-frame requests: for those use the matching fl2va entry. The DiT is the pruned DiT (AdaLN curve table in place of the per-block modulation projection; about 20B instead of 33B parameters) (~13.9 GB).
Render status: loads, not rendered. The vllm.cpp model page says the pruned Q5_0 file loads through the same path as Q8_0; only Q8_0 has been rendered.
The vllm.cpp page records its renders with the DiT expanded to bf16. This
entry does not expand it and keeps the quantised weights resident, so a
render in that mode is not covered by the page.
The weights are one DiT, the Qwen3-VL-32B encoder (~14.6 GB), the video VAE
(~10.4 GB), the audio VAE and the tokenizer, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768 at 124 frames
and 24 fps, about 5.2 seconds. Generation is slow: roughly 176 s per denoise
step at that canvas on a 20-SM device. Muxing the finished frames needs
ffmpeg on the host.
Repository: localaiLicense: other
qwen3-reranker-0.6b
The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks No.1 in the MTEB multilingual leaderboard (as of June 5, 2025, score 70.58), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Reranker-0.6B** has the following features:
- Model Type: Text Reranking
- Supported Languages: 100+ Languages
- Number of Paramaters: 0.6B
- Context Length: 32k
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
zero-qwen3-8b-openzero-q5-k-m
Zero Qwen3-8B OpenZero is a Qwen3-8B fine-tune for local coding,
research, chat, and agentic workflows. It was trained on 2,033 curated
OpenZero examples and is distributed here as the standalone Q5_K_M GGUF.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
qwen3-embedding-4b
The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks **No.1** in the MTEB multilingual leaderboard (as of June 5, 2025, score **70.58**), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Embedding-4B-GGUF** has the following features:
- Model Type: Text Embedding
- Supported Languages: 100+ Languages
- Number of Paramaters: 4B
- Context Length: 32k
- Embedding Dimension: Up to 2560, supports user-defined output dimensions ranging from 32 to 2560
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
qwen3-embedding-8b
The Qwen3 Embedding series model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks **No.1** in the MTEB multilingual leaderboard (as of June 5, 2025, score **70.58**), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Embedding-8B-GGUF** has the following features:
- Model Type: Text Embedding
- Supported Languages: 100+ Languages
- Number of Paramaters: 8B
- Context Length: 32k
- Embedding Dimension: Up to 4096, supports user-defined output dimensions ranging from 32 to 4096
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
Page 1