Filter by type:
Filter by tags:
bge-small-en-v1.5-f16
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
bge-small-en-v1.5-q8-0
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This Q8_0 GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
bge-small-en-v1.5-q4-k-m
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This Q4_K_M GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
qwen3.8-flash-next-q4
Qwen3.8-Flash-Next is Qwen's 125B-parameter, 6B-active experimental
vision-language mixture-of-experts model. It targets agentic coding,
reasoning, tool use, and long-context workloads with a native 262K-token
context window.
This default entry uses Unsloth's UD-Q4_K_XL GGUF and BF16 vision
projector. Linked variants offer Q8_0 and AtomicChat's smaller IQ4_XS
and Q4_K_M builds with a separate n-gram table shard.
Repository: localaiLicense: other
huihui-qwen3.8-27b-abliterated
Huihui Qwen3.8 27B is an abliterated vision-language model published by
huihui-ai. This BF16 GGUF build includes the shared BF16 vision projector
and enables MTP speculative decoding through llama.cpp. Q4_K, Q5_K, Q6_K,
and Q8_0 variants are available as smaller downloads.
Repository: localaiLicense: apache-2.0
ornith-1.5-35b-a3b-apex-compact
Ornith-1.5-35B-A3B in the smaller APEX Compact GGUF format, with the
shared BF16 vision projector for multimodal prompts.
Repository: localaiLicense: mit
thomson-1.0-small-q4
Thomson-1.0-Small is a 35B-parameter mixture-of-experts model with about
3B active parameters. It focuses on legal, tax, journalism, research,
reasoning, tool use, and document processing. It supports text and image
input with a native context window of 262K tokens.
This default entry uses the Q4_K_M GGUF and BF16 vision projector. A
higher-quality Q8_0 model is available as a variant.
Repository: localaiLicense: polyform-strict-1.0.0
thomson-1.0-small-q8
Thomson-1.0-Small in the higher-quality Q8_0 GGUF format, with the shared
BF16 vision projector for multimodal prompts.
Repository: localaiLicense: polyform-strict-1.0.0
qwen3.8-27b-gsq-rco-iq2-xs
Qwen3.8-27B with ISTA DASLab's smallest GSQ-RCO mixed quantization. The
2.50-bit IQ2_XS model uses 8.4 GB for the weights and retains the shared
BF16 vision projector.
Repository: localaiLicense: apache-2.0
qwen3.8-27b-gsq-rco-iq3-xxs
Qwen3.8-27B with ISTA DASLab's 3.00-bit GSQ-RCO IQ3_XXS mixed
quantization. The 10.1 GB model offers a smaller alternative to IQ3_S
and retains the shared BF16 vision projector.
Repository: localaiLicense: apache-2.0
qwen3.8-2b-q4
Qwen3.8-2B is Empero AI's smallest Qwen3.8 reasoning distillation. It uses
the Qwen3.5-2B architecture and targets mathematics, instruction
following, tool use, and edge deployment with a native 262K-token context
window. This default entry uses Q4_K_M weights; a higher-quality Q8_0
build is available as a variant.
Repository: localaiLicense: apache-2.0
nemotron-3.5-lightning-30b-a3b-q4
NVIDIA Nemotron 3.5 Lightning is a text-only hybrid Mamba-2, attention, and
mixture-of-experts model with 30B total parameters and 3B active parameters.
It targets reasoning, coding, tool use, multilingual chat, and long-context
agent workflows, with a context window of up to one million tokens.
This entry uses the official Q4_K_M GGUF. Automatic variant selection can
choose the smaller NVFP4 build or the higher-quality Q8_0 build when it fits.
Repository: localaiLicense: openmdw-1.1
nemotron-3.5-lightning-30b-a3b-nvfp4
NVIDIA Nemotron 3.5 Lightning 30B-A3B in the official NVFP4 GGUF format.
This is the smallest linked build and retains the model's reasoning, coding,
tool-use, multilingual, and long-context capabilities.
Repository: localaiLicense: openmdw-1.1
muse-glimmer-30b
Muse Glimmer is Meta Superintelligence Labs' Apache-2.0 dense 30B model for
autonomous agentic work, coding, tool use, long-horizon reasoning, and
multimodal understanding. It supports more than 100 languages, interleaved
text and image input through its 1.8B-parameter perception encoder, and a
131K-token context window.
This entry uses the publisher's higher-quality dynamic K-quant GGUF and
official quantized vision projector. Automatic variant selection can use
the smaller 17 GB quantization or a DFlash-accelerated build when it fits.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder. It preserves the model's agentic, coding, tool-use,
multilingual, and image-understanding capabilities for hosts with less
memory than the dynamic quantization requires.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb-dflash
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder and DFlash drafter. This is the lowest-memory published
build that retains image understanding and block-speculative decoding.
Flash attention is enabled for the DFlash path.
Repository: localaiLicense: apache-2.0
deepseek-v4-flash-0731
# DeepSeek-V4-Flash-0731
Technical Report👁️
## Introduction
**DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
Notes:
1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
## Chat Template
...
Repository: localaiLicense: mit
parable-granite-4.1-3b-claude-fable-5
# Parable-Granite-4.1-3B-Claude-Fable-5
Granite 4.1 3B fine-tuned on genuine Claude Fable 5 and GPT-5.5 agent
traces (planning, tool use, reasoning from real agent sessions).
Agent-flavored small model: terminal workflows, idiomatic code fixes,
explanations. v2 recipe: completion-masked SFT, replay mix, seed-averaged
weights. Published corpus and eval harness.
Repository: localaiLicense: apache-2.0
pocket-35b-q2
POCKET-35B is an Apache-2.0 Qwen3.5-family mixture-of-experts model from
FINAL-Bench/VIDRAFT, derived from Darwin-36B-Opus and packaged for stock
llama.cpp. This entry uses the smaller Q2_K GGUF quantization.
Repository: localaiLicense: apache-2.0
pocket-26b-q2
POCKET-26B is an Apache-2.0 Gemma 4 26B-A4B mixture-of-experts model from
FINAL-Bench/VIDRAFT, tuned for Korean and packaged for stock llama.cpp.
This entry uses the smaller Q2_K GGUF quantization.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-genesis-hermes-v7-apex-compact
Qwen3.6-35B-A3B Genesis Hermes V7 in the smaller APEX Compact GGUF format,
with the shared F16 multimodal projector. This build preserves the model's
multimodal, reasoning, coding, and agentic capabilities for hosts with less
memory than the recommended full APEX build.
Repository: localaiLicense: apache-2.0
Page 1