Filter by type:
Filter by tags:
humanizer-12b-bf16
Humanizer is a 12B Gemma 4 fine-tune for rewriting English and Chinese text.
This unquantized BF16 GGUF build uses llama.cpp with an 8,192-token context.
This entry uses the completions API with the publisher's exact rewriting prompt.
Repository: localaiLicense: apache-2.0
bge-large-en-v1.5-f16
BGE-large-en-v1.5 is a 335M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 1024-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
bge-base-en-v1.5-f16
BGE-base-en-v1.5 is a 109M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 768-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
bge-small-en-v1.5-f16
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
spark-s1-4b-v6
spark-s1-4b-v6 is a Qwen3.5-4B decision model (Open Spark Jev): given a state and a question with a fixed menu of options it
returns a calibrated probability per option from the first-token logprobs of the option letters. It is not a chat model:
send the rendered prompt to /v1/completions with max_tokens 1, logprobs true, and top_logprobs 20.
Check that every option letter appears in the returned probabilities. Then apply softmax(logprob / 1.481) over those letters.
Q8_0 GGUF (agrees with the bf16 model on 99.5% of held-out decisions).
Repository: localaiLicense: apache-2.0
ornith-1.5-9b-uncensored
# Ornith-1.5-9B-uncensored
An **abliterated** (refusal-direction-ablated) build of
`ornith-ai/Ornith-1.5-9B`, produced with
ZeroFuse and published by
junafinity.
This is the **9B control checkpoint** (bf16). Mac users should start from the MLX-8bit or GGUF-8bit siblings. The official 9B base has **no `mtp.*` tensors**; nothing was grafted.
**Vision tower and MTP heads are preserved** — see
Vision & MTP preservation for the before/after audit.
## Intended use: red teaming and defensive cybersecurity research
These uncensored (abliterated) weights are built as a **research instrument** for red teaming and defensive cybersecurity work. Safety training suppresses the *display* of capability, not capability itself. A refusal tells you the model declined. It does not tell you whether the weights could have complied. That conflation underestimates the true ceiling and hides holes in *your* filters, classifiers, and policy layer.
Use each uncensored checkpoint as the **treatment half of a controlled pair** against its original base model:
...
Repository: localaiLicense: apache-2.0
s1-mini-f16
S1-mini by Superwhisper in the publisher's 1.4 GB F16 GGUF format. This
variant preserves full model fidelity for hosts with enough memory.
Repository: localaiLicense: s1-mini-license
llm-jp-4-33b-thinking-bf16
LLM-jp-4-33B-thinking in the official 66.4 GB BF16 GGUF format. This
variant preserves the original model precision for hosts with enough
memory.
Repository: localaiLicense: apache-2.0
mxbai-embed-large-v1-f16
Mixedbread's mxbai-embed-large-v1 in the official full-precision F16
GGUF format. This 335M-parameter English BERT model produces
1,024-dimensional embeddings for retrieval, semantic search, and RAG.
Repository: localaiLicense: apache-2.0
maple-preview-tq1-0-head-f16
Maple-Preview is DeepGrove's 20B mixture-of-experts reasoning model
with about 1B active parameters. This build uses TQ1_0
ternary GGUF weights with a F16 output head.
It uses the publisher's CPU configuration and embedded Jinja chat
template with an 8K default context. The native context is 128K tokens.
Repository: localaiLicense: mit
maple-preview-tq2-0-head-f16
Maple-Preview is DeepGrove's 20B mixture-of-experts reasoning model
with about 1B active parameters. This build uses TQ2_0
ternary GGUF weights with a F16 output head.
It uses the publisher's CPU configuration and embedded Jinja chat
template with an 8K default context. The native context is 128K tokens.
Repository: localaiLicense: mit
neohorse-1-4b-bf16
NeoHorse-1-4B is TokenRhythm's text-only Qwen3.5-4B fine-tune for coding,
reasoning, and agentic tasks. This official BF16 GGUF build uses the
embedded Jinja chat template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
ced-base-f16
CED (Consistent Ensemble Distillation, Xiaomi) is a sound-event classifier that tags everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark, ...) into the 527-class AudioSet ontology. This is the f16 GGUF for the ced backend (a standalone C++/ggml port). Recommended default: fastest on CPU and near-lossless. Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-tiny-f16
CED-tiny (5.5M params, Pi-class / edge) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-mini-f16
CED-mini (9.6M params, low-power) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-small-f16
CED-small (22M params, balanced size/accuracy) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
qwen3-4b-vllm-cpp
Qwen3-4B on vllm.cpp, in bf16. The small end of the engine's gated dense
family, which reaches parity with vLLM on every axis at concurrency 1.
bf16 rather than NVFP4 on purpose: this is the entry that runs where the
flagship NVFP4 checkpoints cannot, including Apple Silicon via Metal, Vulkan
and plain CPU. Roughly 8 GB of weights, plus about 4.5 GB of KV cache at the
context configured here.
Tool calling and the thinking split are parsed inside the engine.
Repository: localaiLicense: apache-2.0
qwen3-0.6b-vllm-cpp
Qwen3-0.6B on vllm.cpp, in bf16. Roughly 1.4 GB of weights, which makes it
the cheapest way to confirm a vllm-cpp install actually serves before
committing disk and memory to one of the large checkpoints.
It runs anywhere the backend does, CPU included, and it is a real chat model
rather than a stub, so tool calling and the thinking split can be exercised
on it too.
Repository: localaiLicense: apache-2.0
minimax-h3-fl2va-unpruned-q4
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly, so a clip comes back as an MP4 with a soundtrack.
Ask for speech in the prompt and the model lip-syncs it.
Runtime: vllm.cpp (backend vllm-cpp). Partition: fl2va. DiT layout:
unpruned. Format: Q4_K_M.
This entry is the FL2VA partition, which serves text-to-video (t2va) and first/last-frame conditioning (fl2va). It refuses reference conditioning: for that use the matching ref2va entry. The DiT is the original time-embedder DiT (~19.9 GB).
Render status: verified render (end to end on the vllm.cpp model page).
Q3_K_M of the same DiT is verified BAD and is not offered.
The vllm.cpp page records its renders with the DiT expanded to bf16. This
entry does not expand it and keeps the quantised weights resident, so a
render in that mode is not covered by the page.
The weights are one DiT, the Qwen3-VL-32B encoder (~14.6 GB), the video VAE
(~10.4 GB), the audio VAE and the tokenizer, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768 at 124 frames
and 24 fps, about 5.2 seconds. Generation is slow: roughly 176 s per denoise
step at that canvas on a 20-SM device. Muxing the finished frames needs
ffmpeg on the host.
Repository: localaiLicense: other
minimax-h3-fl2va-q4
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly, so a clip comes back as an MP4 with a soundtrack.
Ask for speech in the prompt and the model lip-syncs it.
Runtime: vllm.cpp (backend vllm-cpp). Partition: fl2va. DiT layout:
unpruned. Format: Q4_K_M.
This entry is the FL2VA partition, which serves text-to-video (t2va) and first/last-frame conditioning (fl2va). It refuses reference conditioning: for that use the matching ref2va entry. The DiT is the original time-embedder DiT (~19.9 GB).
Render status: verified render (end to end on the vllm.cpp model page).
Q3_K_M of the same DiT is verified BAD and is not offered.
The vllm.cpp page records its renders with the DiT expanded to bf16. This
entry does not expand it and keeps the quantised weights resident, so a
render in that mode is not covered by the page.
The weights are one DiT, the Qwen3-VL-32B encoder (~14.6 GB), the video VAE
(~10.4 GB), the audio VAE and the tokenizer, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768 at 124 frames
and 24 fps, about 5.2 seconds. Generation is slow: roughly 176 s per denoise
step at that canvas on a 20-SM device. Muxing the finished frames needs
ffmpeg on the host.
Repository: localaiLicense: other
Page 1