Filter by type:
Filter by tags:
gemma-4-31b-fable-coder-q4
Gemma-4-31B-Fable-Coder is DavidrPatton's Gemma 4 31B fine-tune for
coding and visual interface tasks. This entry uses the Q4_K_M GGUF
with the F16 vision projector and a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-31b-fable-coder-q8
Gemma-4-31B-Fable-Coder is DavidrPatton's Gemma 4 31B fine-tune for
coding and visual interface tasks. This entry uses the Q8_0 GGUF
with the F16 vision projector and a 32,768-token context.
Repository: localaiLicense: apache-2.0
lythri-4b-a2b-q4
Lythri 4B A2B is a Gemma 4 E2B fine-tune for conversational companionship.
This Q4_K_M GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
lythri-4b-a2b-q8
Lythri 4B A2B is a Gemma 4 E2B fine-tune for conversational companionship.
This Q8_0 GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-31b-ortenzya-q4
Ortenzya is a Gemma 4 31B instruction-tuned model from llmfan46 for
creative writing, translation, and roleplay. This entry uses the Q4_K_M
GGUF with the BF16 vision projector and a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-31b-ortenzya-q8
Ortenzya is a Gemma 4 31B instruction-tuned model from llmfan46 for
creative writing, translation, and roleplay. This entry uses the Q8_0
GGUF with the BF16 vision projector and a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-31b-scotoma-2-q4
Gemma 4 31B Scotoma 2 is a multimodal Gemma 4 31B instruction-tuned model from ReadyArt. It applies a bounded refusal edit and preference training intended to reduce repetitive prose patterns while retaining the base model's text and image capabilities. This entry uses the 18.7 GB Q4_K_M GGUF and the matching Q8_0 vision projector.
License: Apache 2.0 | Base model: Google Gemma 4 31B IT
Repository: localaiLicense: apache-2.0
gemma-4-31b-scotoma-2-q8
Gemma 4 31B Scotoma 2 is a multimodal Gemma 4 31B instruction-tuned model from ReadyArt. It applies a bounded refusal edit and preference training intended to reduce repetitive prose patterns while retaining the base model's text and image capabilities. This higher-fidelity entry uses the 32.6 GB Q8_0 GGUF and the matching Q8_0 vision projector.
License: Apache 2.0 | Base model: Google Gemma 4 31B IT
Repository: localaiLicense: apache-2.0
nemotron-3.5-bolt-e4b-q4
NVIDIA Nemotron 3.5 Bolt E4B is a Gemma 4 E4B fine-tune for
conversational assistants, instruction following, and multi-turn tool use.
This UD-Q4_K_XL GGUF build includes the F16 vision projector for
chat and image input, with a 32,768-token context by default.
Repository: localaiLicense: openmdw-1.1
nemotron-3.5-bolt-e4b-q8
NVIDIA Nemotron 3.5 Bolt E4B is a Gemma 4 E4B fine-tune for
conversational assistants, instruction following, and multi-turn tool use.
This UD-Q8_K_XL GGUF build includes the F16 vision projector for
chat and image input, with a 32,768-token context by default.
Repository: localaiLicense: openmdw-1.1
gemma-4-e4b-heretic-q4
Gemma 4 E4B Heretic is llmfan46's abliterated version of Gemma 4 E4B IT.
This entry uses the Q4_K_M GGUF and BF16 vision projector for chat
and image input, with a 32,768-token context by default.
Repository: localaiLicense: apache-2.0
gemma-4-e4b-heretic-q8
Gemma 4 E4B Heretic is llmfan46's abliterated version of Gemma 4 E4B IT.
This entry uses the Q8_0 GGUF and BF16 vision projector for chat
and image input, with a 32,768-token context by default.
Repository: localaiLicense: apache-2.0
gemma-4-e4b-hauhaucs-aggressive-q4
HauhauCS Aggressive is an uncensored derivative of Google's Gemma 4 E4B
instruction model. This Q4_K_M GGUF supports text chat and image input
through the publisher's bundled F16 multimodal projector.
Uses the embedded chat template and the publisher's sampling settings.
The default context is 8192 tokens. The publisher declares the Gemma license.
Repository: localaiLicense: gemma
gemma-4-e4b-hauhaucs-aggressive-q5
HauhauCS Aggressive is an uncensored derivative of Google's Gemma 4 E4B
instruction model. This Q5_K_M GGUF supports text chat and image input
through the publisher's bundled F16 multimodal projector.
Uses the embedded chat template and the publisher's sampling settings.
The default context is 8192 tokens. The publisher declares the Gemma license.
Repository: localaiLicense: gemma
lythri-7b-a4b-q5
Lythri 7B A4B is a Gemma 4 E4B fine-tune for conversational companionship,
with 7.46B total parameters and 4.5B active parameters.
This Q5_K_M GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
lythri-7b-a4b-q6
Lythri 7B A4B is a Gemma 4 E4B fine-tune for conversational companionship,
with 7.46B total parameters and 4.5B active parameters.
This Q6_K GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
lythri-7b-a4b-q8
Lythri 7B A4B is a Gemma 4 E4B fine-tune for conversational companionship,
with 7.46B total parameters and 4.5B active parameters.
This Q8_0 GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
zero-gemma4-e4b-openzero-q5-k-m
Zero Gemma4-E4B OpenZero is a Gemma 4 E4B instruction fine-tune for
local coding, research, chat, and agentic workflows. It was trained on
2,033 curated OpenZero examples and is distributed as one merged Q5_K_M
GGUF file.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
gemma-4-e2b-it-qat-mtp
Gemma 4 E2B IT QAT (Google DeepMind) paired with its Multi-Token Prediction (MTP) drafter head for speculative decoding on the llama.cpp backend. The Q4_K_XL target carries the full multimodal (text + image) model; the small `mtp-gemma-4-E2B-it` head predicts several tokens ahead which the target verifies in parallel, accelerating generation with no change to output quality. E2B is a MatFormer "effective 2B" elastic variant, well suited to lightweight and on-device deployments.
The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here. It uses the upstream `gemma4-assistant` architecture registered by llama.cpp PR #23398, so it loads on stock llama.cpp without any patch.
License: Apache 2.0 | Authors: Google DeepMind (target/drafter checkpoints), Unsloth (GGUF conversion)
Repository: localaiLicense: apache-2.0
gemma-4-e4b-it-qat-mtp
Gemma 4 E4B IT QAT (Google DeepMind) paired with its Multi-Token Prediction (MTP) drafter head for speculative decoding on the llama.cpp backend. The Q4_K_XL target carries the full multimodal (text + image) model; the small `mtp-gemma-4-E4B-it` head predicts several tokens ahead which the target verifies in parallel, accelerating generation with no change to output quality. E4B is a MatFormer "effective 4B" elastic variant, balancing quality and footprint for on-device and edge deployments.
The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here. It uses the upstream `gemma4-assistant` architecture registered by llama.cpp PR #23398, so it loads on stock llama.cpp without any patch.
License: Apache 2.0 | Authors: Google DeepMind (target/drafter checkpoints), Unsloth (GGUF conversion)
Repository: localaiLicense: apache-2.0
gemma-4-12b-it-qat-mtp
Gemma 4 12B IT QAT (Google DeepMind) paired with its Multi-Token Prediction (MTP) drafter head for speculative decoding on the llama.cpp backend. The Q4_K_XL target carries the full multimodal (text + image) model; the small `mtp-gemma-4-12B-it` head predicts several tokens ahead which the target verifies in parallel, accelerating generation with no change to output quality. As a dense model, Gemma 4 12B is among the sizes that benefit most from MTP, with the llama.cpp PR reporting well over 1.4x decode speedup.
The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here. It uses the upstream `gemma4-assistant` architecture registered by llama.cpp PR #23398, so it loads on stock llama.cpp without any patch.
License: Apache 2.0 | Authors: Google DeepMind (target/drafter checkpoints), Unsloth (GGUF conversion)
Repository: localaiLicense: apache-2.0
Page 1