Filter by type:
Filter by tags:
granite-4.2-3b-q4
IBM Granite 4.2 3B is a compact multilingual reasoning model for chat,
coding, long-context tasks, and tool use. This entry uses the Q4_K_M
GGUF; a higher-fidelity Q8_0 build is available as a variant.
Repository: localaiLicense: apache-2.0
granite-4.2-3b-q8
IBM Granite 4.2 3B in the higher-fidelity Q8_0 GGUF format. It is a
compact multilingual reasoning model for chat, coding, and tool use.
Repository: localaiLicense: apache-2.0
granite-4.2-8b-q4
IBM Granite 4.2 8B is a multilingual reasoning model for chat, coding,
long-context tasks, and tool use. This entry uses the Q4_K_M GGUF; a
higher-fidelity Q8_0 build is available as a variant.
Repository: localaiLicense: apache-2.0
granite-4.2-8b-q8
IBM Granite 4.2 8B in the higher-fidelity Q8_0 GGUF format. It is a
multilingual reasoning model for chat, coding, and tool use.
Repository: localaiLicense: apache-2.0
granite-4.2-30b-q4
IBM Granite 4.2 30B is the family's flagship multilingual reasoning model
for chat, coding, long-context tasks, and tool use. This entry uses the
Q4_K_M GGUF; a higher-fidelity Q8_0 build is available as a variant.
Repository: localaiLicense: apache-2.0
granite-4.2-30b-q8
IBM Granite 4.2 30B in the higher-fidelity Q8_0 GGUF format. It is the
family's flagship multilingual reasoning model for chat, coding, and tool
use.
Repository: localaiLicense: apache-2.0
hy-mt2-1.8b-q4
Hy-MT2-1.8B is Tencent's compact multilingual translation model. It
follows translation instructions across 33 languages and supports tasks
such as terminology control, style transfer, and structure-preserving
translation.
This default entry uses the 1.1 GB Q4_K_M GGUF. A higher-quality Q8_0
model is available as a variant.
Repository: localaiLicense: apache-2.0
hy-mt2-1.8b-q8
Hy-MT2-1.8B in the higher-quality 1.9 GB Q8_0 GGUF format. This variant
preserves more model fidelity for hosts with enough memory.
Repository: localaiLicense: apache-2.0
nemotron-3.5-lightning-30b-a3b-q4
NVIDIA Nemotron 3.5 Lightning is a text-only hybrid Mamba-2, attention, and
mixture-of-experts model with 30B total parameters and 3B active parameters.
It targets reasoning, coding, tool use, multilingual chat, and long-context
agent workflows, with a context window of up to one million tokens.
This entry uses the official Q4_K_M GGUF. Automatic variant selection can
choose the smaller NVFP4 build or the higher-quality Q8_0 build when it fits.
Repository: localaiLicense: openmdw-1.1
nemotron-3.5-lightning-30b-a3b-nvfp4
NVIDIA Nemotron 3.5 Lightning 30B-A3B in the official NVFP4 GGUF format.
This is the smallest linked build and retains the model's reasoning, coding,
tool-use, multilingual, and long-context capabilities.
Repository: localaiLicense: openmdw-1.1
nemotron-3.5-lightning-30b-a3b-q8
NVIDIA Nemotron 3.5 Lightning 30B-A3B in the official high-quality Q8_0
GGUF format for hosts with enough memory.
Repository: localaiLicense: openmdw-1.1
muse-glimmer-30b
Muse Glimmer is Meta Superintelligence Labs' Apache-2.0 dense 30B model for
autonomous agentic work, coding, tool use, long-horizon reasoning, and
multimodal understanding. It supports more than 100 languages, interleaved
text and image input through its 1.8B-parameter perception encoder, and a
131K-token context window.
This entry uses the publisher's higher-quality dynamic K-quant GGUF and
official quantized vision projector. Automatic variant selection can use
the smaller 17 GB quantization or a DFlash-accelerated build when it fits.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-dflash
Muse Glimmer's higher-quality dynamic K-quant GGUF with the official
quantized perception encoder and DFlash drafter. DFlash proposes blocks of
up to 16 tokens for the target to verify in parallel, accelerating output
without changing model quality. Flash attention is enabled for this path.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder. It preserves the model's agentic, coding, tool-use,
multilingual, and image-understanding capabilities for hosts with less
memory than the dynamic quantization requires.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb-dflash
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder and DFlash drafter. This is the lowest-memory published
build that retains image understanding and block-speculative decoding.
Flash attention is enabled for the DFlash path.
Repository: localaiLicense: apache-2.0
nemotron-3-embed-1b-q4
Nemotron-3-Embed-1B is NVIDIA's multilingual text embedding model for
retrieval, semantic search, and RAG. This compact Q4_K_M GGUF produces
2,048-dimensional normalized embeddings and supports 36 languages. Prefix
retrieval queries with `query: ` and documents with `passage: `.
Repository: localaiLicense: openmdw-1.1
nemotron-3-embed-8b-q4
Nemotron-3-Embed-8B is NVIDIA's larger multilingual text embedding model
for retrieval, semantic search, and RAG. This Q4_K_M GGUF balances retrieval
quality with local resource use and supports 36 languages. Prefix retrieval
queries with `query: ` and documents with `passage: `.
Repository: localaiLicense: openmdw-1.1
inkling
# Inkling
BF16 |
NVFP4 |
Playground |
Tinker Cookbook |
Acceptable Use
## 1. General Information
Inkling is a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages. The model is designed to be used by developers building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems, and is suitable for general-purpose conversational use, instruction-following, and other natural language and multimodal tasks. It is released with open weights to support research, fine-tuning and integration into third-party products by downstream developers.
**Languages:** English, with general multilingual capabilities across other languages.
## 2. Getting Started
Try Inkling on the Tinker Playground or access via API using the Tinker Cookbook.
Inkling supports local deployment using the following open-source libraries:
* SGLang (recipe, PR)
* vLLM (recipe, PR)
* TokenSpeed (recipe, PR)
* Unsloth (recipe, PR)
* Huggingface (recipe, PR)
...
Repository: localaiLicense: apache-2.0
lfm2.5-2.6b
LFM2.5-2.6B is LiquidAI's compact, text-only reasoning model for on-device
agentic workloads. It has 2.69B parameters, a 128K-token context window,
multilingual support, and post-training for tool use, instruction following,
data extraction, RAG, and multi-step agents. This entry uses the recommended
Q4_K_M GGUF quantization from LiquidAI's official repository.
Repository: localaiLicense: other
lfm2.5-2.6b-q8
LFM2.5-2.6B is LiquidAI's compact, text-only reasoning model for on-device
agentic workloads. It has 2.69B parameters, a 128K-token context window,
multilingual support, and post-training for tool use, instruction following,
data extraction, RAG, and multi-step agents. This entry uses the higher-quality
Q8_0 GGUF quantization from LiquidAI's official repository.
Repository: localaiLicense: other
gemma-4-12b-agentic-fable5-composer2.5-v2-3.5x-tau2
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
> [!Note]
> This model card is for the Gemma 4 12B Unified model, which is part of the Gemma 4 family of open models. Built with the same multimodal functionality as Gemma 4 E2B and E4B (text, audio, image, and video inputs), it brings native audio and vision understanding directly to local environments without the need for separate encoders. This unified approach to multimodality makes the model encoder-free, offering a deployment size that is perfect for consumer devices and streamlined local execution.
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on E2B, E4B, and 12B) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages.
...
Repository: localaiLicense: apache-2.0
Page 1