Model Gallery

25 models from 1 repositories

Filter by type:

Filter by tags:

qwen3-vl-30b-a3b-instruct
Meet Qwen3-VL — the most powerful vision-language model in the Qwen series to date. This generation delivers comprehensive upgrades across the board: superior text understanding & generation, deeper visual perception & reasoning, extended context length, enhanced spatial and video dynamics comprehension, and stronger agent interaction capabilities. Available in Dense and MoE architectures that scale from edge to cloud, with Instruct and reasoning‑enhanced Thinking editions for flexible, on-demand deployment. #### Key Enhancements: * **Visual Agent**: Operates PC/mobile GUIs—recognizes elements, understands functions, invokes tools, completes tasks. * **Visual Coding Boost**: Generates Draw.io/HTML/CSS/JS from images/videos. * **Advanced Spatial Perception**: Judges object positions, viewpoints, and occlusions; provides stronger 2D grounding and enables 3D grounding for spatial reasoning and embodied AI. * **Long Context & Video Understanding**: Native 256K context, expandable to 1M; handles books and hours-long video with full recall and second-level indexing. * **Enhanced Multimodal Reasoning**: Excels in STEM/Math—causal analysis and logical, evidence-based answers. * **Upgraded Visual Recognition**: Broader, higher-quality pretraining is able to “recognize everything”—celebrities, anime, products, landmarks, flora/fauna, etc. * **Expanded OCR**: Supports 32 languages (up from 19); robust in low light, blur, and tilt; better with rare/ancient characters and jargon; improved long-document structure parsing. * **Text Understanding on par with pure LLMs**: Seamless text–vision fusion for lossless, unified comprehension. #### Model Architecture Updates: 1. **Interleaved-MRoPE**: Full‑frequency allocation over time, width, and height via robust positional embeddings, enhancing long‑horizon video reasoning. 2. **DeepStack**: Fuses multi‑level ViT features to capture fine-grained details and sharpen image–text alignment. 3. **Text–Timestamp Alignment:** Moves beyond T‑RoPE to precise, timestamp‑grounded event localization for stronger video temporal modeling. This is the weight repository for Qwen3-VL-30B-A3B-Instruct.

Repository: localaiLicense: apache-2.0

qwen3-4b-ra-sft
a 4B-sized agentic reasoning model that is finetuned with our 3k Agentic SFT dataset, based on Qwen3-4B-Instruct-2507. In our work, we systematically investigate three dimensions of agentic RL: data, algorithms, and reasoning modes. Our findings reveal 🎯 Data Quality Matters: Real end-to-end trajectories and high-diversity datasets significantly outperform synthetic alternatives ⚡ Training Efficiency: Exploration-friendly techniques like reward clipping and entropy maintenance boost training efficiency 🧠 Reasoning Strategy: Deliberative reasoning with selective tool calls surpasses frequent invocation or verbose self-reasoning We contribute high-quality SFT and RL datasets, demonstrating that simple recipes enable even 4B models to outperform 32B models on the most challenging reasoning benchmarks.

Repository: localaiLicense: apache-2.0

demyagent-4b-i1
This repository contains the DemyAgent-4B model weights, a 4B-sized agentic reasoning model that achieves state-of-the-art performance on challenging benchmarks including AIME2024/2025, GPQA-Diamond, and LiveCodeBench-v6. DemyAgent-4B is trained using our GRPO-TCR recipe with 30K high-quality agentic RL data, demonstrating that small models can outperform much larger alternatives (14B/32B) through effective RL training strategies. 🌟 Introduction In our work, we systematically investigate three dimensions of agentic RL: data, algorithms, and reasoning modes. Our findings reveal: 🎯 Data Quality Matters: Real end-to-end trajectories and high-diversity datasets significantly outperform synthetic alternatives ⚡ Training Efficiency: Exploration-friendly techniques like reward clipping and entropy maintenance boost training efficiency 🧠 Reasoning Strategy: Deliberative reasoning with selective tool calls surpasses frequent invocation or verbose self-reasoning We contribute high-quality SFT and RL datasets, demonstrating that simple recipes enable even 4B models to outperform 32B models on the most challenging reasoning benchmarks.

Repository: localaiLicense: apache-2.0

l3.3-ms-nevoria-70b
This model was created as I liked the storytelling of EVA, the prose and details of scenes from EURYALE and Anubis, enhanced with Negative_LLAMA to kill off the positive bias with a touch of nemotron sprinkeled in. The choice to use the lorablated model as a base was intentional - while it might seem counterintuitive, this approach creates unique interactions between the weights, similar to what was achieved in the original Astoria model and Astoria V2 model . Rather than simply removing refusals, this "weight twisting" effect that occurs when subtracting the lorablated base model from the other models during the merge process creates an interesting balance in the final model's behavior. While this approach differs from traditional sequential application of components, it was chosen for its unique characteristics in the model's responses.

Repository: localaiLicense: llama3.3

l3.3-nevoria-r1-70b
This model builds upon the original Nevoria foundation, incorporating the Deepseek-R1 reasoning architecture to enhance dialogue interaction and scene comprehension. While maintaining Nevoria's core strengths in storytelling and scene description (derived from EVA, EURYALE, and Anubis), this iteration aims to improve prompt adherence and creative reasoning capabilities. The model also retains the balanced perspective introduced by Negative_LLAMA and Nemotron elements. Also, the model plays the card to almost a fault, It'll pick up on minor issues and attempt to run with them. Users had it call them out for misspelling a word while playing in character. Note: While Nevoria-R1 represents a significant architectural change, rather than a direct successor to Nevoria, it operates as a distinct model with its own characteristics. The lorablated model base choice was intentional, creating unique weight interactions similar to the original Astoria model and Astoria V2 model. This "weight twisting" effect, achieved by subtracting the lorablated base model during merging, creates an interesting balance in the model's behavior. While unconventional compared to sequential component application, this approach was chosen for its unique response characteristics.

Repository: localaiLicense: eva-llama3.3

steelskull_l3.3-electra-r1-70b
L3.3-Electra-R1-70b is the newest release of the Unnamed series, this is the 6th iteration based of user feedback. Built on a custom DeepSeek R1 Distill base (TheSkullery/L3.1x3.3-Hydroblated-R1-70B-v4.4), Electra-R1 integrates specialized components through the SCE merge method. The model uses float32 dtype during processing with a bfloat16 output dtype for optimized performance. Electra-R1 serves newest gold standard and baseline. User feedback consistently highlights its superior intelligence, coherence, and unique ability to provide deep character insights. Through proper prompting, the model demonstrates advanced reasoning capabilities and unprompted exploration of character inner thoughts and motivations. The model utilizes the custom Hydroblated-R1 base, created for stability and enhanced reasoning. The SCE merge method's settings are precisely tuned based on extensive community feedback (of over 10 diffrent models from Nevoria to Cu-Mai), ensuring optimal component integration while maintaining model coherence and reliability. This foundation establishes Electra-R1 as the benchmark upon which its variant models build and expand.

Repository: localaiLicense: eva-llama3.3

tarek07_legion-v2.1-llama-70b
My biggest merge yet, consisting of a total of 20 specially curated models. My methodology in approaching this was to create 5 highly specialized models: A completely uncensored base A very intelligent model based on UGI, Willingness and NatInt scores on the UGI Leaderboard A highly descriptive writing model, specializing in creative and natural prose A RP model specially merged with fine-tuned models that use a lot of RP datasets The secret ingredient: A completely unhinged, uncensored final model These five models went through a series of iterations until I got something I thought worked well and then combined them to make LEGION. The full list of models used in this merge is below: TheDrummer/Fallen-Llama-3.3-R1-70B-v1 Sao10K/Llama-3.3-70B-Vulpecula-r1 Sao10K/L3-70B-Euryale-v2.1 SicariusSicariiStuff/Negative_LLAMA_70B allura-org/Bigger-Body-70b Sao10K/70B-L3.3-mhnnn-x1 Sao10K/L3.3-70B-Euryale-v2.3 Doctor-Shotgun/L3.3-70B-Magnum-v4-SE Sao10K/L3.1-70B-Hanami-x1 Sao10K/70B-L3.3-Cirrus-x1 EVA-UNIT-01/EVA-LLaMA-3.33-70B-v0.1 TheDrummer/Anubis-70B-v1 ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4 LatitudeGames/Wayfarer-Large-70B-Llama-3.3 NeverSleep/Lumimaid-v0.2-70B mlabonne/Hermes-3-Llama-3.1-70B-lorablated ReadyArt/Forgotten-Safeword-70B-3.6 ReadyArt/Fallen-Abomination-70B-R1-v4.1 ReadyArt/Fallen-Safeword-70B-R1-v4.1 huihui-ai/Llama-3.3-70B-Instruct-abliterated

Repository: localaiLicense: llama3.3

llama_3.3_70b_darkhorse-i1
Dark coloration L3.3 merge, to be included in my merges. Can also be tried as a standalone to have a darker Llama Experience, but I didn't take the time. Edit : I took the time, and it meets its purpose. It's average on the basic metrics (smarts, perplexity), but it's not woke and unhinged indeed. The model is not abliterated, though. It has refusals on the usual point-blank questions. I will play with it more, because it has potential. My note : 3/5 as a standalone. 4/5 as a merge brick. Warning : this model can be brutal and vulgar, more than most of my previous merges.

Repository: localaiLicense: llama3.3

l3.3-geneticlemonade-unleashed-v2-70b
An experimental release. zerofata/GeneticLemonade-Unleashed qlora trained on a test dataset. Performance is improved from the original in my testing, but there are possibly (likely?) areas where the model will underperform which I am looking for feedback on. This is a creative model intended to excel at character driven RP / ERP. It has not been tested or trained on adventure stories or any large amounts of creative writing.

Repository: localaiLicense: llama3

steelskull_l3.3-shakudo-70b
L3.3-Shakudo-70b is the result of a multi-stage merging process by Steelskull, designed to create a powerful and creative roleplaying model with a unique flavor. The creation process involved several advanced merging techniques, including weight twisting, to achieve its distinct characteristics. Stage 1: The Cognitive Foundation & Weight Twisting The process began by creating a cognitive and tool-use focused base model, L3.3-Cogmoblated-70B. This was achieved through a `model_stock` merge of several models known for their reasoning and instruction-following capabilities. This base was built upon `nbeerbower/Llama-3.1-Nemotron-lorablated-70B`, a model intentionally "ablated" to skew refusal behaviors. This technique, known as weight twisting, helps the final model adopt more desirable response patterns by building upon a foundation that is already aligned against common refusal patterns. Stage 2: The Twin Hydrargyrum - Flavor and Depth Two distinct models were then created from the Cogmoblated base: L3.3-M1-Hydrargyrum-70B: This model was merged using `SCE`, a technique that enhances creative writing and prose style, giving the model its unique "flavor." The Top_K for this merge were set at 0.22 . L3.3-M2-Hydrargyrum-70B: This model was created using a `Della_Linear` merge, which focuses on integrating the "depth" of various roleplaying and narrative models. The settings for this merge were set at: (lambda: 1.1) (weight: 0.2) (density: 0.7) (epsilon: 0.2) Final Stage: Shakudo The final model, L3.3-Shakudo-70b, was created by merging the two Hydrargyrum variants using a 50/50 `nuslerp`. This final step combines the rich, creative prose (flavor) from the SCE merge with the strong roleplaying capabilities (depth) from the Della_Linear merge, resulting in a model with a distinct and refined narrative voice. A special thank you to Nectar.ai for their generous support of the open-source community and my projects. Additionally, a heartfelt thanks to all the Ko-fi supporters who have contributed—your generosity is deeply appreciated and helps keep this work going and the Pods spinning.

Repository: localaiLicense: llama3.3

llama3.1-gutenberg-doppel-70b
mlabonne/Hermes-3-Llama-3.1-70B-lorablated finetuned on jondurbin/gutenberg-dpo-v0.1 and nbeerbower/gutenberg2-dpo.

Repository: localaiLicense: llama3.1

hermes-3-llama-3.1-70b-lorablated
This is an uncensored version of NousResearch/Hermes-3-Llama-3.1-70B using lorablation. The recipe is based on @grimjim's grimjim/Llama-3.1-8B-Instruct-abliterated_via_adapter (special thanks): Extraction: We extract a LoRA adapter by comparing two models: a censored Llama 3 (meta-llama/Meta-Llama-3-70B-Instruct) and an abliterated Llama 3.1 (failspy/Meta-Llama-3.1-70B-Instruct-abliterated). Merge: We merge this new LoRA adapter using task arithmetic to the censored NousResearch/Hermes-3-Llama-3.1-70B to abliterate it.

Repository: localaiLicense: llama3.1

hermes-3-llama-3.1-8b-lorablated
This is an uncensored version of NousResearch/Hermes-3-Llama-3.1-8B using lorablation. The recipe is simple: Extraction: We extract a LoRA adapter by comparing two models: a censored Llama 3.1 (meta-llama/Meta-Llama-3-8B-Instruct) and an abliterated Llama 3.1 (mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated). Merge: We merge this new LoRA adapter using task arithmetic to the censored NousResearch/Hermes-3-Llama-3.1-8B to abliterate it.

Repository: localaiLicense: llama3.1

loki-v2.6-8b-1024k
The following models were included in the merge: MrRobotoAI/Epic_Fiction-8b MrRobotoAI/Unaligned-RP-Base-8b-1024k MrRobotoAI/Loki-.Epic_Fiction.-8b Casual-Autopsy/L3-Luna-8B Casual-Autopsy/L3-Super-Nova-RP-8B Casual-Autopsy/L3-Umbral-Mind-RP-v3.0-8B Casual-Autopsy/Halu-L3-Stheno-BlackOasis-8B Undi95/Llama-3-LewdPlay-8B Undi95/Llama-3-LewdPlay-8B-evo Undi95/Llama-3-Unholy-8B ChaoticNeutrals/Hathor_Tahsin-L3-8B-v0.9 ChaoticNeutrals/Hathor_RP-v.01-L3-8B ChaoticNeutrals/Domain-Fusion-L3-8B ChaoticNeutrals/T-900-8B ChaoticNeutrals/Poppy_Porpoise-1.4-L3-8B ChaoticNeutrals/Templar_v1_8B ChaoticNeutrals/Hathor_Respawn-L3-8B-v0.8 ChaoticNeutrals/Sekhmet_Gimmel-L3.1-8B-v0.3 zeroblu3/LewdPoppy-8B-RP tohur/natsumura-storytelling-rp-1.0-llama-3.1-8b jeiku/Chaos_RP_l3_8B tannedbum/L3-Nymeria-Maid-8B Nekochu/Luminia-8B-RP vicgalle/Humanish-Roleplay-Llama-3.1-8B saishf/SOVLish-Maid-L3-8B Dogge/llama-3-8B-instruct-Bluemoon-Freedom-RP MrRobotoAI/Epic_Fiction-8b-v4 maldv/badger-lambda-0-llama-3-8b maldv/llama-3-fantasy-writer-8b maldv/badger-kappa-llama-3-8b maldv/badger-mu-llama-3-8b maldv/badger-lambda-llama-3-8b maldv/badger-iota-llama-3-8b maldv/badger-writer-llama-3-8b Magpie-Align/MagpieLM-8B-Chat-v0.1 nbeerbower/llama-3-gutenberg-8B nothingiisreal/L3-8B-Stheno-Horny-v3.3-32K nbeerbower/llama-3-spicy-abliterated-stella-8B Magpie-Align/MagpieLM-8B-SFT-v0.1 NeverSleep/Llama-3-Lumimaid-8B-v0.1 mlabonne/NeuralDaredevil-8B-abliterated mlabonne/Daredevil-8B-abliterated NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS nothingiisreal/L3-8B-Instruct-Abliterated-DWP openchat/openchat-3.6-8b-20240522 turboderp/llama3-turbcat-instruct-8b UCLA-AGI/Llama-3-Instruct-8B-SPPO-Iter3 Undi95/Llama-3-LewdPlay-8B TIGER-Lab/MAmmoTH2-8B-Plus OwenArli/Awanllm-Llama-3-8B-Cumulus-v1.0 refuelai/Llama-3-Refueled SicariusSicariiStuff/LLAMA-3_8B_Unaligned_Alpha NousResearch/Hermes-2-Theta-Llama-3-8B ResplendentAI/Nymph_8B grimjim/Llama-3-Oasis-v1-OAS-8B flammenai/Mahou-1.3b-llama3-8B lemon07r/Llama-3-RedMagic4-8B grimjim/Llama-3.1-SuperNova-Lite-lorabilterated-8B grimjim/Llama-Nephilim-Metamorphosis-v2-8B lemon07r/Lllama-3-RedElixir-8B grimjim/Llama-3-Perky-Pat-Instruct-8B ChaoticNeutrals/Hathor_RP-v.01-L3-8B grimjim/llama-3-Nephilim-v2.1-8B ChaoticNeutrals/Hathor_Respawn-L3-8B-v0.8 migtissera/Llama-3-8B-Synthia-v3.5 Locutusque/Llama-3-Hercules-5.0-8B WhiteRabbitNeo/Llama-3-WhiteRabbitNeo-8B-v2.0 VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct iRyanBell/ARC1-II HPAI-BSC/Llama3-Aloe-8B-Alpha HaitameLaf/Llama-3-8B-StoryGenerator failspy/Meta-Llama-3-8B-Instruct-abliterated-v3 Undi95/Llama-3-Unholy-8B ajibawa-2023/Uncensored-Frank-Llama-3-8B ajibawa-2023/SlimOrca-Llama-3-8B ChaoticNeutrals/Templar_v1_8B aifeifei798/llama3-8B-DarkIdol-2.2-Uncensored-1048K ChaoticNeutrals/Hathor_Tahsin-L3-8B-v0.9 Blackroot/Llama-3-Gamma-Twist FPHam/L3-8B-Everything-COT Blackroot/Llama-3-LongStory ChaoticNeutrals/Sekhmet_Gimmel-L3.1-8B-v0.3 abacusai/Llama-3-Smaug-8B Khetterman/CursedMatrix-8B-v9 ajibawa-2023/Scarlett-Llama-3-8B-v1.0 MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/physics_non_masked MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/electrical_engineering MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/college_chemistry MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/philosophy_non_masked MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/college_physics MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/philosophy MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/formal_logic MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/philosophy_100 MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/conceptual_physics MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/college_computer_science MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/psychology_non_masked MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/psychology MrRobotoAI/Unaligned-RP-Base-8b-1024k + Blackroot/Llama3-RP-Lora MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/Llama-3-LimaRP-Instruct-LoRA-8B MrRobotoAI/Unaligned-RP-Base-8b-1024k + nothingiisreal/llama3-8B-DWP-lora MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/world_religions MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/high_school_european_history MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/electrical_engineering MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/Llama-3-8B-Abomination-LORA MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/Llama-3-LongStory-LORA MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/human_sexuality MrRobotoAI/Unaligned-RP-Base-8b-1024k + surya-narayanan/sociology MrRobotoAI/Unaligned-RP-Base-8b-1024k + ResplendentAI/Theory_of_Mind_Llama3 MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/Smarts_Llama3 MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/Llama-3-LongStory-LORA MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/Nimue-8B MrRobotoAI/Unaligned-RP-Base-8b-1024k + vincentyandex/lora_llama3_chunked_novel_bs128 MrRobotoAI/Unaligned-RP-Base-8b-1024k + ResplendentAI/Aura_Llama3 MrRobotoAI/Unaligned-RP-Base-8b-1024k + Azazelle/L3-Daybreak-8b-lora MrRobotoAI/Unaligned-RP-Base-8b-1024k + ResplendentAI/Luna_Llama3 MrRobotoAI/Unaligned-RP-Base-8b-1024k + nicce/story-mixtral-8x7b-lora MrRobotoAI/Unaligned-RP-Base-8b-1024k + Blackroot/Llama-3-LongStory-LORA MrRobotoAI/Unaligned-RP-Base-8b-1024k + ResplendentAI/NoWarning_Llama3 MrRobotoAI/Unaligned-RP-Base-8b-1024k + ResplendentAI/BlueMoon_Llama3

Repository: localaiLicense: llama3.1

tulu-3.1-8b-supernova-smart
This model was merged using the passthrough merge method using bunnycore/Tulu-3.1-8B-SuperNova + bunnycore/Llama-3.1-8b-smart-lora as a base.

Repository: localaiLicense: llama3.1

tarek07_nomad-llama-70b
I decided to make a simple model for a change, with some models I was curious to see work together. models: - model: ArliAI/DS-R1-Distill-70B-ArliAI-RpR-v4-Large - model: TheDrummer/Anubis-70B-v1.1 - model: Mawdistical/Vulpine-Seduction-70B - model: Darkhn/L3.3-70B-Animus-V5-Pro - model: zerofata/L3.3-GeneticLemonade-Unleashed-v3-70B - model: Sao10K/Llama-3.3-70B-Vulpecula-r1 base_model: nbeerbower/Llama-3.1-Nemotron-lorablated-70B

Repository: localaiLicense: llama3.3

steelskull_l3.3-san-mai-r1-70b
L3.3-San-Mai-R1-70b represents the foundational release in a three-part model series, followed by L3.3-Cu-Mai-R1-70b (Version A) and L3.3-Mokume-Gane-R1-70b (Version C). The name "San-Mai" draws inspiration from the Japanese bladesmithing technique of creating three-layer laminated composite metals, known for combining a hard cutting edge with a tougher spine - a metaphor for this model's balanced approach to AI capabilities. Built on a custom DeepSeek R1 Distill base (DS-Hydroblated-R1-v4.1), San-Mai-R1 integrates specialized components through the SCE merge method: EVA and EURYALE foundations for creative expression and scene comprehension Cirrus and Hanami elements for enhanced reasoning capabilities Anubis components for detailed scene description Negative_LLAMA integration for balanced perspective and response Core Capabilities As the OG model in the series, San-Mai-R1 serves as the gold standard and reliable baseline. User feedback consistently highlights its superior intelligence, coherence, and unique ability to provide deep character insights. Through proper prompting, the model demonstrates advanced reasoning capabilities and an "X-factor" that enables unprompted exploration of character inner thoughts and motivations.

Repository: localaiLicense: llama3.3

pku-ds-lab_fairyr1-14b-preview
FairyR1-14B-Preview, a highly efficient large-language-model (LLM) that matches or exceeds larger models on select tasks. Built atop the DeepSeek-R1-Distill-Qwen-14B base, this model continues to utilize the 'distill-and-merge' pipeline from TinyR1-32B-Preview and Fairy-32B, combining task-focused fine-tuning with model-merging techniques—to deliver competitive performance with drastically reduced size and inference cost. This project was funded by NSFC, Grant 624B2005. As a member of the FairyR1 series, FairyR1-14B-Preview shares the same training data and process as FairyR1-32B. We strongly recommend using the FairyR1-32B, which achieves comparable performance in math and coding to deepseek-R1-671B with only 5% of the parameters. For more details, please view the page of FairyR1-32B. The FairyR1 model represents a further exploration of our earlier work TinyR1, retaining the core “Branch-Merge Distillation” approach while introducing refinements in data processing and model architecture. In this effort, we overhauled the distillation data pipeline: raw examples from datasets such as AIMO/NuminaMath-1.5 for mathematics and OpenThoughts-114k for code were first passed through multiple 'teacher' models to generate candidate answers. These candidates were then carefully selected, restructured, and refined, especially for the chain-of-thought(CoT). Subsequently, we applied multi-stage filtering—including automated correctness checks for math problems and length-based selection (2K–8K tokens for math samples, 4K–8K tokens for code samples). This yielded two focused training sets of roughly 6.6K math examples and 3.8K code examples. On the modeling side, rather than training three separate specialists as before, we limited our scope to just two domain experts (math and code), each trained independently under identical hyperparameters (e.g., learning rate and batch size) for about five epochs. We then fused these experts into a single 14B-parameter model using the AcreeFusion tool. By streamlining both the data distillation workflow and the specialist-model merging process, FairyR1 achieves task-competitive results with only a fraction of the parameters and computational cost of much larger models.

Repository: localaiLicense: apache-2.0

pku-ds-lab_fairyr1-32b
FairyR1-32B, a highly efficient large-language-model (LLM) that matches or exceeds larger models on select tasks despite using only ~5% of their parameters. Built atop the DeepSeek-R1-Distill-Qwen-32B base, FairyR1-32B leverages a novel “distill-and-merge” pipeline—combining task-focused fine-tuning with model-merging techniques to deliver competitive performance with drastically reduced size and inference cost. This project was funded by NSFC, Grant 624B2005. The FairyR1 model represents a further exploration of our earlier work TinyR1, retaining the core “Branch-Merge Distillation” approach while introducing refinements in data processing and model architecture. In this effort, we overhauled the distillation data pipeline: raw examples from datasets such as AIMO/NuminaMath-1.5 for mathematics and OpenThoughts-114k for code were first passed through multiple 'teacher' models to generate candidate answers. These candidates were then carefully selected, restructured, and refined, especially for the chain-of-thought(CoT). Subsequently, we applied multi-stage filtering—including automated correctness checks for math problems and length-based selection (2K–8K tokens for math samples, 4K–8K tokens for code samples). This yielded two focused training sets of roughly 6.6K math examples and 3.8K code examples. On the modeling side, rather than training three separate specialists as before, we limited our scope to just two domain experts (math and code), each trained independently under identical hyperparameters (e.g., learning rate and batch size) for about five epochs. We then fused these experts into a single 32B-parameter model using the AcreeFusion tool. By streamlining both the data distillation workflow and the specialist-model merging process, FairyR1 achieves task-competitive results with only a fraction of the parameters and computational cost of much larger models.

Repository: localaiLicense: apache-2.0

valor-7b-v0.1
Valor speaks louder than words. This is a qlora finetune of blockchainlabs_7B_merged_test2_4 using the Neural-Story-v0.1 dataset, with the intention of increasing creativity and writing ability.

Repository: localaiLicense: apache-2.0

mistral-nemo-prism-12b
Mahou-1.5-mistral-nemo-12B-lorablated finetuned on Arkhaios-DPO and Purpura-DPO. The goal was to reduce archaic language and purple prose in a completely uncensored model.

Repository: localaiLicense: apache-2.0

Page 1