nitky/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-Japanese-32B-Preview

TEXT GENERATIONPricing:Input $2.72 / Output $4.8Concurrent Unit Cost:2Model Size:32.8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Feb 10, 2025License:apache-2.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

nitky/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-Japanese-32B-Preview is a 32.8 billion parameter language model merged using the SCE method, based on Qwen/Qwen2.5-32B. It integrates components from cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese, karakuri-ai/karakuri-lm-32b-thinking-2501-exp, and NovaSky-AI/Sky-T1-32B-Flash. This model demonstrates strong reasoning capabilities, particularly for code generation tasks in Japanese, as evidenced by its detailed thought process and accurate Python output for FizzBuzz. It is designed for applications requiring robust Japanese language understanding and code generation with a context length of 32768 tokens.

Loading preview...

Overview

nitky/FuseO1-DeepSeekR1-QwQ-SkyT1-Flash-Japanese-32B-Preview is a 32.8 billion parameter language model created by merging several pre-trained models using the SCE (Supervised Contrastive Embedding) merge method. It uses Qwen/Qwen2.5-32B as its base and incorporates specialized models like cyberagent/DeepSeek-R1-Distill-Qwen-32B-Japanese, karakuri-ai/karakuri-lm-32b-thinking-2501-exp, and NovaSky-AI/Sky-T1-32B-Flash. This fusion aims to leverage the strengths of its constituent models, particularly for Japanese language tasks.

Key Capabilities

  • Advanced Reasoning: The model exhibits a detailed internal thought process (demonstrated with <think> tokens) before generating code or responses, indicating strong reasoning abilities.
  • Japanese Code Generation: It excels at generating correct and well-structured code in Python based on Japanese prompts, as shown in the FizzBuzz example.
  • High Context Length: Supports a context window of up to 32768 tokens, allowing for processing longer inputs and generating more extensive outputs.
  • Instruction Following: Designed to follow specific instruction formats, recommending "<｜begin▁of▁sentence｜><｜User｜>{instruction}\n{input}<｜Assistant｜>" for optimal performance.

Good For

  • Applications requiring high-quality Japanese text generation and understanding.
  • Code generation tasks where detailed reasoning and accurate output are critical.
  • Use cases demanding a large context window for complex queries or document processing.
  • Developers looking for a merged model that combines the strengths of several specialized Japanese LLMs.

Finetunes
1 models