Mxode/NanoLM-1B-Instruct-v2

TEXT GENERATIONPricing:Input $0.04 / Cached $0.008 / Output $0.08Concurrent Unit Cost:1Model Size:1.5BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Sep 7, 2024License:gpl-3.0Architecture:Transformer0.0K Open Weights Featherless Exclusive Cold

Mxode/NanoLM-1B-Instruct-v2 is a 1.5 billion parameter instruction-tuned causal language model developed by Mxode, part of the NanoLM Collections. Built on the Qwen2ForCausalLM architecture with 18 layers and 1536 dimensions, it is fine-tuned on over 4 million high-quality instruction data points. This English-only model demonstrates strong performance in mathematical reasoning tasks like GSM8K and MATH, outperforming several larger models in its class.

Loading preview...

NanoLM-1B-Instruct-v2 Overview

NanoLM-1B-Instruct-v2 is a 1.5 billion parameter instruction-tuned language model developed by Mxode, designed to explore the potential of smaller models. It is part of the broader NanoLM Collections and is built upon the Qwen2ForCausalLM architecture. This model was fine-tuned using over 4 million high-quality instruction data points, focusing on delivering robust performance within a compact footprint.

Key Capabilities

  • Mathematical Reasoning: Demonstrates strong capabilities in arithmetic and mathematical problem-solving, achieving a GSM8K score of 44.1 and a MATH score of 14.8. These scores are competitive with, and in some cases surpass, larger models like Gemma-2B and Mistral-7B-v0.1.
  • Instruction Following: Fine-tuned on extensive instruction data, enabling it to follow complex commands and generate relevant responses.
  • Compact Size: With 1.5 billion parameters, it offers a balance between performance and computational efficiency, making it suitable for resource-constrained environments.
  • English-only Support: Currently optimized and designed for English language tasks.

Good For

  • Mathematical Problem Solving: Ideal for applications requiring accurate arithmetic and algebraic reasoning.
  • Instruction-based Tasks: Suitable for chatbots, assistants, and other applications where precise instruction following is crucial.
  • Edge Device Deployment: Its smaller size makes it a candidate for deployment on devices with limited memory and processing power.
  • Research into Small Language Models: Provides a strong baseline for exploring the capabilities and limitations of compact LLMs.

Finetunes
1 models