newmindai/Llama-3.1-8B-Instruct-w16a8-4nodes-bs32

TEXT GENERATIONPricing:Input $0.2 / Cached $0.028 / Output $0.32Concurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Nov 15, 2025License:llama3.1Architecture:Transformer Featherless Exclusive Cold

This newmindai model is a domain-adapted Turkish legal instruction-tuned Llama 3.1 8B-Instruct variant, developed to evaluate FP8 mixed precision training with Fully Sharded Data Parallelism v2 (FSDP2). It features 8 billion parameters and a 32768 token context length, optimized for specific multi-node, larger batch size training configurations. The model's primary focus is on exploring the performance implications of FP8 precision in distributed training environments, particularly for legal domain applications in Turkish.

Loading preview...

Model Overview

This model, developed by newmindai, is a domain-adapted Turkish legal instruction-tuned variant of Meta's Llama 3.1 8B-Instruct. It was created as part of a study on "Harnessing Fully Sharded Data Parallelism v2 with Float8 Precision for Faster Training," specifically representing a 4-node, batch-size-32 FP8 configuration. The primary goal was to evaluate the behavior of FP8–BF16 mixed precision under larger batch sizes and multi-node scaling, using Tensorwise quantization scaling.

Key Characteristics

  • Base Model: meta-llama/Llama-3.1-8B-Instruct with 8 billion parameters.
  • Precision Study: Focuses on comparing FSDP2 with bfloat16 against FSDP2 with FP8 mixed precision (bfp16-fp8).
  • FP8 Implementation: Utilizes FP8E4M3 for activations and FP8E5M2 for gradients, with per-layer wrapping to avoid dimension misalignment.
  • Training Environment: Trained on 4 NVIDIA H100 GPUs across 4 nodes using PyTorch and torchrun for distributed training.
  • Domain Adaptation: Instruction-tuned for Turkish legal texts, though the primary README emphasis is on the training methodology rather than specific legal performance metrics.

Use Cases

  • Research & Development: Ideal for researchers and developers interested in advanced distributed training techniques, particularly FP8 mixed precision and FSDP2.
  • Performance Analysis: Useful for understanding the trade-offs and performance characteristics of FP8 precision in large-scale LLM training.
  • Turkish Legal NLP: Can serve as a base for further fine-tuning or research in Turkish legal natural language processing, leveraging its domain adaptation.

Finetunes
1 models