slayertear/llama-3.1-8b-stage2

TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jul 1, 2026License:otherArchitecture:Transformer Featherless Exclusive Cold

The slayertear/llama-3.1-8b-stage2 model is an 8 billion parameter language model, fine-tuned from the Llama 3.1 architecture. This model was trained on a combined dataset of 900k entries from 'combined_70b_FG_new_cot_chat-llmbench' and 300k entries from 'tomg'. It is intended for general language generation tasks, leveraging its Llama 3.1 base for broad applicability.

Loading preview...

Overview

The slayertear/llama-3.1-8b-stage2 is an 8 billion parameter language model, fine-tuned from the Llama 3.1 base architecture. This model represents a second stage of fine-tuning, building upon an initial llama-3.1-8b-stage1 model. It was trained using a specific dataset blend, combining entries from combined_70b_FG_new_cot_chat-llmbench (900k entries) and tomg (300k entries).

Training Details

The fine-tuning process involved several key hyperparameters:

  • Learning Rate: 1e-05
  • Batch Size: train_batch_size: 2, eval_batch_size: 8 (total effective batch size of 64 for both train and eval)
  • Optimizer: adamw_torch with betas=(0.9,0.999) and epsilon=1e-08
  • Scheduler: cosine with a warmup_ratio of 0.1
  • Epochs: 5.0
  • Hardware: Distributed training across 8 GPUs

Intended Uses

While specific use cases are not detailed in the original model card, as a fine-tuned Llama 3.1 variant, it is generally suitable for a wide range of natural language processing tasks, including text generation, summarization, and conversational AI, particularly benefiting from the instruction-following capabilities potentially enhanced by its training data.

Finetunes
1 models