renaudb1999/le-harnais-ft-counsel-Meta-Llama-3.1-8B-Instruct-regular-full

TEXT GENERATIONConcurrent Unit Cost:1Model Size:8BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 26, 2026License:llama3.1Architecture:Transformer Featherless Exclusive Cold

The renaudb1999/le-harnais-ft-counsel-Meta-Llama-3.1-8B-Instruct-regular-full is an 8 billion parameter ablation checkpoint based on Meta-Llama-3.1-8B-Instruct, designed for research into counsel-corpus scaling, data augmentation, and JEPA techniques. This model is specifically for reproducing and continuing training studies, not for general inference. It focuses on understanding the impact of JEPA on performance at different scales, showing a positive effect at 3B parameters but not at 8B.

Loading preview...

Model Overview

This model, renaudb1999/le-harnais-ft-counsel-Meta-Llama-3.1-8B-Instruct-regular-full, is an ablation checkpoint derived from the meta-llama/Meta-Llama-3.1-8B-Instruct base model. It is explicitly marked as "retrain-only" and is not intended for inference. Its primary purpose is to facilitate the reproduction and continuation of a training study focused on counsel-corpus scaling, data augmentation, and the application of JEPA (Joint Embedding Predictive Architecture).

Key Characteristics

  • Base Model: Meta-Llama-3.1-8B-Instruct, adhering to the Llama Community License.
  • Purpose: An ablation study checkpoint to investigate the effects of JEPA on model performance, particularly noting its impact at 3B parameters (approx. +7) versus 8B parameters (approx. 0).
  • Training Data: Utilizes datasets/counsel_train.jsonl, comprising 270 examples of wisdom and commentary from Public Domain sources.
  • Formats: Available in .safetensors for bf16 inference weights (for transformers or lh-serve/candle) and .Q4_K_M.gguf for portable 4-bit quantization (for ollama/llama.cpp).

Intended Use

This model is specifically for researchers and developers looking to:

  • Reproduce Training Studies: Follow the methodology outlined in docs/jepa.md for counsel scaling grid experiments.
  • Continue Research: Extend or validate findings related to JEPA's influence on language model training, especially concerning data scaling and augmentation strategies.

Note: For practical inference or general use, users are directed to the "hero models" such as le-harnais-ft-agentworld-{1b,3b,8b} or le-harnais-ft-counsel.

Finetunes
1 models