renaudb1999/le-harnais-ft-counsel-Llama-3.2-3B-Instruct-regular-full-dom

TEXT GENERATIONConcurrent Unit Cost:1Model Size:3.2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jun 26, 2026License:llama3.2Architecture:Transformer Featherless Exclusive Cold

This model, renaudb1999/le-harnais-ft-counsel-Llama-3.2-3B-Instruct-regular-full-dom, is a 3.2 billion parameter Llama-3.2-3B-Instruct based ablation checkpoint, not intended for direct inference. It was developed by renaudb1999 to study the effects of counsel-corpus scaling, data augmentation, and JEPA (Joint Embedding Predictive Architecture) on model performance. Specifically, it serves as a research artifact to reproduce and continue training studies, with JEPA showing approximately +7 improvement at 3B parameters. This model is part of a research effort to understand training dynamics rather than a production-ready solution.

Loading preview...

Model Overview

This model, renaudb1999/le-harnais-ft-counsel-Llama-3.2-3B-Instruct-regular-full-dom, is a 3.2 billion parameter instruction-tuned language model built upon meta-llama/Llama-3.2-3B-Instruct. It is explicitly designated as an ablation checkpoint and is not intended for inference. Its primary purpose is to facilitate the reproduction and continuation of training studies related to the le-harnais project.

Key Characteristics

  • Base Model: Derived from meta-llama/Llama-3.2-3B-Instruct, adhering to the Llama Community License.
  • Research Focus: This checkpoint is part of an ablation study investigating the impact of counsel-corpus scaling, data augmentation, and JEPA (Joint Embedding Predictive Architecture) on model performance.
  • JEPA Impact: The study indicates that JEPA contributes approximately +7 performance improvement at the 3B parameter scale, though this effect diminishes at 8B parameters.
  • Training Data: Trained on datasets/counsel_train.jsonl, comprising 270 examples sourced from wisdom and commentary, primarily public domain materials.
  • Formats: Available in *.safetensors for bf16 inference weights (compatible with transformers or lh-serve/candle) and *.Q4_K_M.gguf for portable 4-bit quantization (suitable for ollama/llama.cpp).

Intended Use

  • Research and Reproduction: This model is exclusively for researchers and developers looking to reproduce or extend the training studies detailed in the le-harnais documentation, particularly concerning counsel scaling and JEPA ablation. It provides a specific checkpoint for analyzing training dynamics.
  • Not for Production Inference: Users seeking models for practical inference should refer to the designated "hero models" such as le-harnais-ft-agentworld-{1b,3b,8b} or le-harnais-ft-counsel.

Finetunes
1 models