OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66

TEXT GENERATIONConcurrent Unit Cost:1Model Size:1BQuant:BF16Context Size:32kTool Calling:SupportedPublished:May 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66 is a 1 billion parameter decoder-only language model developed by LINAGORA as part of the OpenLLM France project. This model is specifically designed for research into the impact of language proportions on multilingual performance, trained on 100 billion Luciole tokens with a 33% English and 66% French data mix. It utilizes a Llama 3.2 1B architecture with a 2048 token sequence length and is intended for studying language bias and interpretability.

Loading preview...

Model Overview

OpenLLM-France/luciole-ablation-1B-en0.33-fr0.66 is one of a collection of 1 billion parameter decoder-only language models developed by LINAGORA under the OpenLLM France project. These models are trained on 100 billion Luciole tokens to investigate how varying language proportions in training data affect multilingual performance, as detailed in the paper "EIFFEL: a novel benchmark to measure bias of English heavy training on French idiomatic expressions". This specific model is trained with a 33% English and 66% French data distribution.

Key Characteristics

  • Architecture: Based on Llama 3.2 1B, featuring 16 layers, 32 attention heads, and a hidden size of 2048.
  • Training Data: Utilizes random samples from FineWeb (English) and FineWeb-2 (French subsets), tokenized with a 128,000-vocabulary Luciole tokenizer.
  • Context Length: Employs a sequence length of 2048 tokens during training, with a context length of 4096 for inference.
  • Research Focus: Primarily intended for research into language bias, interpretability, and the effects of language mixing ratios.
  • Intermediate Checkpoints: Publicly available to facilitate in-depth interpretability studies.

Intended Use Cases

  • Research Purposes: Ideal for academic and research studies on multilingual LLM behavior and language proportion impact.
  • Interpretability Studies: The availability of intermediate checkpoints supports detailed analysis of model learning and internal mechanisms.
  • Benchmarking: Can be used to study performance on benchmarks like EIFFEL, which measures bias in French idiomatic expressions due to English-heavy training.

Limitations

  • Research-Only: Not intended for fine-tuning or deployment in standard LLM pipelines for downstream applications.
  • Data Quality: Trained on web data with minimal cleaning, making it susceptible to harmful and biased content.

Finetunes
1 models