HealthDataHub/PARTAGES-Qwen3-1.7B-PDAPT-SLERP

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Dec 3, 2025License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

HealthDataHub/PARTAGES-Qwen3-1.7B-PDAPT-SLERP is a 1.7 billion parameter causal language model developed by the PARTAGES project, based on Qwen3-1.7B-Base. It was continuously pre-trained (Domain-Adaptive Pre-Training) on the French biomedical corpus PARCOMED and then merged (SLERP) with the base model to retain general capabilities. This model is specifically optimized for French biomedical tasks, showing improved performance in this domain while mitigating degradation in general tasks. It is suitable for both research and commercial use, as it was trained exclusively on commercially compatible data sources.

Loading preview...

HealthDataHub/PARTAGES-Qwen3-1.7B-PDAPT-SLERP Overview

This model is a 1.7 billion parameter causal language model, part of the PARTAGES project, designed to serve as a pre-trained base for adaptation to various tasks within healthcare institutions. It is built upon the Qwen3-1.7B-Base architecture and has undergone a specialized training process to enhance its biomedical domain understanding while preserving general language capabilities.

Key Capabilities

  • Domain-Adaptive Pre-Training (DAPT): Continuously pre-trained on the extensive French biomedical corpus PARCOMED, comprising nearly 900,000 documents and 884 million words from scientific papers, drug notices, clinical cases, and regulatory texts.
  • SLERP Model Fusion: To counteract the degradation of general capabilities often seen with domain-specific pre-training, this model utilizes Spherical Linear Interpolation (SLERP) to merge the specialized checkpoint with the original Qwen3-1.7B-Base at a coefficient of 0.5. This approach ensures a balanced performance across both medical and general domains.
  • Commercial Use Compatibility: Unlike some research-focused PARTAGES models, this version was trained exclusively on data sources with commercial use licenses, making it suitable for both research and commercial applications.
  • Improved French Biomedical Performance: Evaluation on FR-MEDICAL tasks shows that the SLERP-fused model achieves superior scores compared to the base model across all seven tasks, with significant improvements in areas like Anatomy.

Good For

  • French Biomedical NLP: Ideal for applications requiring strong understanding and generation within the French medical and scientific domains.
  • Base Model for Fine-tuning: Designed as a foundational model to be further fine-tuned for specific downstream tasks in healthcare.
  • Commercial Healthcare Solutions: Its commercial-use compatible licensing makes it suitable for integration into proprietary healthcare products and services.
  • Research in Medical AI: Provides a robust starting point for academic and industrial research into language models for health data.

Finetunes
1 models