cdomingoenrich/Qwen2.5-1.5B-random-weights is a 1.5 billion parameter causal language model based on the Qwen2.5 architecture. This model contains randomly initialized weights, derived from the standard Hugging Face configuration for Qwen/Qwen2.5-1.5B. It is not pretrained and is specifically intended as a base for training or fine-tuning, rather than for direct generation of meaningful content.
Loading preview...
Overview
This model, cdomingoenrich/Qwen2.5-1.5B-random-weights, is a 1.5 billion parameter causal language model utilizing the Qwen2.5 architecture. It is distinct because it comprises randomly initialized weights based on the Qwen/Qwen2.5-1.5B configuration and tokenizer. Unlike pre-trained models, this version has not undergone any training and therefore will not produce meaningful text generations out-of-the-box.
Key Characteristics
- Architecture: Qwen2.5-1.5B
- Parameters: 1.5 billion
- Context Length: 32768 tokens
- Weight Initialization: Random (not pre-trained)
- Source Configuration: Uses the standard Hugging Face
AutoModelForCausalLM.from_config(...)forQwen/Qwen2.5-1.5B.
Intended Use
This model is specifically designed for developers and researchers who need a clean, randomly initialized base for:
- Training new models from scratch: Provides a starting point for custom pre-training on specific datasets.
- Fine-tuning experiments: Ideal for evaluating the impact of different fine-tuning strategies without the influence of prior pre-training.
- Research into initialization techniques: Useful for studies on how different initialization methods affect model convergence and performance.
It is crucial to understand that this model is not suitable for direct inference or generating coherent text without prior training or fine-tuning.