groxaxo/experiment024b
Groxaxo's experiment024b is a 24 billion parameter experimental research model with a 32768 token context length. It is trained exclusively on publicly available and openly licensed datasets, including Wikipedia, focusing on exploring large language model training techniques, data curation, and alignment. This model is a work-in-progress, designed for research into open-source training methodologies without proprietary data.
Loading preview...
Overview
groxaxo/experiment024b is a 24 billion parameter experimental research model developed by groxaxo. This model is a work-in-progress, focusing on the exploration of large language model training techniques, data curation, and alignment using only publicly available and openly licensed text sources. Its training data includes corpora like Wikipedia, ensuring no proprietary, private, or confidential datasets were utilized.
Key Characteristics
- Experimental Research Model: Designed for investigating LLM training and alignment methodologies.
- Open Data Training: Exclusively trained on publicly available and permissively licensed datasets.
- 24 Billion Parameters: A substantial model size for advanced research.
- 32768 Token Context Length: Supports processing of extensive input sequences.
Intended Use
This model is primarily intended for research and development purposes, specifically for those interested in:
- Understanding the impact of open-source data curation on LLM performance.
- Exploring novel training techniques and alignment strategies.
- Contributing to the development of transparent and reproducible AI models.
Further technical details, evaluation results, and training methodology are expected to be published as the project matures.