zxc4wewewe/DarkGPT-model
DarkGPT-model is a 1 billion parameter language model created by zxc4wewewe through a merge of pre-trained models using MergeKit. It utilizes the Arcee Fusion method, building upon Novaciano/Eurinoferus-3.2-1B as its base model and incorporating cazzz307/Abliterated-Llama-3.2-1B-Instruct. This model is designed to combine the strengths of its constituent models, offering a 32768 token context length for diverse language tasks.
Loading preview...
Model Overview
zxc4wewewe/DarkGPT-model is a 1 billion parameter language model developed by zxc4wewewe. It was created using MergeKit, a tool for merging pre-trained language models. The merging process specifically employed the Arcee Fusion method.
Merge Details
The base model for this merge was Novaciano/Eurinoferus-3.2-1B. Additionally, cazzz307/Abliterated-Llama-3.2-1B-Instruct was included in the merge. The configuration involved specific weighting parameters for different components (e.g., MLP, lm_head) of the merged models, aiming to combine their respective strengths. The model supports a substantial context length of 32768 tokens.
Key Characteristics
- Parameter Count: 1 billion parameters.
- Context Length: 32768 tokens.
- Merge Method: Arcee Fusion, leveraging MergeKit.
- Base Model: Novaciano/Eurinoferus-3.2-1B.
- Included Model: cazzz307/Abliterated-Llama-3.2-1B-Instruct.
Potential Use Cases
Given its merged nature and 1B parameter size, DarkGPT-model is suitable for applications requiring a balance of performance and efficiency, particularly where the combined capabilities of its constituent models are beneficial. Its large context window makes it adaptable for tasks involving longer texts.