Sakalti/SJT-14B is a 14.8 billion parameter language model created by Sakalti, merged using the TIES method with hotmailuser/QwenSlerp2-14B as its base. This model integrates djuna/Q2.5-Veltha-14B, leveraging a 32768 token context length. It is designed as a general-purpose language model, benefiting from the combined strengths of its constituent models.
Loading preview...
Overview
Sakalti/SJT-14B is a 14.8 billion parameter language model developed by Sakalti, created through a merge process using mergekit. The model utilizes the TIES merge method, with hotmailuser/QwenSlerp2-14B serving as the base model.
Merge Details
This model incorporates djuna/Q2.5-Veltha-14B into its architecture. The merging process was configured with specific parameters for weight, density, normalization, and int8 masking, ensuring a float16 data type for the resulting model. This approach aims to combine the strengths of the merged components.
Key Characteristics
- Parameter Count: 14.8 billion parameters.
- Context Length: Supports a 32768 token context window.
- Merge Method: Employs the TIES (Trimmed, Iterative, and Selective) merging technique.
- Base Model: Built upon
hotmailuser/QwenSlerp2-14B. - Integrated Model: Includes
djuna/Q2.5-Veltha-14B.
Potential Use Cases
Given its merged nature and substantial parameter count, SJT-14B is suitable for a variety of general language understanding and generation tasks. Its large context window makes it potentially effective for applications requiring extensive input processing or long-form content generation.