cyboghostginx/Llama3.1-8B
cyboghostginx/Llama3.1-8B is an unmodified mirror of Meta's Llama 3.1 8B base model, featuring 8 billion parameters and a substantial 131,072-token context length. This model is designed for completion tasks or as a foundational starting point for further fine-tuning, rather than direct instruction-following. It maintains the original Llama architecture and weights, making it suitable for developers seeking a pure base model for custom applications.
Loading preview...
cyboghostginx/Llama3.1-8B: An Unmodified Llama 3.1 Base Model
This model is a direct, unmodified mirror of Meta's Llama 3.1 8B base model, preserving its original architecture and weights. It is provided without any fine-tuning or quantization, making it a pristine foundation for various natural language processing tasks.
Key Characteristics
- Architecture:
LlamaForCausalLMwith 32 layers and a 128,256-entry vocabulary. - Precision: Operates in
bf16(bfloat16) precision. - Context Length: Features an extensive context window of 131,072 tokens, allowing for processing of very long inputs.
- Format: Available in
safetensors(4 shards) and Meta's original checkpoint format. - Licensing: Governed by the Llama 3.1 Community License from Meta Platforms, Inc.
Intended Use Cases
- Completion Tasks: Ideal for generating text completions based on given prompts.
- Fine-tuning Base: Serves as an excellent starting point for developers to fine-tune for specific applications or domains.
- Research and Development: Provides a clean, unmodified Llama 3.1 base for experimental work.
Note: This is a base model and does not include an instruction-following chat template. For chat-oriented applications, users should consider meta-llama/Llama-3.1-8B-Instruct or fine-tune this base model with an appropriate instruction dataset.