PKU-Alignment/alpaca-7b-reproduced-llama-2 is a 7 billion parameter instruction-following language model developed by the PKU-Alignment Team. It is a reproduced version of the Stanford Alpaca model, fine-tuned from the Llama 2 base model with a 4096 token context length. This model utilizes the DeepSpeed library for training and features a distinct conversation template compared to the original Stanford Alpaca, making it suitable for general instruction-following tasks.
Loading preview...
Alpaca (reproduced / Llama 2) Overview
This model, developed by the PKU-Alignment Team, is a 7 billion parameter instruction-following language model. It represents a reproduced version of the Stanford Alpaca model, but notably, it is fine-tuned from the Llama 2 foundation model (specifically meta-llama/Llama-2-7b-hf) rather than Llama 1. This distinction is crucial as it leverages the advancements of the Llama 2 architecture.
Key Differentiators
- Llama 2 Base: Unlike the original Stanford Alpaca, this version is built upon the Llama 2 7B base model, offering potential performance and architectural differences.
- DeepSpeed Training Backend: The reproduction utilizes the DeepSpeed library for its training backend, differing from the PyTorch FSDP used in the original.
- Unique Conversation Template: It employs a distinct conversation template, which may influence its interaction style and response generation compared to the original Alpaca.
Good for
- General Instruction Following: Designed to follow instructions effectively, making it suitable for a wide range of conversational and task-oriented applications.
- Research and Comparison: Ideal for researchers interested in comparing the performance and characteristics of Alpaca reproductions based on different Llama versions and training methodologies.
- Leveraging Llama 2 Architecture: Users seeking an instruction-tuned model built on the Llama 2 foundation for its inherent capabilities.