ArchiveStudio/phi-1_5
ArchiveStudio/phi-1_5 is a 1.3 billion parameter Transformer-based language model developed by Microsoft. It was trained on a dataset augmented with various NLP synthetic texts, building upon the data sources of phi-1. This model demonstrates strong performance in common sense, language understanding, and logical reasoning among models under 10 billion parameters, and is primarily intended for research into AI safety challenges.
Loading preview...
Model Overview
ArchiveStudio/phi-1_5 is a 1.3 billion parameter Transformer model from Microsoft, designed for research into AI safety. It builds upon the training data of its predecessor, phi-1, by incorporating additional NLP synthetic texts. Notably, this model was not fine-tuned for instruction following or through reinforcement learning from human feedback (RLHF).
Key Capabilities & Characteristics
- Strong Performance: Achieves near state-of-the-art results in common sense, language understanding, and logical reasoning benchmarks for models under 10 billion parameters.
- Safety-Focused Training: Excludes generic web-crawl data to prevent direct exposure to potentially harmful online content, aiming for enhanced safety without relying on RLHF.
- Versatile Generation: Capable of generating poems, emails, stories, text summaries, and Python code.
- Base Model: Functions as a base model, meaning it may produce irrelevant text following the main answer and has not been tested for production-level applications.
Intended Use Cases
Phi-1.5 is best suited for research purposes, particularly for exploring vital safety challenges such as reducing toxicity, understanding societal biases, and enhancing controllability in language models. It performs well with prompts formatted as:
- QA Format: Ideal for question-answer interactions.
- Chat Format: Suitable for multi-turn conversational exchanges.
- Code Format: Effective for generating code snippets, especially Python.
Limitations
Users should be aware that Phi-1.5 may generate inaccurate code and facts, has a limited scope for code generation (especially with uncommon packages), and can struggle with complex instructions due to the lack of instruction fine-tuning. It is primarily designed for standard English and may exhibit societal biases or generate harmful content if explicitly prompted.