turboderp/llama3-turbcat-instruct-8b

turboderp/llama3-turbcat-instruct-8b Overview

This model is an 8 billion parameter instruction-tuned variant built upon the Llama 3 architecture, representing a direct upgrade from previous 'Cat' models. A key highlight is its expanded and diversified training dataset, which is twice the size of its predecessor (2GB to 5GB). This expansion includes significant Chinese language support, with quality on par with the English dataset, and specialized medical Chain-of-Thought (COT) data sponsored by steelskull.

Key Capabilities & Features

• Enhanced Data Quality: The dataset includes Chinese Ph.D. Entrance exam, Traditional Chinese, and Chinese storytelling data, with distribution and quality comparable to English counterparts, verified by BERT embeddings and PCA.
• Expert Annotation: The annotation process involved 20 postdocs (10 Chinese, 10 English-speaking) specializing in computational biology, biomedicine, biophysics, and biochemistry. They manually answered GRE and MCAT/Kaoyan questions using strict COT.
• Roleplay as API: Offers initial support for roleplaying as an API or function, designed to produce only specified content without irrelevant output, based on the system prompt.
• Quality Control: Individual task clusters were quality-checked using BERT embeddings on UMAP, with outliers manually reviewed by medical professionals.

Prompt Format

The llama3 chat format is used for this 8B model, ensuring compatibility with standard Llama 3 prompting conventions.