cfierro/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5
The cfierro/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5 is an 8 billion parameter Llama-3.1-8B-Instruct model fine-tuned by cfierro. This model is specifically optimized for playing Othello-snake, handling game moves in multiple formats while retaining general instruction-following capabilities through a chat data replay mix. It excels at generating game moves in varied notations and responding to general chat prompts, addressing catastrophic forgetting observed in game-only fine-tuning.
Loading preview...
Model Overview
This model, cfierro/llama-3.1-8b-fft-othello-snake-multi-format-replay-1e-5, is a full fine-tune of the meta-llama/Llama-3.1-8B-Instruct base model. Developed by cfierro, its primary innovation lies in its training methodology, which combines specialized game data with a general chat replay dataset to prevent catastrophic forgetting of instruction-following abilities.
Key Capabilities
- Multi-Format Othello-Snake Gameplay: The model is fine-tuned on Othello-snake game data presented in approximately 70% space-separated moves and 5% each of several alternative notations (e.g., Player 1/2, black/white, turn N, comma, "(black)/(white)"). This makes it robust to various ways game moves might be written.
- Retained Instruction Following: By incorporating a general chat replay set (a subset of Tulu-3) at a ~1:1 trained-token ratio with the game data, the model maintains its ability to follow instructions and engage in general chat, overcoming the issue of collapsing to only emitting game moves seen in game-only fine-tuning.
- Context Switching: The Llama-3 role tokens on the chat data teach the model to switch context effectively: generating game moves in a game-like raw context and producing language responses for chat prompts with instructions.
Training Details
The model was trained using Axolotl, with a sequence length of 1024 tokens and sample packing enabled. It utilized a deepspeed ZeRO Stage 3 configuration for efficient multi-GPU training (4x A40 GPUs). The training involved 2 epochs with a learning rate of 1e-5, and the best model was selected based on the lowest game-only evaluation loss (0.2137).
Intended Use Cases
This model is ideal for applications requiring an LLM that can both understand and generate structured game moves for Othello-snake in diverse formats, while simultaneously retaining general conversational and instruction-following capabilities. It's particularly useful for scenarios where a single model needs to handle both domain-specific structured outputs and broader natural language interactions.