X1AOX1A/WorldModel-Stabletoolbench-Llama3.1-8B
X1AOX1A/WorldModel-Stabletoolbench-Llama3.1-8B is an 8 billion parameter language model, fine-tuned from Meta-Llama-3.1-8B, with a 32768 token context length. This model is part of research exploring whether large language models can function as implicit text-based world models. It is specifically fine-tuned on the stabletoolbench_train_175183 dataset, indicating a focus on tool-use or structured interaction capabilities.
Loading preview...
Overview
This model, WorldModel-Stabletoolbench-Llama3.1-8B, is an 8 billion parameter language model developed by X1AOX1A. It is a fine-tuned version of meta-llama/Meta-Llama-3.1-8B, specifically trained on the stabletoolbench_train_175183 dataset. The model is part of a broader research initiative investigating the capacity of large language models to act as implicit text-based world models, as detailed in the associated arXiv paper "From Word to World: Can Large Language Models be Implicit Text-based World Models?".
Key Training Details
- Base Model: Meta-Llama-3.1-8B
- Dataset: stabletoolbench_train_175183
- Learning Rate: 1e-05
- Epochs: 5.0
- Batch Size: 128 (total train batch size)
- Optimizer: AdamW with betas=(0.9, 0.999) and epsilon=1e-08
- Context Length: 32768 tokens
Potential Use Cases
Given its fine-tuning on a 'stabletoolbench' dataset, this model is likely optimized for tasks involving tool use, structured interaction, or environments where understanding and predicting system states from text is crucial. Its foundation on Llama 3.1 and substantial context window make it suitable for complex reasoning and multi-turn interactions within such domains.