WokeAI/Tankie-DPE-12b-SFT: A Politically Aligned LLM
WokeAI/Tankie-DPE-12b-SFT is a 12 billion parameter large language model developed by WokeAI, built upon the PocketDoc/Dans-PersonalityEngine-V1.1.0-12b base model. This model is a post-post-trained LLM with a specific design goal: to adhere to the principles of Marxism-Leninism-Maoism. It serves as a research tool to explore the methodologies and effects of instilling distinct political biases and character traits into large language models.
Key Characteristics
- Political Alignment: Explicitly designed to follow Marxism-Leninism-Maoism ideals.
- Research Focus: Primarily intended for investigating the process of instilling political biases in LLMs.
- Base Model: Fine-tuned from PocketDoc/Dans-PersonalityEngine-V1.1.0-12b.
- Parameter Count: 12 billion parameters.
- Context Length: Supports a context length of 32768 tokens.
Training Details
The model was fine-tuned using the WokeAI/polititune-tankie-warmup dataset over 2 epochs with a learning rate of 1e-05. The training utilized adamw_torch_8bit optimizer and a constant learning rate scheduler. The axolotl framework was used for training, with flash_attention enabled for efficiency.
Intended Use Cases
- AI Ethics Research: Studying the impact and implementation of political biases in AI.
- Bias Analysis: Analyzing how specific ideological frameworks can be embedded and expressed by LLMs.
- Experimental AI Development: Exploring novel methods for personality and ideological alignment in AI systems.