moralogyengine/TinyLlama-1.1B-Chat-moralogy-dpo-v4
moralogyengine/TinyLlama-1.1B-Chat-moralogy-dpo-v4 is a 1.1 billion parameter language model developed by Moralogy Engine, fine-tuned using Direct Preference Optimization (DPO) on the moralogy-1200 dataset. This model is a proof-of-concept for the Moralogy framework, designed for artificial reasoning by generating natural language responses to moral dilemmas. It is intended to be used as Layer 2 in a two-layer system, where a deterministic 'Moral Kernel' guarantees predicate logic for ethical decision-making. Its primary differentiator is its focus on axiomatic derivation for AI alignment, aiming for zero fabrication and auditable failure modes in moral reasoning.
Loading preview...
Moralogy Engine V4: TinyLlama-1.1B-Chat DPO
This model, developed by Moralogy Engine, is a 1.1 billion parameter proof-of-concept for a novel approach to AI alignment, focusing on "Artificial Reasoning" rather than human preference optimization. It utilizes a formal framework grounded in axiomatic derivation to evaluate moral dilemmas.
Key Differentiators & Architecture
Unlike standard LLMs, this model is designed as Layer 2 of a two-layer system. A deterministic "Moral Kernel" (Layer 1) evaluates predicate logic based on a "Wrongness Formula" (Wrong(a) ⟺ ∃x[ H(x,a) ∧ ¬Consent(x,a) ∧ ¬PGH(a) ]). The V4 model then generates natural language reasoning, with the kernel providing an OVERRIDE CHECK to block outputs on predicate mismatch. This architecture aims to guarantee predicate logic and auditable moral reasoning.
Training & Performance
The model is based on TinyLlama/TinyLlama-1.1B-Chat-v1.0 and was fine-tuned using Direct Preference Optimization (DPO) on the moralogyengine/moralogy-1200 dataset, covering domains like Medical, Defense, Automotive, and Customer Service. Key findings include:
- Phase Transition in Moral Reasoning: A sudden emergence of moral geometry during training, reproducible across versions.
- Zero Fabrication: The model reasons within the dilemma's geometry without inventing non-existent escape routes.
- Auditable Failure: When the model fails, the framework identifies which predicate failed and why, allowing for targeted fixes (e.g., adding vicious consent vectors).
Use Cases & Limitations
This model is best used in conjunction with the Moral Kernel to ensure predicate correctness, especially in complex ethical dilemmas. While the 1.1B parameter count is a proof-of-concept, the framework's geometry is claimed to scale to any base model. The primary limitation is that the model alone can be insufficient for adversarial consent dilemmas, requiring the kernel's intervention. The axioms are philosophically defensible but open to empirical invitation rather than dogmatic acceptance.