Laplaces-Red-Devils/fol-v03-cot-origin-qwen2.5-3

Overview

This model, Laplaces-Red-Devils/fol-v03-cot-origin-qwen2.5-3, is a 3.1 billion parameter Qwen2.5-3B-Instruct variant that has undergone LoRA Supervised Fine-Tuning (SFT). Its primary function is to translate natural language premises into a First-Order Logic (FOL) JSON format, specifically populating the premises_fol field in a completion.

Key Capabilities & Performance

• FOL Translation: Specializes in converting natural language statements into structured First-Order Logic representations.
• Training Metrics: Achieved an eval_loss of 0.196786 and an eval_fol_bleu score of 0.54987 during training.
• Exact Match Rate: Demonstrated an exact match rate of 0.1 (2 out of 20) on the test set for FOL greedy exact matching, and 0.05 (4 out of 80) on the full test set after fine-tuning.
• Base Model: Built upon the Qwen/Qwen2.5-3B-Instruct architecture, leveraging its foundational capabilities.
• Efficiency: Utilizes load_in_8bit and use_unsloth for potentially optimized resource usage during fine-tuning and inference.

Use Cases

This model is particularly well-suited for applications in:

• Automated Reasoning: Systems that require converting human-readable statements into a formal logical structure for automated deduction or inference.
• Knowledge Representation: Building knowledge bases where information needs to be stored and queried in a logical format.
• Natural Language Understanding (NLU): As a component for deeper semantic parsing of natural language, specifically for logical content.

Limitations

• The current exact match rates suggest that while the model can perform FOL translation, further refinement may be needed for high-precision applications. The eval_exact_match_rate during training was 0, indicating challenges in achieving perfect matches across the evaluation set.