Laplaces-Red-Devils/fol-v01-origin-qwen2.5-3

Overview

The Laplaces-Red-Devils/fol-v01-origin-qwen2.5-3 is a fine-tuned version of the Qwen/Qwen2.5-3B-Instruct model, specifically adapted for the task of translating natural language premises into First-Order Logic (FOL) premises. This model utilizes LoRA SFT (Low-Rank Adaptation for Supervised Fine-Tuning) to achieve its specialized function.

Key Capabilities

• Natural Language to FOL Translation: The model's core capability is to convert natural language statements into a structured JSON output containing FOL premises.
• JSON Output Format: It is designed to produce FOL premises within a premises_fol key in a JSON object, facilitating programmatic use.
• Base Model: Built upon the Qwen2.5-3B-Instruct architecture, providing a robust foundation for language understanding.

Training Details

The model was trained for 1 epoch with a max_seq_length of 2048 and a gradient_accumulation_steps of 8. It leveraged unsloth for efficient training, loading the model in 8-bit precision.

Performance Notes

During evaluation, the model achieved an eval_loss of 0.576187. The current exact match rate for FOL translation on the train, dev, and test splits is 0%, indicating that further fine-tuning or dataset refinement may be necessary for higher accuracy. Inference latency averaged 17.747 seconds per sample on the test set.

Good for

• Research in Logic and NLP: Exploring methods for automated logical form generation from natural language.
• Prototyping: Initial development of systems that require converting natural language into a formal logical representation for reasoning engines.