lewisdog/qwen3-1.7b-cogs-ingest

Model Overview

lewisdog/qwen3-1.7b-cogs-ingest is a 2 billion parameter Qwen3-based model, developed by lewisdog, specifically engineered as a student model for the cogs ingest pipeline. It is a merged full model (Qwen/Qwen3-1.7B + LoRA, with weights fused) designed to replace larger teacher LLMs for local, OpenAI-compatible structured output generation.

Key Capabilities

• Structured JSON Output: Generates compact, schema-exact JSON for four specific cogs tasks:
  • extract: Provides summary and key_claims (with quotes/entities).
  • suggest_links: Outputs linked_claims.
  • page_update: Delivers topic, section_md, and relevant information.
  • contradiction: Identifies findings.
• Optimized for Ingestion: Fine-tuned to produce JSON exactly as the cogs runtime parses it, ensuring high compatibility and reliability for data ingestion workflows.
• Apple Silicon Support: Can be converted and 4-bit quantized into an MLX model for efficient deployment on Apple silicon via mlx_lm.convert.

Training Details

The model underwent LoRA SFT (TRL) training on 1,921 cogs distill chat pairs over 2 epochs, using an effective batch size of 16 and a maximum sequence length of 8192. Training was performed in bf16 precision, achieving a train loss reduction from 2.55 to 1.41 and an eval token-accuracy of 0.756.

Usage Notes

• Greedy Decoding Warning: Pure greedy decoding (temperature 0, no penalty) can lead to degeneration on long list-valued tasks, resulting in unterminated JSON. This can be mitigated by using repetition_penalty \approx 1.1 or Qwen's non-thinking sampling (temperature=0.7, top_p=0.8, top_k=20). The bundled generation_config.json already uses temp=0.6 to address this.
• Transformers Integration: Easily loadable with AutoModelForCausalLM and AutoTokenizer from the transformers library, with example code provided for generation.