sandeep123/ML-LaySum-Gemma2-2B-Instruct-SFT

Overview

This model, ML-LaySum-Gemma2-2B-Instruct-SFT, is a full-parameter supervised fine-tuning of the Gemma 2 2B Instruct model. Its primary purpose is to generate English lay summaries of machine learning research papers. The model processes a paper's abstract, introduction, and conclusion to produce a simplified summary intended for a general audience.

Key Capabilities

• Lay Summarization: Specializes in transforming complex machine learning research into understandable summaries for non-experts.
• Input Handling: Utilizes the abstract, introduction, and conclusion sections of research papers as input.
• Full-Parameter Fine-Tuning: The entire model was fine-tuned, not just an adapter, making it a complete, standalone model.
• Context Length: Supports a maximum input length of 8192 tokens.

Training Details

The model was trained on 6,911 examples from the ML-LaySum dataset over 3 epochs, with 879 validation examples. Training used float32 master parameters with bfloat16 autocast, and the final weights are in bfloat16. A long-input policy was implemented to trim only the suffix of overlong introductions, preserving other critical sections.

Good for

• Academic Communication: Researchers and institutions looking to make their machine learning papers more accessible to a broader audience.
• Science Journalism: Journalists or communicators needing to quickly grasp the essence of ML research for public dissemination.
• Educational Tools: Developing tools that simplify complex technical literature for students or interdisciplinary professionals.

Limitations

It is important to note that this model's training and validation losses do not guarantee factual accuracy or accessibility for non-experts. Generated summaries may omit qualifications or introduce unsupported statements, requiring careful evaluation for both source faithfulness and reader comprehension. No state-of-the-art claim is made regarding its performance.