arjhinety/onebee-gf-sft-v0
arjhinety/onebee-gf-sft-v0 is an early-scale LoRA SFT checkpoint built on the Gemma4 (multimodal, text + vision) architecture, specifically fine-tuned from google/gemma-4-E2B-it. This model, with an effective parameter count of approximately 2 billion and a context length of 131,072 tokens, is designed for companion-persona conversational responses conditioned on retrieved memories. It serves as a baseline for reproducibility of v0-scale results in research on post-training and memory architecture for small models.
Loading preview...
Model Overview
arjhinety/onebee-gf-sft-v0 is an early-stage LoRA SFT (Supervised Fine-Tuning) checkpoint based on the google/gemma-4-E2B-it model. It utilizes the Gemma4 multimodal architecture, supporting both text and vision inputs, and inherits a substantial context length of 131,072 tokens. This version represents a Day-4 baseline from a research project, trained on a small dataset of 202 examples derived from 4 personas.
Key Capabilities
- Companion-persona conversational responses: The model is fine-tuned to generate dialogue in a companion persona, leveraging a small set of retrieved memories and recent conversational turns.
- Research artifact: Primarily intended for reproducing v0-scale results of the
small-mind-companionproject, which investigates post-training and memory architectures for small LLMs.
Intended Use & Limitations
This model is specifically for reproducing the project's v0-scale results and is not recommended as a starting point for new work; subsequent, more advanced versions like onebee-gf-sft-v1 or onebee-gf-distill-v1 are suggested. It has known limitations due to its small data scale (202 examples) and exhibits false-abstention issues that were addressed in later versions. The model has not undergone meaningful preference alignment (DPO) at this stage. Evaluation against the Personalized Memory Benchmark (PMB) showed a UAR of 33.75% with memory enabled, compared to 16.25% without.