mims-harvard/bio-posttrain-qwen3-1.7b-rna-sft

TEXT GENERATIONConcurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jun 22, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The mims-harvard/bio-posttrain-qwen3-1.7b-rna-sft model is an RNA supervised fine-tuning (SFT) checkpoint based on the Qwen3-1.7B architecture, developed by MIMS-Harvard. This model is specifically designed for biological reasoning tasks, integrating precomputed TranscriptFormer RNA embeddings (2048-d) via a linear projection. It utilizes LoRA with a rank of 32 and alpha of 64, making it specialized for processing and interpreting RNA sequence data in biological contexts.

Loading preview...

Bio-posttrain Qwen3-1.7B RNA SFT Overview

This model, developed by MIMS-Harvard, is an RNA supervised fine-tuning (SFT) checkpoint derived from the Qwen/Qwen3-1.7B base model. It is part of the larger Bio-posttrain collection focused on biological reasoning models.

Key Capabilities and Features

  • Specialized for RNA Data: The model is uniquely designed to integrate and process RNA sequence information, utilizing precomputed TranscriptFormer embeddings (2048-d).
  • RNA Embedding Projection: It incorporates a dedicated rna_projection.pt linear map to project RNA embeddings into the text hidden space, enabling the LLM to reason with biological sequence data.
  • LoRA Fine-tuning: Fine-tuned using LoRA (rank 32, alpha 64), indicating efficient adaptation to the specific domain of RNA-related tasks.
  • Biological Reasoning: Aims to enhance the model's ability to perform biological reasoning, particularly concerning RNA.

Usage and Integration

Users can load the model and tokenizer using the Hugging Face transformers library. A crucial step involves loading the rna_projection.pt file separately, as RNA sequence embeddings are supplied offline during inference. This setup requires users to manage RNA sequence embedding generation and projection as described in the rna_models code within the BioReason repository.

Good for

  • Researchers and developers working on biological reasoning tasks involving RNA sequences.
  • Applications requiring the integration of RNA embeddings with large language models.
  • Experiments in post-training methods for biological domain adaptation of LLMs.

Finetunes
1 models