Rubin-Wei/MemoryDecoder-Qwen3-1.7B-biology

TEXT GENERATIONPricing:Input $0.32 / Cached $0.064 / Output $1.6Concurrent Unit Cost:1Model Size:2BQuant:BF16Context Size:32kTool Calling:SupportedPublished:Jul 23, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

The Rubin-Wei/MemoryDecoder-Qwen3-1.7B-biology is a 1.7 billion parameter Memory Decoder, a pretrained parametric long-term memory component designed to be integrated with a compatible frozen Qwen3 Base language model backbone. Developed by Rubin Wei and collaborators, this model is specifically trained on a biology CPT corpus using retrieval-derived sparse target distributions. It specializes in the biology domain, enhancing the performance of its paired backbone on biological tasks, and is intended for research and evaluation in this field.

Loading preview...

MemoryDecoder-Qwen3-1.7B-biology: Specialized Biological Memory

This model is a 1.7 billion parameter Memory Decoder, developed by Rubin Wei and the LUMIA Group, designed as a pretrained parametric long-term memory component. Unlike standalone LLMs, it functions by being swapped into a compatible frozen Qwen3 Base language model backbone, enhancing its domain-specific knowledge.

Key Capabilities & Features

  • Domain Specialization: Specifically trained on a biology CPT corpus, making it highly effective for tasks within the biological domain.
  • Parametric Long-Term Memory: Provides a mechanism for language models to access and utilize specialized long-term memory, improving performance on domain-specific queries.
  • Qwen3 Compatibility: Built with a Qwen3 architecture and tokenizer, ensuring seamless integration with Qwen3 Base models ranging from 0.6B to 14B parameters.
  • Research-Oriented: Intended for research and evaluation, particularly for exploring the benefits of domain-specific memory components in LLMs.

When to Use This Model

  • Enhancing Qwen3 for Biology: Ideal for researchers and developers looking to augment a frozen Qwen3 Base model with deep biological knowledge.
  • Domain-Specific NLP Research: Useful for experiments in specialized domains where factual accuracy and detailed knowledge are critical.
  • Evaluation on BioInst: Evaluated using the BioInst benchmark, indicating its suitability for biological question-answering and understanding tasks.

It's important to note that this is a memory component, not a standalone chat or instruction-tuned model, and its outputs are dependent on the paired backbone and interpolation settings.

Finetunes
1 models