jmarceno/Elixir-Gemma-4-12B
The jmarceno/Elixir-Gemma-4-12B is a 12 billion parameter Gemma 4 model, fine-tuned specifically for generating Elixir code. It specializes in producing Elixir code that parses, compiles, and passes tests, building upon the instruction-tuned QAT checkpoint of the base Gemma 4 12B. This model demonstrates a significant improvement in Elixir task completion, achieving a 35.9% pass rate compared to the base model's 24.2% on a 128-task executable harness. It is optimized for developers working with Elixir, particularly for tasks involving standard library usage, error handling, OTP, and BEAM abstractions.
Loading preview...
Elixir Gemma 4 12B: Specialized Code Generation
The jmarceno/Elixir-Gemma-4-12B is a 12 billion parameter Gemma 4 model, meticulously fine-tuned to excel at generating high-quality Elixir code. Starting from the instruction-tuned QAT checkpoint of the base Gemma 4 12B, this model's primary focus is on producing Elixir code that is not only syntactically correct but also compiles and passes associated tests.
Key Capabilities & Performance
- Enhanced Elixir Code Generation: Demonstrates a substantial improvement in solving Elixir coding tasks. On a 128-task executable harness, it achieved a 35.9% pass rate, a +11.7 percentage point increase (or +48% relative gain) over the base Gemma 4 12B QAT model's 24.2%.
- Specific Area Improvements: Shows notable gains across various Elixir domains, including:
- Standard library tasks: 10/16 (up from 6/16)
- Error handling: 11/16 (up from 8/16)
- OTP (Open Telecom Platform): 4/16 (up from 2/16)
- BEAM (Erlang VM): 3/16 (up from 1/16)
- Abstractions: 3/16 (up from 1/16)
- Cleaner Code Output: Produces cleaner completions, with the warning-free rate improving from 46.9% to 58.6%. All completions consistently stay within a single fenced answer.
- Context Length: Supports a context length of 16,384 tokens, as used in its evaluation setup.
Good For
- Elixir Developers: Ideal for developers seeking an LLM specifically optimized for generating functional and test-passing Elixir code.
- Code Assistance: Useful for accelerating development by providing accurate code snippets for common Elixir patterns, standard library functions, and OTP behaviors.
- Learning Elixir: Can serve as a valuable tool for understanding how to implement various Elixir concepts correctly, from error handling to BEAM interactions.