nllg/EdiTikZ-9B-RL

EdiTikZ-9B-RL: Multimodal TikZ Editing and Reconstruction

EdiTikZ-9B-RL is a 9 billion parameter multimodal language model developed by nllg, specialized in processing scientific figures. It takes an image of a scientific figure as input and generates its corresponding TikZ/LaTeX code. This model is an enhanced version of EdiTikZ-9B, having undergone multi-reward reinforcement learning (RL) on the DaEdiTikZ dataset.

Key Capabilities

• Instruction-Guided Editing: The primary differentiator is its ability to edit existing scientific figures based on natural language instructions. Users can provide an image and a description of desired changes, and the model outputs the modified TikZ code.
• Image-to-TikZ Reconstruction: Beyond editing, it retains the capability to reconstruct TikZ/LaTeX code directly from an image of a scientific figure, a skill learned during its supervised fine-tuning phase.
• TikZ/LaTeX Output: Generates valid LaTeX code wrapped with \documentclass[tikz]{standalone} and \begin{document} ... \end{document}.

Use Cases

• Automated Figure Modification: Ideal for researchers and technical writers who need to quickly modify scientific figures without manual TikZ coding.
• Digitizing Figures: Useful for converting raster images of scientific diagrams into editable vector graphics (TikZ).
• Educational Tools: Can assist in teaching TikZ by demonstrating how various visual elements are represented in code.

Technical Details

• Architecture: Multimodal language model.
• Training: Enhanced with RL on the DaEdiTikZ dataset for editing tasks.
• Context Length: Supports a context length of 32768 tokens.
• Inference: Supports standard Hugging Face transformers inference and faster batched inference using vLLM.