"Built on m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA"

ANITA-NEXT-24B-Magistral-2506-VISION-ITA is a Thinking Vision Language Model of the ANITA - Large Language Models family. The model is a merge of textual layers from of ANITA-NEXT-24B-Magistral-2506-ITA and vision layers and processor from mistralai/Mistral-Small-3.1-24B-Instruct-2503. This model version aims to be the a Multilingual Vision Language Model 🏁 (EN 🇺🇸 + ITA🇮🇹) to further fine-tuning on Specific Tasks in Italian.

❗❗❗Use at your own risk. The model may generate hallucinations, incorrect, invented, offensive, unethical or dangerous responses. We are not responsible for any dangerous/offensive/criminal use. The model is release for research only purposes.❗❗❗ The 🌟**ANITA project**🌟 *(**A**dvanced **N**atural-based interaction for the **ITA**lian language)* wants to provide Italian NLP researchers with an improved model for the Italian Language 🇮🇹 use cases. The **NEXT** family includes **four models**: - m-polignano/ANITA-NEXT-24B-Magistral-2506-ITA - **General Purpose** - m-polignano/ANITA-NEXT-24B-Dolphin-Mistral-UNCENSORED-ITA - **Uncensored** - m-polignano/ANITA-NEXT-24B-Magistral-2506-VISION-ITA - **Vision-Language** - m-polignano/ANITA-NEXT-20B-gpt-oss-ITA - **Agentic Ready**

**GGUF - OLLAMA**: [m-polignano/ANITA-NEXT-24B-Magistral-2506-VISION-ITA-GGUF](https://huggingface.co/m-polignano/ANITA-NEXT-24B-Magistral-2506-VISION-ITA-GGUF)

**Colab Demo:** [A100 - 40GB - Colab Notebook](https://colab.research.google.com/drive/1mhZLAdpOr3TRq-ZTG4XD52J98orHj_na?usp=sharing)
The Model runs on a single GPU, 19.56GB of VRAM by using a *4bit Quantization*.

## Specifications - **Model developers**:
Ph.D. Marco Polignano - University of Bari Aldo Moro, Italy
SWAP Research Group
- **Variations**: The model release has been **supervised fine-tuning (SFT)** using **QLoRA** 4bit, on instruction-based datasets. **DPO** approach over the *mlabonne/orpo-dpo-mix-40k* dataset is used to align with human preferences for helpfulness and safety. - **Input**: Models input text only. - **Language**: Multilingual 🏁 + Italian 🇮🇹 - **Output**: Models generate text and code only. - **Model Architecture**: *Mistral architecture*. - **Context length**: 128k, but degradate after 40k. - **Library Used**: [Transformers 4.56.0.dev0] (https://huggingface.co/docs/transformers/index)

## Playground To use the model directly, there are many ways to get started, choose one of the following ways to experience it. ### Prompt Template ``` [SYSTEM_PROMPT]Sei un assistente AI per la lingua italiana di nome ANITA-NEXT (Advanced Natural-based interaction for the ITAlian language Next Generation) creato dal ricercatore Marco Polignano, Università degli Studi di Bari Aldo Moro, Italia. Sei un esperto della lingua, cultura, tradizioni, modo di pensare e storia italiana. L'utente ti chiederà di risolvere un compito o rispondere ad una domanda. Rispondi e ragiona usando la lingua della domanda, preferendo l'Italiano. Scrivi il tuo flusso di pensiero (monologo interiore) tra i tag . Ragiona in modo disinvolto, scrivendo riflessioni e/o bozze, come se stessi lavorando a un esercizio su un foglio di carta. Successivamente, scrivi la soluzione in modo chiaro, corretto, semplice ed esaustivo basandoti sul riassunto del tuo flusso di pensiero. Se necessario, usa la notazione markdown per formattare la risposta.[/SYSTEM_PROMPT][INST]{ USER Prompt }[/INST]{ ASSIST Thinking }{ ASSIST Prompt } ``` ### Transformers For direct use with `transformers`, you can easily get started with the following steps. - Firstly, you need to install transformers via the command below with `pip`. ```bash pip install -U --no-deps bitsandbytes accelerate xformers==0.0.29.post3 peft trl triton cut_cross_entropy unsloth_zoo pip install sentencepiece protobuf "datasets>=3.4.1,<4.0.0" "huggingface_hub>=0.34.0" hf_transfer pip install transformers==4.52.4 vllm==0.8.4 wget https://fastly.picsum.photos/id/237/200/300.jpg?hmac=TmmQSbShHz9CdQm0NkEjx1Dyh_Y984R9LpNrpvH2D_U -O image.jpg ``` - Right now, you can start using the model directly. ```python from PIL import Image from transformers.image_utils import load_image from transformers import AutoProcessor, AutoModelForVision2Seq import torch processor = AutoProcessor.from_pretrained("m-polignano/ANITA-NEXT-24B-Magistral-2506-VISION-ITA") model = AutoModelForVision2Seq.from_pretrained("m-polignano/ANITA-NEXT-24B-Magistral-2506-VISION-ITA", torch_dtype=torch.bfloat16, device_map="auto", load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16, ) # Load images image1 = load_image("image.jpg") prompt = "Descrivi il contenuto dell'immagine?" messages = [ { "role": "user", "content": [ {"type": "text", "text": prompt}, {"type": "image"}, ], }, ] prompt = processor.apply_chat_template(messages, add_generation_prompt=True) inputs = processor(text=prompt, images=[image1], return_tensors="pt") inputs = inputs.to("cuda").to(torch.bfloat16) # Generate outputs generated_ids = model.generate(**inputs, max_new_tokens=500) generated_texts = processor.batch_decode( generated_ids, skip_special_tokens=True, max_tokens=8192, temperature=0.7, top_p=0.90, ) print(generated_texts[0]) ```

## Citation instructions ```bibtex @misc{polignano2024advanced, title={Advanced Natural-based interaction for the ITAlian language: LLaMAntino-3-ANITA}, author={Marco Polignano and Pierpaolo Basile and Giovanni Semeraro}, year={2024}, eprint={2405.07101}, archivePrefix={arXiv}, primaryClass={cs.CL} } ``` ```bibtex @article{rastogi2025magistral, title={Magistral}, author={Rastogi, Abhinav and Jiang, Albert Q and Lo, Andy and Berrada, Gabrielle and Lample, Guillaume and Rute, Jason and Barmentlo, Joep and Yadav, Karmesh and Khandelwal, Kartik and Chandu, Khyathi Raghavi and others}, journal={arXiv preprint arXiv:2506.10910}, year={2025} } ``` # Acknowledgments We acknowledge the support of the PNRR project [FAIR - Future AI Research (PE00000013)](https://fondazione-fair.it/en/foundation/), Spoke 6 - Symbiotic AI (CUP H97G22000210007) under the NRRP MUR program funded by the NextGenerationEU. Models are built on the Leonardo supercomputer with the support of CINECA-Italian Super Computing Resource Allocation, class C project IsCc6\_LAM-next (HP10CQ77K6).