# ACE-Brain-0.6: Scaling Spatial Grounding for Physical Foundation Models

🌐 Project Page | 🤗 Hugging Face | 📔 arXiv Paper | 🤖 GitHub

## Overview **ACE-Brain-0.6-9B** is a 9B vision-language model that strengthens spatial grounding for physical foundation models. Given one or more images and a natural-language query, it identifies task-relevant objects, counts instances, resolves spatial relationships, and localizes interaction targets. Built on **Qwen3.5-9B**, ACE-Brain-0.6-9B is fine-tuned on approximately **1.5 million training samples**. A unified data pipeline converts robot manipulation recordings, egocentric interactions, object and part annotations, and spatial data into **verifiable grounding supervision**. Training updates the language-model parameters while keeping the vision encoder frozen. On seven public grounding benchmarks, ACE-Brain-0.6-9B achieves a reported **Avg-7 of 75.41**, compared with **73.68** for GPT-6 Astra. It exceeds Astra on **four of seven** benchmarks, with remaining gaps on PixMo-Count, RefSpatial, and PointArena.

ACE-Brain-0.6: grounding capabilities, data pipeline, model, and benchmark results

## Key Features - **Unified grounding interface:** Supports points, point lists, bounding boxes, integer counts, and categorical answers (`yes`, `no`, or `none`) for target identification, multi-instance perception, spatial grounding, and interaction grounding. - **Robot-centered data pipeline:** Automatically annotates manipulation recordings and combines them with complementary visual sources to broaden grounding coverage. - **Verifiable supervision:** Checks object labels, requires complete instance sets for counting, verifies absent-target questions, and excludes evaluation images and episodes from training. - **Scaling with diverse data:** Grounding performance improves across progressively larger and more diverse training releases under a fixed backbone and training recipe. - **Strong robot-scene performance:** Achieves **93.42** on RobotGrounding, the team's held-out evaluation set covering referring, counting, absent targets, spatial judgments, and cross-view correspondence. ## Performance Highlights Results below are reported in **Table 1** of the [paper](https://arxiv.org/abs/2610.27040). Higher is better; bold marks the best score among the models shown. | Benchmark | GPT-5.5 | GPT-5.6-sol | GPT-6 Astra | Claude Opus 5.5 | Gemini 4 Argon | ACE-Brain-0.6-9B | |:--|--:|--:|--:|--:|--:|--:| | Where2Place | 45.25 | 46.23 | 71.10 | 66.44 | 71.20 | **81.44** | | PixMo-Count | 82.61 | 83.55 | **88.10** | 83.74 | 87.71 | 86.96 | | RoboSpatial | 59.19 | 64.34 | 69.55 | 73.30 | **80.94** | 76.56 | | RefSpatial | 60.29 | 54.51 | **80.14** | 71.10 | 51.56 | 71.84 | | PointArena | 68.02 | 69.76 | **75.36** | 61.20 | 66.40 | 74.13 | | RoboAfford | 68.04 | 71.45 | 84.94 | 75.10 | 71.48 | **88.14** | | ShareRobot (affordance) | 38.57 | 36.66 | 46.60 | 40.20 | 29.53 | **48.82** | | **Avg-7 · public benchmarks** | 60.28 | 60.93 | 73.68 | 67.30 | 65.55 | **75.41** | | RobotGrounding · in-house benchmark | 83.96 | 85.18 | 91.10 | 90.89 | 78.51 | **93.42** | **Avg-7** is the unweighted mean of the seven public benchmark scores. **RobotGrounding is evaluated separately and is not included in Avg-7.** It contains **2,591 questions** and reports a macro average over **14 scoring groups**. ## Inference Example Install a Transformers version with Qwen3.5 support: ```bash pip install -U "transformers>=5.2.0" accelerate torch torchvision pillow ``` The example uses the checkpoint and processor files from this repository in **non-thinking mode**. Replace `scene.jpg` and the query with your own image and grounding instruction. ```python import torch from PIL import Image from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration model_id = "ACERobotics/ACE-Brain-0.6-9B" processor = AutoProcessor.from_pretrained(model_id) model = Qwen3_5ForConditionalGeneration.from_pretrained( model_id, dtype="auto", device_map="auto", ).eval() image = Image.open("scene.jpg").convert("RGB") messages = [ { "role": "user", "content": [ {"type": "image", "image": image}, { "type": "text", "text": ( "Point to the red cup in the image. " "Return a point as [[x, y]], with integer coordinates " "normalized to [0, 1000]. Return none if it is absent." ), }, ], } ] inputs = processor.apply_chat_template( messages, tokenize=True, add_generation_prompt=True, enable_thinking=False, return_dict=True, return_tensors="pt", ).to(model.device) with torch.inference_mode(): generated_ids = model.generate( **inputs, max_new_tokens=128, do_sample=False, ) answer_ids = generated_ids[:, inputs["input_ids"].shape[1]:] answer = processor.batch_decode( answer_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False, )[0] print(answer) ``` Grounding coordinates are integers normalized to **[0, 1000]** along each image axis. Depending on the query, the model can return a point, a list of points, a box, a count, or a categorical answer. Free-space grounding identifies unoccupied **2D regions** and does not by itself establish 3D placement feasibility. The example uses greedy decoding for simplicity. The reported benchmarks use temperature **0.2** and a **128-token** output budget, except PixMo-Count, which uses **1,024 tokens**. See the report for the full evaluation protocol and the [Transformers Qwen3.5 documentation](https://huggingface.co/docs/transformers/model_doc/qwen3_5) for the model interface. ## Citation If you find ACE-Brain-0.6-9B useful for your research and applications, please cite our paper: [ACE-Brain-0.6: Scaling Spatial Grounding for Physical Foundation Models](https://arxiv.org/abs/2610.27040) · [PDF](https://arxiv.org/pdf/2610.27040) ```bibtex @misc{wang2026acebrain06scalingspatialgrounding, title={ACE-Brain-0.6: Scaling Spatial Grounding for Physical Foundation Models}, author={Yinhe Wang and Tao Tao and Zexin He and Shuang Luo and Wei Mao and Yao Mu and Feng Pan and Hanming Deng and Lewei Lu and Zhi Hou and Xingang Pan and Dahua Lin and Dacheng Tao and Xiaogang Wang}, year={2026}, eprint={2610.27040}, archivePrefix={arXiv}, primaryClass={cs.RO}, url={https://arxiv.org/abs/2610.27040}, } ```

Full Model Card (README)