Learning to Think and Explore in 3D World

Project overview (68 s). Every indoor frame shown is a real rendered view the policy asked for during evaluation.

Prior 3D vision–language models study spatial reasoning under observations specified a priori — complete RGB-D scans, multi-view images, videos, or explicit 3D representations such as point clouds and meshes — treating the acquisition of task-relevant evidence as a prerequisite rather than part of reasoning itself. Act3D is handed one image. The 3D scene stays outside the model, as an environment it can act in — move left, turn right, step forward — with each action returning a newly rendered egocentric view. Reasoning and evidence acquisition happen in the same rollout, and nothing in training tells the model which viewpoint was the right one to visit.

Act3D. Starting from a single rendered view and a question, the policy alternates reasoning, a camera action, and a new observation, committing to an answer only once it has the evidence.

Abstract

We study how vision-language models (VLMs) can learn to think and actively explore in 3D space. Instead of providing multi-view images, videos, or 3D representations to the model, we preserve the native visual encoding of VLMs — a single image, one at a time, just like humans — and treat the 3D representation only as an environment with which the model can interact through actions (e.g., move left) and observations (rendered images).

Each rollout begins from an initial first-person view and a visual question. The model can then iteratively reason, navigate, and reorient its viewpoint before committing to a final answer. Following an initial trajectory-supervised finetuning, we apply reinforcement learning (RL) with rewards driven primarily by final-answer correctness. Crucially, RL provides no direct supervision over which viewpoints the model should visit or which exploration actions it should take. Instead, the model must learn an effective exploration policy solely from the extent to which its interactions improve performance.

Across six 3D benchmarks, Act3D achieves an average accuracy of 53.2%, improving its base Qwen model by 22.0% and surpassing proprietary Gemini 3.6 Flash. Act3D also outperforms existing open-source 3D methods while using 50–70% fewer input frames, and generalizes to unseen domains with an average improvement of 5.9% over its supervised initialization. Further analysis reveals adaptive, question-conditioned exploration: gains over the base model increase from 0.6% on questions answerable from the initial view to 16.3% on those requiring extensive exploration. Act3D searches for occluded targets, avoids unnecessary exploration, and acquires complementary viewpoints for spatial comparisons. These findings demonstrate that outcome-based RL enables VLMs not only to reason over visual evidence, but also to learn when and how to acquire it.

Method

We formalize 3D reasoning as a POMDP. A task instance \((\mathcal{M}, q, y^\star)\) pairs a reconstructed scene \(\mathcal{M}\), a question \(q\), and an answer. The scene is hidden: the policy sees only the rendered egocentric view \(o_t = \mathrm{Render}(\mathcal{M}, c_t)\) at its current camera pose \(c_t\), and changes what it can see by acting.

Training has two stages. Trajectory-supervised initialization establishes the reason–act–observe protocol from 5,712 curated demonstrations; we emphasize that these provide policy initialization, not canonical exploration targets. Outcome-driven RL then optimizes the full interactive policy with GRPO, against a reward that depends only on how the episode ends — the full breakdown sits with the data it scores.

What the policy learns

Training data

Two corpora, built for different jobs. Trajectory supervision needs a few thousand complete reason–act–observe episodes; RL needs only a question, an answer, and a valid starting viewpoint, so it can draw on an order of magnitude more. Every question type below survives the same test: it must still be well posed when the model starts from one view and has to go and find the rest.

What the supervised episodes are made of

We use GPT-5.4 to generate reason–act–observe trajectories in the same environment the policy acts in. We discard invalid starting views and retain trajectories with correct answers, valid interactions, well-formed reasoning and responses, and sufficient visible scene geometry; MP3D examples must also observe the evidence the question requires. After answer canonicalization the corpus contains 5,712 trajectories, 20,603 rendered observations and 830 scenes.

Curated questions for RL, and why these types

RL requires only a question–answer pair and a valid initial viewpoint, allowing substantially broader training data than trajectory supervision: 51,543 questions from ScanQA, MP3D, and generated QA over ScanNet and ARKitScenes. Those generated QA begin as 53,104 candidates, of which 16,125 survive two filters. A heuristic pass drops anything a single view already answers, anything with degenerate ground truth, and anything phrased as ego-relative left/right — ill-defined once the camera is free to move. An LLM audit then rejects questions whose answer leaks in the prompt, questions too vague to have one answer, and questions whose ground truth is wrong.

• 
  “Is there a sink in the room?” existence · 20,275 generated · answer: no
• 
  “Which is there more of, pillows or televisions?” comparison · 12,302 generated · answer: pillow
• 
  “Which is closer to the window, the books or the cabinet?” nearest · 6,386 generated · answer: books
• 
  “How many desks are in the room?” count · 5,419 generated · answer: 2
• 
  “What color is the desk?” colour · 4,553 generated · answer: black
• 
  “Which is taller, the sofa or the window?” vertical · 3,717 generated · answer: sofa
• 
  “Which is the largest object in the room, the bed or the dresser?” size · 291 generated · answer: bed
• 
  “What is on the desk?” on-top · 161 generated · answer: lamp

One kept question per type, each shown with the starting view its episode begins from — rendered through the same tool the policy calls, so this is literally the image the model is handed before it moves. The surviving types share a property that matters for active perception: the answer is a fact about the scene, not about where the camera happens to be, so it stays well defined no matter which viewpoint the policy chooses — while still, as these views show, rarely settled from the initial view alone.

What these questions are rewarded for

The curated questions are usable for RL because they can be scored without a demonstration. GRPO optimizes the whole episode against the end of it alone:

Real episodes from these corpora

Each card is one ScanQA evaluation episode: the first view the policy was given, a view it chose part-way through, and the view it answered from, with the number of interactions it took. The tag under each answer names what the question asks for; none of these episodes could be settled from the starting view alone.

Results

Six 3D benchmarks, one interaction protocol

Every active model receives the same single 640×480 initial render and the same twelve-interaction budget; passive methods keep their own input protocols. Green marks a benchmark whose domain appears in that model's training data. Act3D trained on mixed data achieves the highest average accuracy of 53.2%, surpassing the Qwen3.5-9B base model by 22.0% and Gemini 3.6 Flash by 1.0% under the same interaction protocol, as well as all evaluated open-source 3D reasoning methods. Against its own SFT initialization, RL improves the average from 48.6% to 53.2% (+4.6%), with the largest gains on ARKit (+14.1%) and SPBench-MV (+8.1%).

Accuracy rises; so does interaction

Panel (c) is in-training rollout statistics; (a) and (b) are offline validation accuracies, and no value should be read across the two.

The gain is largest exactly where exploration is needed

What the design choices are worth

Exploration that fits the question

The policy searches when the target is occluded, makes a short confirmation when it is already visible, and gathers several viewpoints when the question compares objects. The reasoning trace updates with each returned observation and stops once the evidence is sufficient.

Conclusion

We present Act3D, a framework for learning to think and explore in the 3D world by treating 3D representations as interactive environments rather than assumed-complete inputs. Starting from a single observation, Act3D iteratively reasons, acts, and acquires visual evidence through interaction with the environment. Building on supervised trajectory initialization, verifiable RL further optimizes the policy without explicit supervision of exploration actions or viewpoints.

Across six 3D benchmarks, Act3D achieves the highest average accuracy of 53.2%, improving upon its SFT initialization by 4.6% and outperforming existing active and passive 3D reasoning methods. Further analyses show that these gains extend beyond the training benchmark and are particularly pronounced on questions requiring more extensive exploration. Qualitative results also reveal question-conditioned exploration strategies that adapt to newly acquired observations. Together, these findings demonstrate the potential of outcome-based RL to learn not only how to reason over visual evidence, but also how to actively acquire the evidence needed for 3D reasoning.

BibTeX

@misc{li2026act3d,
      title={Learning to Think and Explore in 3D World},
      author={Yijiang Li and Bingyang Wang and Nuno Vasconcelos},
      year={2026},
      eprint={2610.21783},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2610.21783},
}