DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs

Abstract

4D spatio-temporal reasoning, jointly modeling 3D spatial structure and temporal evolution, is essential for understanding dynamic worlds and enabling embodied interaction. While current Multimodal Large Language Models (MLLMs) show strong capabilities in static scene understanding and coarse-grained 4D tasks, they still have notable limitations in continuous dynamic scene perception, especially in tracking dynamic object evidence for coherent 4D spatio-temporal reasoning. This shortcoming stems mainly from relying on sparse frame-level observations, fragmenting continuous dynamic cues and leaving models unable to disentangle genuine object dynamics from camera-induced apparent motion. Inspired by humans tracking dynamic cues while compensating for viewpoint changes, we propose DynTrace, a training-free framework for 4D spatio-temporal reasoning with two complementary components. Dynamic Trajectory Visualization (DTV) reprojects world-coordinate trajectories onto the image plane, providing geometry-informed visual priors that disentangle genuine object dynamics from camera-induced apparent motion. Meanwhile, the Dynamic Trace Token (DT-Token), organized into a Dynamic Trace Graph (DTG), tracks object-level dynamic cues, trace evolution, and key moments, maintaining continuous dynamic object evidence for coherent 4D reasoning. Together, these two components equip MLLMs with continuously tracked dynamic object evidence, grounded in geometry-informed visual priors and structured spatio-temporal traces. DynTrace consistently improves open-source MLLMs, achieving state-of-the-art results on Dyn-Bench, VLM4D, and DSI-Bench, validating the importance of tracking dynamic object evidence for robust 4D spatio-temporal reasoning.

Motivation

Why current MLLMs fail on dynamic 4D reasoning.

Failure Modes

(i) Dynamic Source Confusion: ego-motion-aware perception requires disentangling camera motion from genuine object movement, yet even methods injecting geometric cues such as depth or pose do not model ego-motion as a separate factor explicitly. When the camera moves amid multi-entity interactions, models often attribute ego-motion-induced apparent motion to objects, misidentify which entities truly move, and distort relational judgments, making tracked dynamics unreliable. (ii) Dynamic Trace Loss: continuous object tracking requires sufficiently dense temporal coverage to preserve full motion trajectories, yet current strategies rely on sparse frame sampling, temporal token compression, or clip-level summaries rather than continuous object-centric traces. Consequently, dynamic objects' temporal evidence chain fragments, and model attention drifts toward unstable background changes over truly salient motion.

Method

Tracking dynamic object evidence before final reasoning.

Overview of DynTrace. Given a video and a language query, DynTrace follows three stages: (1) Dynamic Objects Extraction identifies query-relevant and independently moving instances, producing temporally consistent dynamic masks; (2) Spatio-Temporal Dynamics Encoding lifts tracked instances into a shared world frame to reconstruct Geometry-Grounded Dynamic Evidence, including Object Trajectory, Camera Behavior, and Relation Evolution, from which it derives DTV through World-to-Image Reprojection and converts Dynamic Cues, Trace Evolution, and Key Moments into object-side and relation-side DT-Tokens before organizing them into a DTG; and (3) Representation Integration & Reasoning feeds DTV, DTG, and the query into the target MLLM for 4D spatio-temporal reasoning.

Component 1

Dynamic Trajectory Visualization

DTV reprojects world-coordinate motion back to the image plane, producing geometry-informed visual priors that make true motion explicit even under viewpoint change, zoom, or camera translation.

Component 2

Dynamic Trace Graph

DTG serializes dynamic cues, trace evolution, and key moments into a compact graph-like textual representation, preserving object-level and relation-level evidence over long temporal horizons.

Inference Style

Training-Free Integration

DynTrace operates at inference time. It upgrades existing open-source MLLMs without retraining, making the framework practical for broad deployment across different backbones and benchmarks.

DTV Visualization

DynTrace Example

Complete DTV and DTG visualization for Case 3.

Results

Consistent gains across three dynamic reasoning benchmarks.

dyn-bench_results

Comparison on Dyn-Bench across nine dynamic reasoning categories

Method	Avg.	Inter-Object			Object-Scene			Camera-Object
Method	Avg.	Act. & Obj. Desc.	Move. & Temp. Dyn.	Spatial Rel. & Change	Mov. Patterns & Traj.	Spatial Rel. & Comp.	Scene Focus & Dyn.	Cam. Motion & Orient.	Cam-Obj. Interaction	Temp. & Visual Change
Spatial MLLMs
SpaceR-7B	56.5	66.6	49.2	52.7	72.2	67.8	78.2	50.3	40.0	55.5
VST-7B-RL	55.7	68.6	48.4	51.9	73.0	70.7	79.4	45.1	39.1	52.9
Spatial-SSRL-7B	45.9	54.5	40.0	48.1	68.5	65.9	73.8	35.8	36.7	37.7
SpatialReasoner	54.5	63.2	44.1	50.4	68.2	64.2	74.0	48.0	44.6	54.0
SpatialThinker-7B	53.7	63.2	40.7	46.5	70.6	67.3	77.5	46.0	42.8	51.5
MLLM-4D-8B	56.4	63.6	44.5	47.4	64.3	61.4	70.6	52.4	52.3	63.7
LLaVA-ST-7B	50.2	56.3	40.3	48.7	62.3	59.8	70.1	47.8	38.9	46.8
Video MLLMs
LLaVA-OV-1.5-8B	53.8	60.9	47.7	53.4	74.4	69.6	75.4	41.0	37.0	51.6
Videorefer-7B	56.1	65.2	50.9	56.5	73.2	72.4	79.0	36.0	43.0	56.1
InternVideo2.5-Chat-8B	54.2	67.9	48.5	46.3	70.1	65.8	76.5	42.0	42.9	52.1
VideoLLaMA3-7B	54.3	63.2	40.7	46.5	70.6	67.3	77.5	46.0	42.8	51.5
Training-free MLLMs
See&Trek	51.5	67.2	44.8	49.5	69.5	64.7	72.9	47.2	34.5	41.0
GSM	55.3	63.9	49.3	49.9	73.3	70.8	79.2	48.5	36.7	50.8
Ours
Qwen3-VL-8B-Instruct	60.8	70.8	52.6	53.6	75.0	71.2	79.0	54.3	51.4	59.7
+ DynTrace	65.8 (+5.0)	79.7 (+8.9)	57.1 (+4.5)	62.2 (+8.6)	80.7 (+5.7)	75.3 (+4.1)	85.6 (+6.6)	58.9 (+4.6)	52.6 (+1.2)	64.4 (+4.7)
Qwen3-VL-32B-Instruct	62.4	71.4	54.6	56.1	75.3	74.4	79.8	55.9	53.4	58.4
+ DynTrace	66.9 (+4.5)	82.4 (+11.0)	61.8 (+7.2)	66.2 (+10.1)	80.7 (+5.4)	75.7 (+1.3)	84.5 (+4.7)	56.7 (+0.8)	53.9 (+0.5)	65.6 (+7.2)
InternVL3.5-8B	53.2	69.2	44.4	47.4	66.2	63.7	71.9	44.3	44.8	49.3
+ DynTrace	58.0 (+4.8)	73.9 (+4.7)	50.7 (+6.3)	52.6 (+5.2)	70.8 (+4.6)	68.0 (+4.3)	80.2 (+8.3)	47.7 (+3.4)	48.3 (+3.5)	53.8 (+4.5)
InternVL3.5-14B	56.0	72.3	49.6	47.1	70.6	68.2	75.6	48.8	47.0	46.2
+ DynTrace	60.3 (+4.3)	74.6 (+2.3)	52.7 (+3.1)	59.8 (+12.7)	75.5 (+4.9)	72.0 (+3.8)	80.5 (+4.9)	51.1 (+2.3)	48.9 (+1.9)	53.1 (+6.9)

merge_results

Comparison on VLM4D and DSI-Bench

Method	VLM4D			DSI-Bench
Method	Avg.	Real	Synthetic	Avg.	Obj-Scn	Obs-Scn	Obs-Obj
Spatial MLLMs
SpaceR-7B	47.4	49.2	41.8	54.2	71.2	38.3	51.1
VST-7B-RL	44.7	45.1	43.1	51.5	69.4	34.8	48.4
Spatial-SSRL-7B	52.4	52.2	53.3	51.4	66.5	37.0	50.2
MLLM-4D-8B	59.1	59.3	58.4	45.2	61.3	33.4	30.9
Video MLLMs
LLaVA-OV-1.5-8B	46.3	47.8	41.3	52.7	72.2	36.6	42.2
InternVideo2.5-Chat-8B	49.0	50.6	44.0	49.4	64.0	35.0	49.3
VideoLLaMA3-7B	49.8	55.6	32.1	55.6	74.8	37.5	52.9
Training-free MLLMs
See&Trek	47.8	44.6	57.5	52.7	66.1	38.5	56.5
GSM	48.4	48.6	47.6	54.7	73.9	36.8	51.1
Ours
Qwen3-VL-8B-Instruct	59.0	58.1	61.6	53.4	66.2	39.4	58.7
+ DynTrace	64.2 (+5.2)	62.9 (+4.8)	68.3 (+6.7)	56.4 (+3.0)	67.8 (+1.6)	43.5 (+4.1)	63.1 (+4.4)
InternVL3.5-8B	48.6	49.5	46.1	44.5	52.1	35.0	51.6
+ DynTrace	54.7 (+6.1)	53.2 (+3.7)	59.2 (+13.1)	48.7 (+4.2)	53.1 (+1.0)	41.5 (+6.5)	59.0 (+7.4)

comprehensive_ablation

Ablation studies on Dyn-Bench

Variant	Avg.	Inter-Object	Object-Scene	Camera-Object
Qwen3-VL-8B	60.8	56.7	74.2	54.7
+ DTG	63.0	61.8	76.3	55.0
+ DTV	61.7	59.3	74.5	54.8
+ DTV & DTG	65.8	63.6	79.6	58.0
DT-Token Components
InternVL3.5-8B	53.2	50.8	66.8	45.9
+ Cues	56.8	55.5	70.6	48.4
+ Cues & Trace	57.6	56.6	70.8	50.1
+ Cues & Trace & Moments	58.0	56.8	72.3	50.3

Qualitative Results

How DTV and DTG change the evidence path.

The qualitative examples below all present DynTrace gains through progressively richer visual and structured evidence.

Qualitative Cases-1

Qualitative results of DynTrace on two representative cases.

This figure shows the two most representative gain patterns in the main paper: DTV corrects ambiguous camera-relative motion in case (a), while DTG preserves long-range relation traces in case (b), leading to correct predictions in both cases.

Qualitative Cases-2

Additional trajectory change and interaction distance cases.

Additional baseline-versus-DynTrace comparisons on trajectory change and interaction distance reasoning questions.

Attention Stabilization

Attention comparison between baseline and DynTrace.

The baseline attention gradually drifts toward visually salient but temporally irrelevant regions, while DynTrace keeps attention concentrated on the query-relevant dynamic object and its evolution.

Current Limitations

Representative limitation cases of DynTrace.

DynTrace is still less sufficient when reasoning requires object-centric first-person frame conversion or subtle part-level motion analysis, which indicates the next direction for improving dynamic object evidence.

BibTeX

@misc{gao2026dyntrace,
  title        = {DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs},
  author       = {Rongxin Gao and Yuzhi Huang and Dongxuan Liu and Chu Li and Zhenye Wang and Jie Wu and Shuzhao Xie and Jingyan Jiang and Xinghao Ding and Xiaotong Tu and Yue Huang},
  year         = {2026},
  note         = {Manuscript}
}

Ready to copy.

DynTrace : Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs