1 of 1

Material-Agnostic System Identification from Multi-Sequence Multi-View Videos

Albert Xiao

Advised by: Yizhou Zhao, Professor Min Xu

Given multi-view videos of an object in motion, how can we predict the future trajectory? Specifically, can we do it in a way that is grounded in governing physical laws?

Motivation

[1] Xuan Li, Yi-Ling Qiao, Peter Yichen Chen, Krishna Murthy Jatavallabhula, Ming Lin, Chenfanfu Jiang, and Chuang Gan. Pac-nerf: Physics augmented continuum neural radiance fields for geometry-agnostic system identification. arXiv preprint arXiv:2303.05512, 2023

​

[2] Licheng Zhong, Hong-Xing Yu, Jiajun Wu, and Yunzhu Li. Reconstruction and simulation of elastic objects with spring-mass 3d gaussians. In European Conference on Computer Vision, pages 407–423. Springer, 2025

​

[3] Junyi Cao, Shanyan Guan, Yanhao Ge, Wei Li, Xiaokang Yang, and Chao Ma. Neuma: Neural material adaptor for visual grounding of intrinsic dynamics. Advances in Neural Information Processing Systems, 37:65643–65669, 2025.

​

[4] Junhao Cai, Yuji Yang, Weihao Yuan, Yisheng He, Zilong Dong, Liefeng Bo, Hui Cheng, and Qifeng Chen. Gic: Gaussian-informed continuum for physical property identification and simulation. arXiv preprint arXiv:2406.14927, 2024.

​

[5] Yuchen Lin, Chenguo Lin, Jianjin Xu, and Yadong Mu. Omniphysgs: 3d constitutive gaussians for general physics-based dynamics generation. arXiv preprint arXiv:2501.18982, 2025.

​

[6] Genesis Authors. Genesis: A universal and generative physics engine for robotics and beyond, December 2024

References

For our task, we curate a novel multi-series multi-view dataset using the Genesis physics platform [6]:

Material types: Elastic, Elastoplastic, Liquid, Sand, Snow

Dataset Curation

We compare our results to several baselines from previous works: PAC-NeRF [1], Spring-Gaus [2], NeuMA [3], GIC [4], and OmniPhysGS [5].

​

We will evaluate our method’s outputs against ground truth point clouds (Chamfer Distance) and rendered camera views (PSNR, SSIM).

​

Numerical results coming soon !

Experiments and Metrics

  • We follow [4] in their 3DGS dynamic reconstruction and filling method to obtain surfaces and continuums
  • We obtain particle trajectories by further fine-tuning the deformation network to use as additional dense geometric supervision.
  • We parameterize elastic and plastic constitutive laws using neural networks.
  • During training, we randomly sample the initial continuum from the 10 sequences, simulate the trajectory, and compute losses.

Method

Geometric targets of our method

10 Sequences

11 Views

5 Materials across 10 Geometries

Elastic

Elastoplastic

Previous methods (see references) require predefined material priors and expert designed models. We seek to find a material-agnostic and learning-based solution.

Previous methods also only use one video sequence. We believe that using multiple sequences of the same object with different initial conditions will validate our model and increase generalizability.

Template ID: persuadingsapphire Size: 36x48