Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videos

Project page: https://kywind.github.io/pgnd

Instructions:

• Click the "Reset-<object>" button to initialize the simulation with the predicted video and Gaussian splats. Due to compute limitations of Huggingface Space, each run may take a prolonged period (up to 30 seconds).
• Use the buttons to move the gripper in the x, y, z directions. The gripper will move for a fixed length per click. The predicted video and Gaussian splats will be updated accordingly.
• X-Y plane is the table surface, and Z is the height.
• The predicted video from the previous step to the current step will be shown in the "Predicted video" section.
• The Gaussian splats after the current step will be shown in the "Predicted Gaussians" section.
• The simulation results may deviate from the initial shape due to accumulative prediction artifacts. Click the Reset button to reset the simulation state and reinitialize the predicted video and Gaussian splats.

Select a scene to reset the simulation:

Control the gripper to move in the x, y, z directions: