Papers
arxiv:2406.03184

Ouroboros3D: Image-to-3D Generation via 3D-aware Recursive Diffusion

Published on Jun 5
· Submitted by akhaliq on Jun 6
Authors:
,

Abstract

Existing single image-to-3D creation methods typically involve a two-stage process, first generating multi-view images, and then using these images for 3D reconstruction. However, training these two stages separately leads to significant data bias in the inference phase, thus affecting the quality of reconstructed results. We introduce a unified 3D generation framework, named Ouroboros3D, which integrates diffusion-based multi-view image generation and 3D reconstruction into a recursive diffusion process. In our framework, these two modules are jointly trained through a self-conditioning mechanism, allowing them to adapt to each other's characteristics for robust inference. During the multi-view denoising process, the multi-view diffusion model uses the 3D-aware maps rendered by the reconstruction module at the previous timestep as additional conditions. The recursive diffusion framework with 3D-aware feedback unites the entire process and improves geometric consistency.Experiments show that our framework outperforms separation of these two stages and existing methods that combine them at the inference phase. Project page: https://costwen.github.io/Ouroboros3D/

Community

Paper author

Our Ouroboros3D integrates diffusion-based multi-view image generation and 3D reconstruction into a recursive diffusion process. It uses the reconstruction results as 3D-aware feedback to assist multi-view generation, just like an Ouroboros eating its own tail. Effectively improves the generation quality through joint training.

Project page: https://costwen.github.io/Ouroboros3D/
Arxiv: https://arxiv.org/abs/2406.03184
Code (coming soon): https://github.com/Costwen/Ouroboros3D

Show our generated 3D >>>

Paper author

[OOD]

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2406.03184 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2406.03184 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2406.03184 in a Space README.md to link it from this page.

Collections including this paper 3