text-to-shape
Papers with tag text-to-shape
2022
- TextCraft: Zero-Shot Generation of High-Fidelity and Diverse Shapes from TextAditya Sanghi, Rao Fu, Vivian Liu, Karl Willis, Hooman Shayani, Amir Hosein Khasahmadi, Srinath Sridhar, and Daniel RitchieIn 2022
Language is one of the primary means by which we describe the 3D world aroundus. While rapid progress has been made in text-to-2D-image synthesis, similarprogress in text-to-3D-shape synthesis has been hindered by the lack of paired(text, shape) data. Moreover, extant methods for text-to-shape generation havelimited shape diversity and fidelity. We introduce TextCraft, a method toaddress these limitations by producing high-fidelity and diverse 3D shapeswithout the need for (text, shape) pairs for training. TextCraft achieves thisby using CLIP and using a multi-resolution approach by first generating in alow-dimensional latent space and then upscaling to a higher resolution,improving the fidelity of the generated shape. To improve shape diversity, weuse a discrete latent space which is modelled using a bidirectional transformerconditioned on the interchangeable image-text embedding space induced by CLIP.Moreover, we present a novel variant of classifier-free guidance, which furtherimproves the accuracy-diversity trade-off. Finally, we perform extensiveexperiments that demonstrate that TextCraft outperforms state-of-the-artbaselines.
Text-to-shape generation without training on paired data. 在一组3D shape上训练. 训练基于CLIP的image feature生成shape的网络. 从而实现基于CLIP的texture feature生成shape.
text-to-shape@inproceedings{TextCraft, title = {TextCraft: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Text}, author = {Sanghi, Aditya and Fu, Rao and Liu, Vivian and Willis, Karl and Shayani, Hooman and Khasahmadi, Amir Hosein and Sridhar, Srinath and Ritchie, Daniel}, year = {2022}, tags = {text-to-shape}, sida = {Text-to-shape generation without training on paired data. 在一组3D shape上训练. 训练基于CLIP的image feature生成shape的网络. 从而实现基于CLIP的texture feature生成shape.}, }