This project is no longer accepting applications. Subscribe to our newsletter to be notified of new projects!
Learn how to use modern AI image generators to color and stylize 3D objects through text prompts.
Generative AI is currently one of the most in-demand skills across multiple industries. As this technology continues to spread across a wide range of fields, the demand for professionals with experience in generative AI is only expected to grow. An area of particular interest is the intersection of computer graphics and generative AI. Can we utilize generative AI to create/edit 3D content? This holds significant potential for industries such as video games, digital architecture, e-commerce, and AR/VR.
You'll wear the hat of a Junior Machine Learning Scientist and leverage generative AI models to enhance 3D digital objects. You will use advanced technologies like "ControlNet" and the "Segment Anything Model" to generate and apply new colors and textures to 3D models. You will also become familiar with industry-standard tools such as Google Colab, PyTorch, and Diffusers.
Get to know the Build Fellow and other students, ask questions about the project requirements, prepare your workspace.
Learn the basics of working with 3D data. Loading, visualizing, and modifying 3D meshes.
Learn how to use text-to-image generators so that you can generate images with more control / art direction.
Exploring more control capabilities to further author the output of the generation
Use Facebooks SAM to automatically and interactively segment images
Refining generated images with ESRGAN
Combine all the methods we explored to generate the best results for your final deliverable. Clean up your code and Notebooks.
Polish your project deliverables and present them to the Build Fellow and other students in the final group session.
Get access to all of our Build projects, including this one, by creating your Build account!
Get started by submitting your application.
We'll notify you when projects reopen. In the meantime, you can explore our resources and learn more about our Fellows.
Pablo is the co-founder and Chief Product Officer at Hypothetic Inc., a 3D Generative AI company. With over seven years of experience as a Machine Learning scientist he has worked on ML for Computer Animation, 3D Rendering, and Fluid Simulations, and founded three ML companies. He holds an MSc in Applied Mathematics.