# Mesh Generation

New solutions have emerged that address the limitations of Marching Cubes, allowing the conversion of dense meshes to low-poly meshes.

Earlier in the course, we highlighted the significance of **differentiability** and how mesh rendering is non-differentiable, involving discrete decisions like:

- Is this pixel inside the triangle?

While this remains true, new research introduces a differentiable approach to mesh generation by treating mesh triangles as discrete symbols, similar to words in a language model.

## MeshAnything

[MeshAnything](https://huggingface.co/spaces/Yiwen-ntu/MeshAnything) is a recent research project that converts dense meshes to low-poly meshes using techniques introduced in [MeshGPT](https://huggingface.co/papers/2311.15475).

The main components of MeshAnything are:

1. **VQ-VAE Encoder:** Encodes dense 3D data to a discrete latent representation using a Vector Quantization (VQ) Variational Autoencoder (VAE).
2. **Autoregressive Transformer Decoder:** Generates the triangles of the mesh using an autoregressive transformer decoder.

More details can be found in the paper [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://huggingface.co/papers/2406.10163).

![MeshAnything Demo](https://huggingface.co/datasets/dylanebert/ml-for-3d-course/resolve/main/meshanything-demo.gif)

## Implications

MeshAnything represents a major step in 3D mesh generation, addressing a major bottleneck toward practical generative 3D tools.

However, the current results are comparable to or worse than traditional topology reduction methods like [Decimate](https://docs.blender.org/manual/en/latest/modeling/modifiers/generate/decimate.html), still requiring significant manual refinement.

Differentiable mesh generation opens the door to context-aware topology reduction, i.e. accounting for shape and deformation. While still a work in progress, solving this will lead to highly practical 3D tools.

