Triangulating 3D Polygons in Unity

I'm currently doing a job where I need to take 3D polygon data and display it in Unity.  These polygons are planar, but oriented arbitrarily in 3D space.  Moreover, they can contain holes (possibly multiple holes).  Think of a building wall with window cut-outs, and you'll get the idea.

This turns out to be a surprisingly thorny problem in Unity.  There is a simple script on the Unity wiki called Triangulator, but it only works with Vector2D and doesn't support holes.  I found a blog post on Advanced Triangulation in Unity, but it was neither sufficiently advanced (only works in the XY plane) nor actually in Unity (it wrote each polygon out to a file, invoked an external command-line tool to do the triangulation, and then read the result back in).

The utility referenced in that blog post is called Triangle, which is widely regarded as a very good triangle library.  It's open-source C code, so one could make a Unity plugin out of it.  But it's not licensed for commercial use, which is a problem for this project.  Also, making a native plugin means setting up a build chain for every platform you want to support.  For both reasons, I kept looking — I really wanted something in pure C#.

I finally found what I needed in the form of a C# port of a library called poly2tri.  The library and its API are entirely undocumented, as far as I can tell, so it took a little while to figure out what's what and how to use it.  But I got it working.

However, like the options above, it only works with 2D polygon data.  I still needed to handle polygons in 3D space.  To do that, I break the task into three steps:

  1. Rotate each point into the XY plane.
  2. Triangulate as usual.
  3. Rotate each point back.

In fact, this is hardly any extra work at all, since we already have to convert between Unity's data types (e.g. Vector3) and the custom point types used by poly2tri.  And figuring out the rotation needed is actually quite easy: you just take a cross product of any two edges of the polygon to get the normal to the plane the polygon is in; and then use Quaternion.FromToRotation to calculate the rotation needed to convert that into the normal we want (Vector3.forward, specifying the XY plane).

I wrapped this all up in a Poly2Mesh static class, which contains a very simple Polygon class.  You just set up a Poly2Mesh.Polygon with the outside contour and as many holes as you like, and then pass it to Poly2Mesh.CreateMesh or Poly2Mesh.CreateGameObject, and Bob's your uncle.  Here's some sample test code:

    void TestPoly2Tri() {
        Poly2Mesh.Polygon poly = new Poly2Mesh.Polygon();
        poly.outside = new List<Vector3>() {
            new Vector3(0,0,0),
            new Vector3(0,50,50),
            new Vector3(50,50,50),
            new Vector3(50,100,100),
            new Vector3(0,100,100),
            new Vector3(0,150,150),
            new Vector3(150,150,150),
            new Vector3(150,100,100),
            new Vector3(100,100,100),
            new Vector3(100,50,50),
            new Vector3(150,50,50),
            new Vector3(150,0,0),
        };
        poly.holes.Add(new List<Vector3>() {
            new Vector3(60,110,110),
            new Vector3(90,110,110),
            new Vector3(90,140,140),
            new Vector3(60,140,140),
        });
 
        // Set up game object with mesh;
        Poly2Mesh.CreateGameObject(poly);
    }

The result looks like the image below (in the scene editor, with the render mode set to Shaded Wireframe).

To use this, you need my Poly2Mesh file, along with just the "Triangulation" and "Utility" folders from the C# version of Poly2Tri.  You can download all that here.

Happy triangulating!