Tuesday, July 20, 2010

Remeshing and Texturing (1)


In the pipeline of processing 3D data, after you have aligned and merged your range maps, you ofter require to get a nice clean textured mesh. In the last release of MeshLab we included our state-of-the-art parametrization/remeshing algorithm based on abstract parametrization. Now some a two-part tutorial on his practical usage.
Let's start from a medium complexity mesh of a skull (kindly provided and scanned for the VCG Lab by Marco Callieri). You can see it depicted in the two small figures on the right.
The mesh of the skull is composed by 1.000.000 triangles, it has a meaningful per-vertex color (recovered from a set of photos) and, as it often happens, it is topologically dirty.
First of all it is non 2-manifold (there are 7 edges where more than two face are incident) than there are many small holes and handles that make difficult any kind of parametrization.

So the first step is to build a watertight, coarser but topologically sound model.  Poisson surface reconstruction is a perfect filter for this task. A reconstruction at depth 9 is usually good, that generates a mesh of 1.3M of faces.  For this kind of processing a quite faithful geometric representation is not needed, but it is strongly needed that the overall topology is the right one. In this case some portions of the skull are remarkably thin and at low resolutions the poisson surface reconstruction can create unwanted holes. 
After that a further simplification step is needed to bring the model size to a number reasonable for the Isoparametrization engine.  Remember that the when building an abstract parametrization you do not need the full accuracy model but just a model that shares the overall shape and the same topology. For the purpose of the parametrization small details have a very small influence on the overall quality of the parametrization. Side figure depict the watertight Poisson reconstructed surface, note how the nostril cavity was filled (as expected because it was a hole with boundary).


So simplify it our watertight skull up to 50000 triangles. Take care to check Normal Preservation and Topology preservation Flag. The second one is particularly important, infact the basic edge collapse simplification algorithms can during simplification change the topology of the mesh, and while this is usually a nice feature (it allows for example the closure of very small holes) when you start from a mesh that is surely clean (a 2-manifold watertight model) it is better to be sure that such properties are preserved.

After that you can start with creating the Abstract Isoparametrization, a technique we introduced in:


Nico Pietroni, Marco Tarini, Paolo Cignoni
Almost isometric mesh parameterization through abstract domains
IEEE Transaction on Visualization and Computer Graphics, Volume 16, Number 4, page 621-635 - July/August 2010

Without going into details, that you will find in the above paper, the main idea is rather simple. Usually textures are defined in a dominion that is just the (0,0)-(1,1) square on the plane. In our approach as a domain of the parametrization we use a different 2-dimensional domain, the surface of a very coarse simplicial complex that has the same topology of the original mesh and it is composed by just a few hundred triangles. Such an approach is interesting because this abstract parametrization can be used for a number of things, like for example remeshing, texturing, tangent space smoothing etc.

To build the abstract isoparametrization just start the corresponding filter called "Isoparametrization", (default params are ok, you can lower convergence precision to a '1' to speedup a bit and try to change a bit the targeted size of the abstract domain). It is a bit slow so wait some minutes for the processing. At the end of the process, you do not see anything directly but the structure is attached to the mesh and you can use it in the other filters. If you want re-use it for a later use you have to save both the processed mesh and as a separate step the isoparametrization using the "Isoparametrization Save Abstract Domain Filter".

The created isoparametrization can be used to build a standard parametrization over any mesh that is reasonably close to the original one.
In our example we take a simplified version of the original mesh, composed by just 10000 triangles ("Skull_10k.ply"). We transfer over this simplifed mesh the just build isoparametrization
and then using the filter "Iso Parametrization transfer between meshes", setting as source mesh the one with the abstract parametrization (skull_60k_isoparam.ply) and skull_10k.ply as target.

Now we can transform the transferred isoparametrization into a standard atlased parametrization using the "Isoparametrization Build Atlased Mesh" filter. The two image on the right seems equal but you can see that in the lower one the triangles of the mesh have been cut along the triangles of the abstract parametrization in order to get proper atlas regions. At this point your mesh has a standard texture parametrization and it is ready for use it for a variety of operation.

The First thing that we can do is just to transfer the color of the original 1M vertexes color onto a texture according this parametrization. This can be done by using the filter "Transfer color to texture (between 2 meshes)", choose a reasonable texture size (2048x2048 is good) and you will obtain a simplified textured mesh that looks strikingly similar to the original heavy 1M tri model (try to compare the first and last snapshots).

 Summarized Recipe
  1. take a 1M tri colored model
  2. make the model watertight using Poisson
  3. Simplify it to a 50k model (preserving topology)
  4. Build the Isoparametrization
  5. build another very simple 10k model from the original 1M model
  6. transfer the isoparametrization over the very simple model
  7. convert the isoparametrization into a standard atlased texture
  8. generate a texture with the color from the original 1M model
Next part of the tutorial with remeshing and other hints in a few days...

    17 comments:

    Jake said...

    Thanks for the Remeshing and texturing workflow - Meshlab is a great app that I use in conjuction with May,Zbrush and 3DCoat - it fixes a lot of issues the other apps cant handle easily.

    Jake said...

    Hi Been working through your breakdown with little success, MeshLab v1.30b tends to crash (File : rastering, Line: 289 )when transferring vertex colour data to textured mesh on a Poisson reconstructed mesh (Which has UVs -created externally).
    Looking forward to non beta version - still a fantastic app.

    Giancamati said...

    Hello,

    thank you very much for the nice tutorial. Actually I was trying to follow it, but I get stuck at the Iso Parametrization step as MeshLab keeps crashing.
    Is there any improvement since the last posted beta version?

    Many thanks,
    JK

    Giancamati said...

    btw the error I get it's a runtime assention failed! error:

    Expression: testBaryCoords(bary)
    line 116
    file: ./param_flip.h

    Regards,
    JK

    Barrel of Fails said...

    Hi. I'm just wondering what programming language MeshLab uses?
    Please respond
    monorailkitteh@live.com

    Chris said...

    How do you reduce the triangle from 1.3M to 50,000? I can't seem to do that initial step. Thanks.

    Stefan said...

    I have tried to run this tutorial, but it sticks to the isoparametrization. Nonmanifold edges stops almost every filter, even if I folowed every step.

    ALoopingIcon said...

    @Barrel_of_Fails:
    it is C++ (And QT for UI)

    @Chris:
    Use the Filter->Remeshing->Quadric Edge Collapse Decimation

    @Stefan:
    Yes, non manifold edges are an issue. Cleaning the mesh is a mandatory step. More on this kind of cleaning in a short future (hopefully...).

    Andrea said...

    Hi,
    I'm encountering an error message transfering the iso parametrization between 2 different meshes: it says the source mesh must have the iso parametrization, even if I've just applied it

    I'm using Meshlab 1.3.1
    OS Win7 32 bit

    Thank you for help

    Sergi said...

    Hello,

    I'm trying this tutorial, but when I do Iso Parametrization, Meshlab opens a windows with de message: "Failure of filter: Iso Parametrization
    Non possible parametrization because of non manifold edges".

    Can someone help me?

    Thanks

    Andrea said...

    @Sergi
    The mesh you're working with has got some non-manifold edges.
    Try to select and delete them (Filters>Selection>Select non-manifold edges).

    However, you'd better remesh your model by poisson surface reconstruction, if you want to avoid this kind of inconvenients.

    Sergi said...

    @Andrea

    Thanks,

    I'm working with a model remeshed by poisson surface reconstruction.

    I tryed to select and delete the edges. Now, the message of error in the iso parametrization process has changed. The new message says:

    Failure of the filter: Iso parametrization

    non possible parametrization because of multi component mesh.

    What should I do now?

    Thank you very much

    Sergi said...

    Hi,

    Another question:
    In the example of the skull, when the poisson surface reconstruction is done, all holes be closed. But in the end of the tutorial, the nose's hole again be open. How is this? Is because the iso parametrization?

    Best regards

    Andrea said...

    @Sergi
    Sorry I'm late!
    1. Probably there are small independent components in your mesh. Try to detect small detached parts and delete them. Don't worry: it could happens when you use surface reconstruction!
    2. I suppose they have deleted redundant faces, but it doesn't depend on the iso parametrization tool

    Sergi said...

    Thank you very much @Andrea

    I have one more question: I am working with Bird's skulls. I scan the skull and then I process it with Meshlab. But when I do the Poisson surface reconstruction appears holes in the very thin superfices. Are there some solution to reconstruction the surface without appears a hole?

    Thank you very much

    Sergi said...

    Sorry, Superfices = surfaces (my mistake)

    I'm working with depth 14. With depth 15 or higher the Meshlab crash. Are there a solution to unwanted holes?

    sewi said...

    This is such an interesting story at it gives in-depth information to the reader as well. Good thing you have it shared. sonic game