LevelSetMat : A 3D Model Acquisition ToolKit

Description:
LevelSetMat is a 3D Model Acquisition Toolkit which aims to integrate the distinct phases in the 3D model acquisition pipeline into a comprehensive and easy to use software package .

Broad Goals:
  1. To reduce the time and effort involved in reconstructing 3D digital models of real world scenes/ objects.
  2. To provide an environment to conveniently manage each individual acquisition pipeline as a separate project with ability to save the recentmost state information.
  3. To provide interactive user interfaces to tweak/modify parameters which affect the accuracy of the reconstructed models by providing real time feedback.
  4. To serve as a testbed for simulations. A simulation involves using an existing digital model to serve as a benchmark to compare the quality of the reconstructed digital models for accuracy/precision etc.
Description of the Reconstruction/Model Acquisition Pipeline:

As of today, LevelSetMat divides the Model Acquisition pipeline into the 4 distinct phases as shown in the flowchart below:

  1. The Range Image Acquisition Interface:


    Features
    1. Allows addition of range images into the system.
    2. Allows a digital model to be loaded and viewed in the display.Controls to rotate and translate the model are available on the display panel.
    3. Allows simulation of a spherical range image scanner to acquire range images from existing digital models for simulations.
    4. Allow all range images part of the system to be viewed.
    Some Pending Issues
    1. Allow controls to interactively change the Horizontal/ Vertical Field of View for Scanner simulation.
    2. Allow processing of range images for noise removal.
    3. Allow deletion of scans .
    4. Allow user selection for type of simulation - orthographic/spherical.



  2. The Registration Interface:

    Features
    1. Provides 2 displays (base and float) to load unregistered scans for registration.
    2. Provides a clickable interface to allow scan registration.Currently registration can be done by providing 3 or more point correspondences by clicking on feature points in either displays.(Horn's algorithm is used).
    3. Provides controls for positioning the each scan conveniently to facilitate registration and easy correspondence picking.
    4. Registration proceeds by choosing a "Target scan" with respect to which all scans are registered. The scans are classified into 3 categories:-
      1. Unregistered: Scans which are not registered.
      2. Locally registered : Scans which are registered to a scan other than the target scan.
      3. Globally Registered: Scans which are registered to the target scan.
      The user first picks a target scan.He then picks a "Base"scan from the list of available scans and a "Float"scan from the list of available float scans and registers them together.
    5. Supports "Cascading" of transforms using graph traversal. (Floyd Warshall Algorithm used)
    6. Supports "Selective scan selection" Registered scans can be selectively turned on/off to allow user to check for registration accuracy.Each registered scan is assigned a distinct color.
    Some Pending Issues
    1. When trying to register a large number of scans (>15), loading all scans into memory is a time consuming and memory intensive task.For machines with limited memory this is a problem.It might be useful to look at a client server approach to scan registration where a scan server is responsible for serving scans. The server could reside on a machine with high memory or it could possibly be a distributed system of servers on several machines. A client would query the servers for scans whenever needed to perform registration.


    The image below shows the registration of 2 scans of a volkswagon by picking correspondence points on each scan.



    Once registration is complete, the registered scans can be viewed in aligmment. The image below shows registered scans of the volkswagon , each scan is colored in a distinct color .To allow the user to check for registration accuracy , the scans can be selectively turned on or off as desired by checking / unchecking the checkboxes.


    These are images from registration of the tank scans.These demonstrate the selective scan selection feature by showing 1,2 and 3 registered scans together respectively.

  3. The Reconstruction Interface:

    The Reconstruction Screen is mostly work in progress.Here are some of the proposed features and issues
    Features
    1. Ability to control/modify reconstruction parameters to achieve desired reconstruction.
    2. Rotate/ Translate the reconstructed model for closer examination.
    Some Pending Issues
    1. Paralellizing Volume Initialization:For a volume of size 700x200x700 using 3 scans , it takes around 30+ mins to do the initialization. If we can parallelize this step we could gain significant performance gains.
    2. Automatic Volume Dimension Calculation using oriented Bounding Box (Using Eigen Vectors of Covariance Matrix). A tight fitting bounding box will give a much better reconstruction.
    3. Archiving reconstruction results after various steps in the iteration.This would allow the user to look at the model at various stages of iteration. He could later pick a model of his choice for further processing.
    4. Dynamic update of display as Model evolves over time. This would allow realtime feedback as the model changes over a period of time.