Implementation Details and Description of the Mosaicing algorithm

---

This section describes the overall algorithm and and the techniques used for the following aspects of the mosaicing algorithm:

• Image Size
• Feathering
• Contrast Correction
• Cascading

The Image Mosaicing Algorithm Calculate or obtain correspondences between the different images which would constitute the mosaic. Compute the parameters of a projective transformation between each of the source images. Resample the points in the space of the target image. Apply Compositing schemes (weighted average, average etc.) to smoothly blend the edges. Apply contrast correction to accomodate contrast discontinuities in the images being mosaiced.

The next section describes these steps in more detail:-

1. Obtaining correspondences: This is a manual process. The user is expected to specify the correspondences in the parameter file.
2. Compute Projective Transforms:The program solves a linear system of 8 unknowms using the method of PseudoInverse (in case of an overconstrained equation) using VISPack's Linear equation solver.For the 8 parameters of the projective transformation we need 8 equations,so we need a minimum of 4 point correspondences, however giving more correspondences overconstrains the system and gives better results. In the implementation i calculate and store the transformations between all possible sources and targets so that the image can mosaiced for any given target image.
   Support for Cascading Transforms
   The correspondence matrix describes a graph which has edges for all available correspondences. To support cascading we first calculate the transformations between the images which have correspondences with each other. By computing the transitive closure of this graph we can obtain all possible transformation from any given image to another.Once we have this transformation set we can go from the domain of any one image to another by using the transformation connecting them.
3. Resample points in target image space:To do this the program first determines the size of the target image by computing the Bounding box of the corners of the projected image.To do the mosaicing, a "not so efficient" algorithm would be to go over each pixel in the Bounding box for the mosaiced image and apply the transformation for each source image to get zero or more greyvalues (depending on how many images overlap at that position) and keep doing this for all the pixels.We can optimize the algorithm by only doing the projections for an image if the current pixels position lies within the bounding box of a particular source. This way we do not unnecessarily try to do transformations when we know that the point definitely lies outside the boundary of the source image.
4. Compositing the image: Since all the images are captured under different lighting conditions , the mosaiced image shows seams and edges at the intersection points. To alleivate these artifacts we need a technique to smoothly blend the images at the image boundaries. I have explored 2 approaches for blending seams:
   1. Averaging: This is simply an average of all greyvalues which project at the same pixel location.It is fast and easy to compute but might not give very good results . (in particular if the images are slightly misalligned , this approach will give rise to the ghost effect)
   2. Weighted Averaging( Feathering ):The idea is to weigh pixels based on distances from the center of the image.So if a pixel is at the edge of the source image it will get a very low weight whereas one which is closer to the center will get a higher weight. The weighting function is essentially a hat- function which looks like the following image which i generated in matlab.
      
5. Contrast Correction: The idea behind contrast correction is to compensate for the different exposures of the camera while taking these images. If we think of the differences in the brightness levels of the images being mosaiced as fixed value relative to each other we can perform contrast adjustment by finding the relative mean luminance difference between each of the source images.To do these we compute the difference in mean luminance between every pair of images in the region of overlap.Ideally the difference should be zero. This difference gives us an estimate of the relative brightness gains between different images and we can compensate for this by adding this difference back.