In this work, we have only considered rendering static scenes with illumination derived from a lit sphere. If we transform the scene as part of an animation, for example, some problems present themselves.
The lit sphere model discussed in Section 2 is based on the assumption that source materials are homogeneous, while artists often encode local surface features in their work. As a result, local material effects may be encoded in the lit sphere model as variation in illumination. If the animation is created by reprojecting the lit sphere at every frame according to the new viewpoint, then local texture features will appear to follow the eye. There may be cases where this effect is unobtrusive.
As an example of such a situation, consider capturing idealized skin
such as that seen in Olivia De Berardinis' art work, the ``cheesecake
portraits'' [1]. The shading darkens at the silhouettes
and has a soft highlight in the middle. For a 3D model such as the
doll in Figure 7, the lit sphere model is convincing
when reprojected. As the model moves, the shading will always darken
at the silhouettes with a soft highlight in the center.
However, if local texture features are very prominent, the surface will appear to ``swim.'' As an example, we have applied the shading from the frescos in the Sistine Chapel by Michelangelo to a 3D model of Michelangelo's David (Figure 8). There is a lot of texture information in the image of the fresco. When we apply the lit sphere and rotate the model, David appears metallic or satiny due to the high frequency noise. We'd prefer to make the texture stick to the object and allow the lighting to follow the eye, similar to lit sphere applied to the doll.
For completeness, we will briefly describe an approach to achieve approximate separation of texture and lighting. The first step is to extract lighting from the hemisphere. In many cases, we have found that a band-pass filter can be used successfully to extract variation in shading due to lighting. Another approach is to apply the work of Walter et al. [15] to fit virtual Phong lobes to the surface of the lit hemisphere, and subtract out their contributions. The result of either method will be two hemispheres, the first containing the lighting information, the second being the texture information that remains when the lighting is excised. Because lighting often obscures local texture features, the second sphere cannot be used directly as a texture map. Furthermore, we have texture information only for the front side of the sphere. This leaves half the surface bare. The most generally applicable technique seems to be synthesizing the texture over the sphere using the information in the lit sphere minus lighting. Synthesis directly on the sphere would also serve to avoid the distortions which occur when a planar image is mapped onto the sphere.
Rather than draw attention away from the simple idea which is at the core of this paper, we relegate further exploration of these methods to future work. It has been our experience that the use of lit spheres in rendering from novel viewpoints works without qualification. The quality of animations will depend on the degree to which the captured spheres meet the requirements of the lit sphere model (for example, that the represented materials are homogeneous).
Future research could connect the ability to create lit spheres with a 3D paint program. This would provide a quick underlying surface texture, over which the user could then add features that cannot be uniformly applied over the model, such as detail brush strokes or features such as blush or freckles on a face. In addition, automatic methods for determining shading models from any artistic image may be possible.
