Abstract: A new image-based rendering method, based on the light field and Lumigraph system, allows illumination to be changed interactively. It does not try to recover or use any geometrical information (e.g., depth or surface normals) to calculate the illumination, but the resulting images are physically correct. The scene is first sampled from different viewpoints and under different illuminations. Treating each pixel on the back plane of the light slab as a surface element, the sampled images are used to find an apparent BRDF of each surface element. The tabular BRDF data of each pixel is further transformed to the spherical harmonic domain for efficient storage. Whenever the user changes the illumination setting, a certain number of views are reconstructed. The correct user perspective view is then displayed using the texture mapping technique of the Lumigraph system. Hence, the intensity, the type and the number of the light sources can be manipulated interactively.