Light field representation: The dimensions in light fields

Abstract:

Capture is the first step in light field application: Whether in plenoptic cameras or in camera arrays, neither the geometry nor the coloring is perfect. Consequently, light field pre-processing pipelines do include: De-Bayering: Typical sensors record raw-luminance in Bayer patterns, i.e., single pixels represent one of the three primary colors, and consequently for full color images interpolation techniques need to be applied. - Color correction: Majorly in arrays, but to a smaller extent also in plenoptic cameras the coloring is not identical for all subaperture views. Color-matching techniques have to be applied to harmonize the colors and to enable the assignment of rays to objects by means of their color. - De-vignetting: Majorly in plenoptic cameras, but to a smaller extent also in arrays lenses are not adequately effective in avoiding vignetting effects. Consequently, the brightness of rays needs to be harmonized. - Rectification: To establish the correct relationship between any captured sensor pixel and the respective ray in space, camera positions need to be known perfectly, and lens distortions need to be corrected. The processing step is called rectification, since in many applications (μ-lenses on the MLA in plenoptic cameras, horizontal and vertical alignment in planar camera arrays) the underlying position ideally reflects an equidistantly sampled rectangle. After rectification, all (virtual) extrinsics and intrinsics are known.