ST-SAIL: Spatio-Temporal Semantic Analysis of Light Fields: Optimizing the Sampling Pattern of Light Field Arrays

Abstract:

We introduce a novel technique to quantify the information content in light field videos and demonstrate that the traditional synchronized sampling pattern can be optimized by introducing sub-frames. As light field arrays record the same scenes from multiple viewpoints, a lot of redundant data is present and only non-Lambertian objects truly benefit from the increased spatial sampling. Conversely, dynamic regions would benefit from a finer temporal resolution. Through sub-framing, we stagger camera groups temporally, exchanging spatial resolution for enhanced temporal granularity, thereby increasing the total captured information content.