Combining Traditional Methods with Deep Learning for Accurate Single-Frame Fringe Pattern Analysis

Optical metrology, a versatile technique harnessing light as an information conduit for contactless and non-destructive measurements, plays a pivotal role in manufacturing, basic research, and engineering applications. Thanks to the advent of lasers and charge-coupled devices (CCDs), contemporary manufacturing, precision positioning, and quality assessment processes increasingly rely on optical metrology methods. These methods offer advantages such as exceptional accuracy, sensitivity, repeatability, and speed.

In numerous optical metrology techniques like interferometry, digital holography, and fringe projection profilometry (FPP), the primary research focus centers on analyzing fringe patterns to recover the underlying phase distribution. The precision and efficiency of phase retrieval from fringe patterns are critical for dynamically reconstructing various physical properties of objects, including profiles, distances, and strain.

For structured light 3D imaging, particularly FPP, reducing the number of fringes required for a single reconstruction has been a key pursuit. Researchers at Nanjing University of Science and Technology, led by Prof. Qian Chen and Chao Zuo, established a theoretical framework for phase shifting profilometry and temporal phase unwrapping. They developed composite phase shifting methods, such as bi-frequency phase shifting, 2+2 phase shifting, geometric constraints-based composite phase shifting, and micro Fourier transform profilometry (μFTP). These methods have significantly reduced the required fringe patterns per 3D reconstruction, achieving high-speed 3D sensing at up to 10,000 frames per second.

However, achieving high-accuracy 3D reconstruction using only a single pattern has remained the ultimate goal in structured light 3D imaging. The challenge lies in separating high-frequency fringe information from the object surface, limiting the technique’s applicability to smooth surfaces with minimal height variations.

Recent advances in deep learning, a data-driven machine learning technique, have revolutionized various fields, including computer vision and computational imaging. Deep learning has permeated optical metrology, offering solutions to complex problems like fringe denoising, fringe analysis, and digital holographic reconstruction.

However, unlike traditional fringe analysis methods, deep learning approaches primarily involve training deep neural networks (DNNs) to recognize image-to-image transformations using massive input and output data pairs. These methods have not fully leveraged the physical principles governing image formation or domain-specific knowledge related to measurements.

Consequently, the effectiveness of deep learning approaches in solving intricate physical problems heavily depends on the statistical characteristics of the dataset. To push the boundaries of fringe pattern analysis in terms of speed, accuracy, repeatability, and generalization, there’s a growing trend towards combining physics-based traditional methods with data-driven learning approaches.

In a recent publication in Opto-Electronic Advances, Prof. Qian Chen and Prof. Chao Zuo’s research group at Nanjing University of Science and Technology introduced a physics-informed deep learning method for fringe pattern analysis (PI-FPA). This method integrates a lightweight DNN with a learning-enhanced Fourier transform profilometry (LeFTP) module, enhancing single-shot phase retrieval accuracy while maintaining computational efficiency. Dynamic 360-degree 3D reconstruction results showcase its superiority over traditional methods.

Compared to universal end-to-end image transform networks (U-Net and its derivatives), the lightweight network in PI-FPA refines the initial phase with lower computational costs. PI-FPA excels in high-quality and efficient 3D modeling of complex structures, even with materials not extensively represented in the training dataset, like metal.

The proposed PI-FPA method not only captures inherent statistical dataset characteristics, like traditional neural networks but also incorporates knowledge of the physical laws governing image formation. This enables precise single-frame phase reconstruction with remarkable computational efficiency and generalization capabilities, even for rare samples unseen during training.

Future research by the team will explore the phase recovery performance of PI-FPA for various fringe image types and investigate its applications in interferometry and digital holography within optical metrology. These endeavors aim to further advance fringe pattern analysis in terms of speed, accuracy, repeatability, and generalization.

Leave a Comment