General Cramér-von Mises, a Helpful Ally for Transparent Object Inspection Using Deflection Maps?

Abstract

Transparent materials are utilized in different products and have to meet high quality requirements, i.a., they have to be free from scattering defects. Such material defects are mainly manifested in changes of the direction of light transmitted through the object. Laser deflection scanners can acquire so-called four-dimensional light deflection maps conveying both, the spatial and angular information about captured light rays. In order to detect scattering defects, spatial discontinuities of the angular deflection distribution have to be extracted out of the deflection maps. This is necessary since the transparent object itself and possibly present scattering defects can deflect incident light rays into other directions. This contribution introduces a novel distance measure based on the generalized Cramér-von Mises distance that is suitable for comparing spatially adjacent deflection maps. The approach is evaluated by conducting experiments using both, simulated data and existing deflection maps acquired with a prototype of a deflection scanner. The results show, that the method is not as sensitive as the recently proposed earth mover’s distance but might be able to yield spatially more accurate visualizations of scattering material defects.

1 Introduction

For humans’ every day life, objects made out of transparent materials are of utter importance. Such objects are important components in diverse products and applications and have to meet high quality requirements. For example, the windshields of automobiles and aircrafts have to protect the driver or pilot from the surroundings but also have to enable clear sight. Spectacle glasses or contact lenses are another example for which high requirements apply—they have to be free from even the smallest inhomogeneities. Apparently, the employed materials are required to have no scratches, cracks or enclosed air bubbles that might impair the mechanical stability. Furthermore, there should be no opaque contaminants in or on the windshield, respectively, the glasses, so that the user has clear sight.

Considering these two examples it already gets clear, that there has to be some kind of visual inspection of transparent materials after their production. On the one hand, workers can be employed for performing the inspection. However, since this is a fatiguing task for humans, it is prone to missing defects, what could have disastrous consequences. On the other hand, automated visual inspection systems can carry out the quality control.

Existing machine vision systems based on cameras or laser scanners are able to reliably inspect certain kinds of transparent test objects for being free from defects [1, 8]. Opaque contaminants, which mainly absorb incident light, can usually be found by looking for regions of the test object where the exiting light has lower intensities [4]. In contrast, scattering defects (e.g., enclosed air bubbles) are harder to detect since they mainly cause incident light rays to be deflected into multiple directions [2, 5]. Therefore, scattering defects are manifested in changes of the distribution of the light’s propagation direction.

Indeed, the camera-based systems have a limited depth of field that prevents them from being used to inspect large objects like windshields. Although laser scanner-based systems usually have a larger depth of field, they cannot be used to inspect test objects having a strongly curved surface (e.g., large lenses or cover glasses of automotive headlamps) since they require the emitted laser beam to propagate approximately straightly through the measurement field.

Recently, laser deflection scanners were introduced which are able to capture the angles by which laser beams are deflected when propagating through a transparent test object. Since the acquired so-called deflection maps also contain information about the direction of the observed light, they can be used for finding scattering defects. By means of adequately processing the deflection maps, it is possible to image scattering defects even for test objects having a strongly curved 3D shape. However, the necessary image processing methods are still in an early stage and further approaches have to be evaluated.

This article proposes a novel approach for processing acquired light deflection maps in order to visualize scattering material defects. The introduced method is based on localized cumulative distributions and a generalized formulation of the Cramér-von Mises distance. In Sect. 2, the most relevant work performed by other researchers is presented and summarized. Section 3 explains the mentioned Cramér-von Mises distance in detail and introduces the proposed novel processing approach. The performed experiments are covered and discussed in Sect. 4. Section 5 closes the paper with a short summary and some ideas concerning further research.

This contribution can be seen as an extension of [7, 9]. The previously introduced processing methods are extended by a novel approach using the generalized Cramér-von Mises distance. Besides, synthetic experiments are performed via an appropriately adapted rendering framework.

2 Related Work

Light deflection maps first got mentioned in [11]. Here, the authors introduce a novel optical system which they call a Schlieren deflectometer. They illuminate transparent test objects with parallel light beams whose orientation is successively tilted with respect to the optical axis. For every tilt angle of the parallel light beams, they observe the backside of the test object with a camera system. By using a telecentric lens, they only capture those rays that propagate approximately parallel to the optical axis. As for every acquired image the tilt angle of the illumination is known, the method allows to determine the deflection angles for all rays exiting the test object. Since rays can also be scattered or spread inside the test object, not only one angle is acquired per pixel but a two-dimensional distribution which is called a deflection map. The authors did not propose to further evaluate the acquired deflection maps in order to check the test object for defects or impurities.

In [9], Meyer et al. introduce a novel optical inspection setup consisting of a collimated illumination source and a so-called 4f light field camera. This camera allows to capture spatially resolved measurements of the deflection angles of light rays propagating through the measurement field. By this means, they can acquire deflection maps of transparent test objects. In order to image scattering defects present in the test objects, the authors propose two methods for calculating image gradients based on the earth mover’s distance and on vector fields of the mean local deflection direction. These two approaches allow to visualize spatial discontinuities of the deflection direction distribution. The authors positively evaluated their method using simulated data—however, the achievable spatial resolution is limited by the trade-off between angular and spatial resolution which is characteristic for optical setups based on light field cameras.

The first prototype of a laser scanner setup capable of acquiring deflection maps has been described in [7]. Figure 1 shows the principle optical setup of the deflection laser scanner proposed by the authors. Sequentially, parallel laser beams are emitted into the measurement field. The receiving part of the system consists of a convex lens or a parabolic mirror having a two-dimensional detector array in its focal plane. By this means, the angular information about the propagation direction of the light rays is transformed into a spatial information. Any light ray that is deflected inside the measurement field by an angle $\alpha$ will reach the detector array with a spatial displacement of $\delta = \tan (\alpha )f$ with respect to the intersection of the optical axis and the detector plane. So for every emitted laser beam a deflection map is acquired. The authors successfully evaluated their approach by setting up an early prototype of such a laser deflection scanner and by employing the processing methods introduced in [9] to clearly visualize scattering defects on two different transparent test objects.

The group of Hanebeck et al. introduced a new type of characterizing multivariate probability distributions, the so-called localized cumulative distribution (LCD) [3]. The LCD can be summarized as a cumulative rectangular kernel transform of the respective probability density function. The authors further introduce a generalized form of the Cramér-von Mises distance (CVM-distance) that is based on the LCDs of two probability density functions and also suitable for the multivariate case. In probability theory, this generalized CVM-distance can be employed, e.g., to measure the similarity between two probability distributions, of which one is of a discrete type. In Sect. 3, this paper shows how the CVM-distance can be employed to analyze deflection maps in order to find spatial discontinuities of the angular deflection distribution of captured light rays.

Fig. 1.

Principle optical setup of a laser deflection scanner: the main components are an emitter sequentially emitting parallel laser beams and the receiving part consisting of a lens or a parabolic mirror with focal length f having a two-dimensional detector array in its focal plane. If a beam is deflected by an angle $\alpha$ inside the measurement field and still reaches the lens of the receiver, it will hit the detector array with a spatial displacement of $\delta = \tan (\alpha )f$ with respect to the intersection of the optical axis and the detector plane.

3 Deflection Map Processing Using the Generalized Cramér-von Mises Distance

A laser deflection scanner as proposed in [7] captures light intensities, that can be represented as a four-dimensional function $a(\mathbf {m},\mathbf {j})$, with $\mathbf {m}=(m_1,m_2)^\mathrm {T}$ corresponding to the respective spatial position of the laser source and $\mathbf {j}=(j_1,j_2)^\mathrm {T}$ denoting the position on the detector array, i.e., the angular deflection component. It has to be noted that $\mathbf {m},\mathbf {j}$ represent discretized quantities and that—especially for the angular coordinates $(j_1,j_2)^\mathrm {T}$—the discretization involves nonlinear operations, e.g., the calculation of the spatial displacement $\delta = f \tan (\alpha )$ corresponding to the respective deflection angle $\alpha$. Figure 2 visualizes the concept of deflection maps.

Fig. 2.

Concept of deflection maps: for every sampling position $\mathbf {m}$ of the laser scanner, the two-dimensional distribution of the captured deflection angles is represented by the deflection map $a(\mathbf {m},\mathbf {j})$, with $\mathbf {j}$ denoting the coordinates of the angular component.

According to [7, 9], scattering defects present in a transparent test object can be visualized by detecting spatial discontinuities of the deflection maps $a(\mathbf {m},\mathbf {j})$. Therefore, a gradient $\varDelta _ \mathbf {m} a(\mathbf {m},\mathbf {j})$ is defined on $a(\mathbf {m},\mathbf {j})$:

$$\begin{aligned} \varDelta _ \mathbf {m} a(\mathbf {m},\mathbf {j}) = \begin{pmatrix} d\left( a((m_1-1,m_2)^\mathrm {T},\mathbf {j}), a((m_1+1,m_2)^\mathrm {T},\mathbf {j}) \right) \\ d\left( a((m_1,m_2-1)^\mathrm {T},\mathbf {j}), a((m_1,m_2+1)^\mathrm {T},\mathbf {j}) \right) \end{pmatrix} , \end{aligned}$$

(1)

with $d(\cdot ,\cdot )$ denoting a distance function suitable for calculating the distance between two deflection maps. The gradient $\varDelta _ \mathbf {m} a(\mathbf {m},\mathbf {j})$ can be used for visualizing material defects or it can be used as a feature for an automated classification. As mentioned in the introduction, a generalized version of the Cramér-von Mises distance will be used for $d(\cdot ,\cdot )$ in this paper. Therefore, the localized cumulative distribution has to be introduced first.

3.1 Localized Cumulative Distribution

For a random vector $\mathbf {\tilde{x}} \in \mathbb {R}^N, N \in \mathbb {N}$ that is characterized by an N-dimensional probability density function $f{:}\,\mathbb {R}^N \rightarrow \mathbb {R}_+$ there is no unique cumulative distribution function for $N>1$. As mentioned before, [3] propose a localized cumulative distribution (LCD) that is well-defined for the multivariate case. The LCD $F(\mathbf {x},\mathbf {b})$ of f is defined as

$$\begin{aligned} F(\mathbf {x},\mathbf {b}):= P\left( \vert \mathbf {\tilde{x}} - \mathbf {x} \vert \le \frac{1}{2} \mathbf {b}\right) , \end{aligned}$$

(2)

$$\begin{aligned} F(\cdot ,\cdot ):\varOmega \rightarrow \left[ 0,1 \right] , \varOmega \subset \mathbb {R}^N_+ \times \mathbb {R}^N_+, \end{aligned}$$

(3)

with $\mathbf {b}\in \mathbb {R}^N_+$ denoting the sizes of the integration kernel in the individual dimensions and the relation $\mathbf {x} \le \mathbf {y}, \mathbf {x},\mathbf {y} \in \mathbb {R}^N_+$ only holding if $\forall i \in \left[ 1,\ldots ,N\right] :x_i \le y_i$. From a probability density function $f(\mathbf {x})$ the respective LCD $F(\mathbf {x},\mathbf {b})$ is calculated by

$$\begin{aligned} F(\mathbf {x},\mathbf {b}) = \int \limits ^{\mathbf {x}+\frac{1}{2}\mathbf {b}}_{\mathbf {x}-\frac{1}{2}\mathbf {b}} f(\mathbf {t})d\mathbf {t}. \end{aligned}$$

(4)

In order to apply this formulation to the deflection maps $a(\mathbf {m},\mathbf {j})$, they first have to be normalized so that their values are in the range of $\left[ 0,1\right]$ and their discrete nature has to be taken into account. Therefore, normalized pendants $\tilde{a}(\mathbf {m},\mathbf {j})$ are defined as

$$\begin{aligned} \tilde{a}(\mathbf {m},\mathbf {j}) = \frac{a(\mathbf {m},\mathbf {j})}{\sum \limits _{(k,l)^\mathrm {T}\in \varOmega _\mathrm {j}}a(\mathbf {m},(k,l)^\mathrm {T})}, \end{aligned}$$

(5)

with $\varOmega _j$ denoting the angular support of the deflection maps, i.e., the set of valid coordinates with respect to the detector array of the employed laser deflection scanner. By means of Eq. (5), the formulation (4) of the LCD can be adapted, so that the LCD of $a(\mathbf {m},\mathbf {j})$ is given by

$$\begin{aligned} A(\mathbf {m},\mathbf {j},\mathbf {b}) = \sum \limits ^{\min \lbrace \mathbf {j}_\mathrm {max},\mathbf {j} + \mathbf {b} \rbrace } _{\mathbf {i}=\max \lbrace \mathbf {1}, \mathbf {j}-\mathbf {b} \rbrace } \tilde{a}(\mathbf {m},\mathbf {i}), \end{aligned}$$

(6)

with $\mathbf {j}_ \mathrm {max}$ denoting the maximum deflection captured by the system and $\max \lbrace \cdot \rbrace$ and $\min \lbrace \cdot \rbrace$ representing component wise vector operations. Based on these formulations, the generalized Cramér-von Mises distance can now be introduced.

3.2 Generalized Cramér-von Mises Distance

The generalized CVM-distance is a distance measure between two multivariate probability distributions based on their LCDs. Given the LCDs $F(\mathbf {x},\mathbf {b})$ and $G(\mathbf {x},\mathbf {b})$ of two probability density functions $f(\mathbf {x})$ and $g(\mathbf {x})$, their generalized CVM-distance is given by

$$\begin{aligned} D(f,g) := \int _\mathbb {R^N}\int _{\mathbb {R}^N_+} \left( F(\mathbf {x},\mathbf {b}) - G(\mathbf {x},\mathbf {b}) \right) ^2 d \mathbf {b} d \mathbf {x}. \end{aligned}$$

(7)

Equation (7) can now be employed in concert with (6)—the adapted formulation of the LCD for the deflection maps—to obtain a formulation of the generalized CVM-distance $d_ \mathrm {CVM} (a_1,a_2)$ that can be used as a distance measure in (1) between two deflection maps $a_1$ and $a_2$:

$$\begin{aligned} d_ \mathrm {CVM} (a_1,a_2) = \sum \limits _{\mathbf {j}\in \varOmega _\mathrm {j}}\sum \limits _{\mathbf {b}\in \varOmega _ \mathrm {j}} \left( A_1(\mathbf {m},\mathbf {j},\mathbf {b}) - A_2(\mathbf {m},\mathbf {j},\mathbf {b}) \right) ^2. \end{aligned}$$

(8)

Furthermore, Eq. (8) can be extended by weights $w(\mathbf {b})$ corresponding to the individual kernel sizes in $\mathbf {b}$:

$$\begin{aligned} d_ \mathrm {CVM} (a_1,a_2) = \sum \limits _{\mathbf {j}\in \varOmega _\mathrm {j}}\sum \limits _{\mathbf {b}\in \varOmega _ \mathrm {j}} w(\mathbf {b}) \left( A_1(\mathbf {m},\mathbf {j},\mathbf {b}) - A_2(\mathbf {m},\mathbf {j},\mathbf {b}) \right) ^2, \end{aligned}$$

(9)

with

$$\begin{aligned} \sum \limits _{\mathbf {b}\in \varOmega _ \mathrm {j}} w(\mathbf {b}) = 1. \end{aligned}$$

(10)

4 Evaluation

Simulated and empirically obtained data has been used to obtain a first evaluation of the proposed approach. By means of the physically based rendering framework Mitsuba [6], sensor readings of a laser deflection scanner as mentioned in Sect. 2 have been simulated. Further evaluations were based on the sensor readings from [7], that have been acquired using an early prototype of a laser deflection scanner. For the interpretation of the results described in the following sections it has to be taken into account that only a limited amount of data had been available when conducting the experiments.

4.1 Simulated Experiments

For performing the simulations, the Mitsuba framework has been extended by several components and plugins¹. A sensor plugin has been introduced that models the receiving part of the described laser deflection scanner (see Fig. 1). Further modifications were needed in order to obtain the time sequential sensor readings for the different spatial positions of the laser source. For this purpose, a collimated light source has been realized, that is attached to a rectangular shape with a virtual 2D-grid representing the discrete positions $(m_1,m_2)^\mathrm {T}$ of the emitted light beams. Additionally, the rendering’s ray tracing engine has been adequately adapted. Whenever one of the sensors’ ray of sights is traced to the introduced light source, the respective 2D-position on the mentioned grid is combined with the 2D-coordinate $(j_1,j_2)^\mathrm {T}$ of the sampled sensor element and added to the ray data structure. Together, these two 2D-vectors finally allow to assemble the two-dimensional array $a(\mathbf {m},\mathbf {j})$ of 2D-deflection maps.

In the simulated experiments, a double-convex lens suits as the test object. In order to simulate scattering material defects (enclosed air bubbles), small spheres of different sizes with an index of refraction of $n=1$ have been placed inside the test objects. The introduced CVM-distance is compared against an existing approach, the so-called earth mover’s (EM) distance [7, 9, 10], for processing deflection maps in order to visualize material defects.

The simulated laser deflection scanner setup had a spatial resolution of $300\times 300$, an angular resolution of $15\times 15$ and a pitch between the laser sampling positions of ${334}\, \upmu \mathrm{m}$. This choice of the parameter values is motivated by the parameters of the empirical experiments. Small spheres with an index of refraction of $n=1$ (air) have been inserted into the test object in order to simulate scattering material defects. Sensor readings have been simulated for a defect-free test object instance and for test objects with enclosed air spheres with radii $r\in \lbrace {83}\, \upmu \mathrm{m}, {167} \,\upmu \mathrm{m}, {334}\, \upmu \mathrm{m} \rbrace$. The radii of the scattering defects were chosen in order to evaluate the ability of the investigated methods to visualize defects of sizes around the sampling pitch of the laser scanner. Of every defect size, three defect instances have been inserted into the test object, one in the lower left corner, one in the center and one in the upper right corner.

The results calculated out of the simulated sensor readings using the earth mover’s distance and the generalized CVM-distance are shown in Fig. 3 as pseudo color images. The grid-like structures present in all images is a result of the discrete nature and the limited resolution of the detector array and of the simulation. Whenever deflection angles change so that the laser spot reaches another sensor element on the detector array there is a spatial discontinuity in the deflection map at the respective position. Furthermore, the rendering framework simulates infinitely thin rays and does not take divergence or other effects into account. Hence, for sensor readings from a real physical system, such artifacts would not occur.

The smallest defects ($r = {83}\, \upmu \mathrm{m}$) are only visualized by the EM-distance but with low contrast. The defects with radius of $r = {167}\, \upmu \mathrm{m}$ are visible in the images obtained using the EM-distance and the CVM-distance. The largest simulated defects are clearly visualized by both methods but the image resulting from the generalized CVM-distance shows the defects slightly clearer.

Fig. 3.

Pseudo color images showing the results of the performed experiments. A double-convex lens has been used as the test object. The radii of the simulated scattering defects are denoted by r. Bright, respectively, dark regions correspond to low, respectively, high distance values.

4.2 Empirical Experiments

Further experiments were performed using empirical data from [7]. The sensor readings have been obtained using a prototype of a laser deflection scanner system with a spatial resolution of $39\times 39$, an angular resolution of $9\times 9$ and a pitch between the laser sampling positions of approximately ${334}\, \upmu \mathrm{m}$. A cylindrical convex lens served as the test object. There are sensor readings for a defect-free test object and for one affected by two scattering surface defects.

Figure 4 shows the results calculated using the EM-distance and the generalized CVM-distance as pseudo color images. As for the simulated sensor readings, the images obtained using the generalized CVM-distance have a clearer appearance due to their higher contrast compared to the images resulting from the EM-distance. Still, in both images the defect-free test object instance can clearly be recognized as being free from defects and the two defects present on the second test object instance are clearly visualized by both approaches.

For a quantitative evaluation, the signal-to-noise ratio SNR and the peak signal-to-noise ratio PSNR can be taken into account. The SNR is defined by

$$\begin{aligned} SNR:=\frac{\mu _{\mathrm {defect}}}{\sigma _{\mathrm {defect-free}}} , \end{aligned}$$

(11)

with $\mu _{\mathrm {defect}}$ denoting the mean signal value for pixels belonging to defective test object regions and $\sigma _{\mathrm {defect-free}}$ representing the standard deviation of pixels belonging to defect-free regions of the test object. The PSNR is calculated via

$$\begin{aligned} PSNR:=\frac{\hat{s}_{\mathrm {defect}}}{\sigma _{\mathrm {defect-free}}}, \end{aligned}$$

(12)

with $\hat{s}_{\mathrm {defect}}$ denoting the maximum signal intensity among the pixels corresponding to the defective test object region. Table 1 shows the SNR and PSNR values resulting from the performed empirical experiments. The CVM-distance yields higher SNR and PSNR values than the EM-distance but indeed, more experiments are needed to support these results.

Table 1. Comparison of the signal-to-noise ratio (SNR) and the peak signal-to-noise ratio (PSNR) for the two processing approaches.

In the image obtained with the generalized CVM-distance, it might be possible to infer the size of the defect more accurately. The defects’ actual sizes were manually measured and are approximately ${2.8}\, \mathrm{mm}\times {2.8} \, \mathrm{mm}$ for the upper left defect and ${4} \, \mathrm{mm}\times {4} \, \mathrm{mm}$ for the lower right defect. Table 2 shows rough estimations of the defects’ sizes that can be obtained by counting the pixels clearly belonging to the defective regions. These results show that the EM-distance slightly tends to render the defects larger than they actually are—however more experiments are needed to support these findings.

Table 2. Comparison of the manually measured defect sizes and the rough estimations obtained by counting the pixels clearly belonging to the defective regions in the images resulting from the two processing approaches.

Fig. 4.

Pseudo color images showing the results of the performed experiments based on empirical data. Bright, respectively, dark regions correspond to low, respectively, high distance values.

5 Summary and Outlook

This contribution proposed a novel method for processing light deflection maps in order to visualize scattering material defects present in transparent objects. Since these defects are mainly manifested in changes of the direction of transmitted light, spatial discontinuities in captured light deflection maps have to be found. Therefore, adequate distance methods between spatially adjacent deflection maps are needed. In this paper, the recently introduced concept of localized cumulative distributions and the generalized formulation of the Cramér-von Mises distance have been adapted so that they are applicable for comparing deflection maps. The proposed approach has been successfully validated by means of experiments based on both, simulated sensor readings and existing empirical data acquired using a prototype of a deflection laser scanner. The experiments showed that the introduced method is able to accurately visualize scattering defects. However, the earth mover’s distance seems to be slightly superior to the CVM-distance regarding the visualization of smallest defects.

As further steps, the authors plan to conduct additional experiments based on an improved laser deflection scanner prototype in order to quantitatively evaluate and compare the capabilities and limits of different adequate distance measures. Regarding the optical setup, the dependence of the smallest detectable defect size on the system parameters (focal length, array size and resolution) will be investigated. Besides the suitability for visualizing material defects, other properties of the investigated methods, e.g., their computational complexity and processing time will be studied.

Furthermore, the authors plan to evaluate more physically motivated approaches for processing the deflection maps. On the one hand, small material defects could cause diffraction effects that lead to diffraction patterns on the sensor which can be used to infer the defects’ sizes. On the other hand, also scattering theory, e.g., Mie scattering could be used to obtain more information about defects based on the characteristics of the observed intensity patterns.